Add a mechanism to Forth that allows code sections/chapters to be loaded from a large 'file' with a simple command. How the file is stored and what it contains is implementation-dependent. What matters is the interface. The sections or chapters are accessible through one or more keywords.
The interface consists of the following word: NEED
NEED
( "name" -- i*x )
Make sure “name” is present in the dictionary. If not, load it from a library. If “name” already exists, do nothing.An optional word as suggested by Ulrich Hoffmann
FROM
( "name" -- i*x )
Open the library “name” for use with NEED
Optional words as suggested by Willem Ouwerkerk
NEEDED
( a b -- i*x )
Make sure that name represented by the string a b is present in the dictionary. If not, load it from a library. If name already exists, do nothing..LIB
( -- )
Show all section/chapter headers of the current library in one or more columnsFunction: NEEDED ( a +n -- i*x ) \ a +n is keyword a) Check if the word represented by the string a +n exists in the current search order. b) Do nothing the string was found, otherwise start searching the active library for a keyword that matches this string. c) When the keyword is found load the source code chapter otherwise issue an error message Function: NEED ( "name" -- i*x ) Perform the function of NEEDED using the parsed "name". Function: .LIB ( -- ) Print all chapter headers in a human readable form. Function: FROM ( "name" -- ) Make the library "name" the current active library When "name" was not found issue an error message
Here, I will elaborate on an example that stores the source in Flash memory. For this implementation, there are the following design requirements:
- Simple structure ( idea A.N. ) - Quick search function - Extensible: OPEN-LIB CHAPTER CLOSE-LIB WIPE-LIB - Easy to use: NEED etc.
This library mechanism is based on two control characters 09
and 0D
.
They act as separators and are not part of the text itself.
The 09
marks the end beginning of a chapter, like this:
Note that: The library starts with a dummy 09
so that the first keyword line can be found.
09
chapter-1 09
chapter-2 09
last-chapter 09
The 0D
marks the end of a line:
0D
line-2 0D
last-line 0D
The first line of each library chapter starts with case insensitive keyword(s) separated by spaces. The word CHAPTER
builds this the first line, and provides this line with a
backslash to prevent the keywords for being executed while loading a chapter. Then it adds all source code until the %%
marker is found. The end of each chapter is marked by 09
The end of the library is kept in a pointer that is updated when we extend the library.
chapter 2TUCK \ Copy top double below second : 2TUCK 2swap 2over ; ( x1 x2 x3 x4 -- x3 x4 x1 x2 x3 x4 ) %% chapter 2ROT \ Rotate third double to top of stack : 2ROT 2>r 2swap 2r> 2swap ; %% chapter -2ROT \ Rotate top double to third double position on the stack : -2ROT 2swap 2>r 2swap 2r> ; %%
The above library chapters leave the following ASCII structure in Flash memory.
When after the <09> there is no more text, it's the end of the library.
\ 2TUCK<0D> \ Copy top double below second<0D> : 2TUCK 2swap 2over ; ( x1 x2 x3 x4 -- x3 x4 x1 x2 x3 x4 )<0D> <09> \ 2ROT<0D> \ Rotate third double to top of stack<0D> : 2ROT 2>r 2swap 2r> 2swap ;<0D> <09> \ -2ROT<0D> \ Rotate top double to third double position on the stack<0D> : -2ROT 2swap 2>r 2swap 2r> ;<0D> <09>
Function: NEEDED ( a +n -- i*x ) a) Save the string a +n and make the string uppercase b) If this string is already present in the dictionary, do nothing c) Start searching the library for this string d) When a keyword is found load the source code section until 09 is found e) When a keyword is not found issue an error message Function: NEED ( "name" -- i*x ) a) Parse the next word form the input stream and perform the function of NEEDED Function: .LIB ( -- ) a) Scan the library for a 09 b) When it's not the last, print the found library chapter/section header
Function: OPEN-LIB ( -- ) Open the library by reading the last incomplete file sector to the sector buffer. Adapt the library write administration in such a way that new source code will be added behind whats read from Flash. When the library is empty, lay down the start of a section header with a 0D & 09 character. Function: CLOSE-LIB ( -- ) Close the library, writing the last added code (incomplete sector) Update the end of library pointer Function: WIPE-LIB ( -- ) Erase a stored library completely, restore the end of library pointer Function: CHAPTER ( -- ) Add a new library section, first add the section header. Note the given keywords in uppercase, sepatated by spaces and ended with a 0D. Read and store the Forth source code behind it until the end of source marker %% was found
This sample is taken from the noForth t library mechanism. Here the library is stored in the remaining flash that is not needed by the noForth binaries. For readability some noForth specific words are replaced.
create KEYWORD 20 allot \ Hold keyword 1008,1000 constant LIB \ Library start address LIB value LIBHERE \ Current end address of library : LIB-FIND ( a +n –- sa ) \ a,+n=name, sa=begin-source-section keyword place \ Save keyword keyword count upper \ Convert to uppercase libhere lib \ Library address range begin 2dup > 0= throw 2dup \ Nothing found? 0D scan nip over 1+ \ Get line with keywords, skip \ begin 2dup > while \ Line not done? bl skip dup >r \ Skip leading spaces bl scan \ After keyword r@ over r> - \ Keyword length keyword count s<> 0= \ Keyword found, leave source address if 2drop nip exit then repeat 2drop 09 scan 1+ \ Find library section, skip 09 again ; : LIB-REFILL ( source-id –- f ) to ib false \ Start of new line ib c@ 09 = ?exit \ End of source section, ready ib FF + ib 0D scan \ Find 0D (end of current line) dup ib - to #ib \ Calc. & save line length 1+ to source-id drop \ Save start of next line (behind 0D) >in ! true ; \ Refill succeeded : LIB-LOAD ( a –- i*x ) ?dup 0= ?exit \ Do nothing when zero @input >r 2>r \ Save current input source to source-id \ Set new input source #ib >in ! interpret \ Force a REFILL when loading source section 2r> r> !input ; \ ib #ib,>in@ source-id restore input source : NEEDED ( a +n –- i*x ) ['] lib-refill to 'refill \ Add library REFILL LIB-FIND keyword find nip 0= \ in the current forth search order and lib-load ; : NEED ( a +n –- i*x ) bl word count needed ; : .LIB ( –- ) \ Show library contents by printing keyword lines libhere lib \ Lib. range begin cr 20 for 1+ 1+ 2dup > 0= if \ Skip backslash, not past end of file? 2drop rdrop exit \ Ready then dup >r 0D scan \ Get end of keyword line r@ over r> - type \ Show keyword line 28 hor - 0 max spaces \ In fixed column size hor 28 > if cr then 09 scan 1+ \ Find section marker next key bl <> until 2drop ;
Note that: This sample code uses calls to the built-in ROM API functions.
It uses them for reading, writing and erasing flash memory.
10000000 constant XIP \ Start of XIP memory create BUFFER 180 allot \ Sector buffer with overflow (noForth t) 0 value PTR \ Buffer index : LC, buffer ptr + c! incr ptr ; ( b -- ) \ Compile a byte in the library buffer : LM, bounds ?do i c@ lc, loop ; ( a +n -- ) \ Compile the string a +n in the library buffer : LIBWRITE ( +n -- ) \ Write library sector libhere xip - buffer 100 write-flash \ Write lib. sector to flash dup +to libhere negate +to ptr \ To next lib. block & correct pointer buffer 100 FF fill \ Erase first buffer buffer 100 + buffer ptr move ; \ Move overflow to sector buffer : BUFFER-FULL ( -- ) \ Buffer overflow, write & restore ptr FF > if 100 libwrite then ; : ADD-LIB ( text -- ) \ Add library section, save it when a buffer is full begin buffer-full \ Buffer overflow, write & restore refill drop \ Read new line ib 2 s" %%" s<> while \ No (%%) delimiter? ib #ib lm, 0D lc, \ Store line repeat refill drop ; \ Read next line : (-BL) ( -- ) parea bl skip nip ib - >in ! ; \ Skip leading spaces in IB : CHAPTER ( text -- ) \ Contruct new library section s" \ " lm, \ New lib. header begin (-bl) bl parse \ Find keywords ?dup while 2dup upper lm, bl lc, \ Store uppercase in buffer repeat drop 0D lc, \ Add formfeed when done add-lib 09 lc, ; : OPEN-LIB ( -- ) \ Open a lib. for writing, read incomplete lib. sector too! libhere lib <> if \ Lib. not empty? libhere 100 /mod 100 * \ Get previous sector & ptr length dup to libhere \ Correct lib. pointer & calc. XIP address buffer 100 move to ptr {w \ Read uncomplete sector to buffer, open flash then 0 to ptr {W ; \ First lib. entry, open flash : CLOSE-LIB ( -- ) \ Close library, save unfinshed buffer too buffer-full ptr libwrite W} ; \ Write last sector if any & close flash : WIPE-LIB ( -- ) \ Remove previous stored library from flash lib xip - \ Convert to start sector address libhere lib - FF000 and \ Convert to erase sector length, max. 1 Mbyte 1000 + {W wipe-flash W} \ Erase one sector extra lib to libhere ; \ And reset library pointer
This examples shows how many different code parts can be used inside the library.
open-lib chapter version versie vsn cr .( NEED version 0.32 ) cr .( Library version 0.22 ) cr .( Size is about 174 kBytes ) %% chapter RESTORE-LIB v: inside \ Restore LIBHERE in case it got lost libhere hx 40000 + lib hx FF scan nip to libhere cr .( Libhere ) libhere u. cr .( Size ) libhere lib - dm . .( bytes ) freeze v: fresh %% chapter VIEW ( A library consists of 09 {tab}, 0D {cr} and ASCII chars ) ( ranging from blank to ~ others chars are not allowed ) v: inside also definitions \ Show the library source code of a chapter : .LINE ( a1 -- a2 ch ) begin c@+ dup bl < 0= while emit repeat ; : .DIVIDER ( -- ) cr 10 for ." {} " next cr ; : LIBTYPE ( a -- ) begin 10 0 do \ Max. 16 lines at a time cr .line 09 = \ End of source chapter? if .divider leave then \ Show divider & ready loop dup libhere < while \ More library source to be done? key bl <> if drop exit then \ Yes, ask key, stop on non space repeat drop ; v: extra definitions : VIEW ( ccc -- ) bl-word count lib-find libtype ; v: fresh %% chapter PIN SQUERY ( GPIO -- ) \ Change GPIO pin for S? need [IF] dup dm 30 2 within [if] drop dm 24 [then] \ Invalid switch pin? 1 cfg 1+ c! \ GPIO-xx for S? 4 cfg @ abs 4 cfg ! \ Make sure to (re)start the second image config cr .( Test S? ) s? . %% chapter 12MHZ \ change clock frequency dm 12 0 cfg ! \ Set frequency in MHz 4 cfg @ abs 4 cfg ! \ Make sure to (re)start the second image config \ Test new configuration %% close-lib
When displayed the library chapters look like this:
\ VERSION VERSIE VSN cr .( NEED version 0.33 ) cr .( Library version 0.22 ) cr .( Size is about 174 kBytes ) \ RESTORE-LIB v: inside \ Restore LIBHERE in case it got lost libhere hx 40000 + lib hx FF scan nip to libhere cr .( Libhere ) libhere u. cr .( Size ) libhere lib - dm . .( bytes ) freeze v: fresh \ VIEW ( A library consists of 09 {tab}, 0D {cr} and ASCII chars ) ( ranging from blank to ~ others chars are not allowed ) v: inside also definitions \ Show the library source code of a section : .LINE ( a1 -- a2 ch ) begin c@+ dup bl < 0= while emit repeat ; : .DIVIDER ( -- ) cr 10 for ." {} " next cr ; : LIBTYPE ( a -- ) begin 10 0 do \ Max. 16 lines at a time cr .line 09 = \ End of source section? if .divider leave then \ Show divider & ready loop dup libhere < while \ More library source to be done? key bl <> if drop exit then \ Yes, ask key, stop on non space repeat drop ; v: extra definitions : VIEW ( ccc -- ) bl-word count lib-find libtype ; v: fresh \ PIN SQUERY ( GPIO -- ) \ Change GPIO pin for S? need [IF] dup dm 30 2 within [if] drop dm 24 [then] \ Invalid switch pin? 1 cfg 1+ c! \ GPIO-xx for S? 4 cfg @ abs 4 cfg ! \ Make sure to (re)start the second image config cr .( Test S? ) s? . \ 12MHZ \ change clock frequency dm 12 0 cfg ! \ Set frequency in MHz 4 cfg @ abs 4 cfg ! \ Make sure to (re)start the second image config \ Test new configuration