Syntactic Elements

ABSTRACT
If you write for portability, compatibility with straightforward dual-wordlist scheme is worth care.

The Technical Committee is going to give up the concepts of compilation and execution semantics. It has not been surprising to me because I knew about a contradiction within the concept of POSTPONE and compilation/execution/etc. semantics.

But the following fact was surprizing to me: all we standard readers were misinterpreting the intent of the Technical Committee. That is, the 1994 formulation of the Standard does not adequately present the intent of TC.

Here's what seems to be intended:

IF is a word that cannot be redefined by standard means. In particular, you cannot:
1) redefine it as a non-immediate word;
2) redefine it as an immediate word either.

The Technical Committee is going to introduce a concept of syntactic element. In other languages such things are called reserved words. It is not true that you cannot redefine "reserved words", but it is true that you cannot do it portably.

The intent of Technical Committee has been to allow straightforward dual-wordlist implementations be standard (opposed to the single-wordlist scheme where both immediate and non-immediate words go to the same wordlist). This means that only programs that are written to run on both single-wordlist and dual-wordlist systems are standard. And this means that it is not true that only such dual-wordlist systems that can run programs written for single-wordlist systems are standard. In other words, the differences between single-wordlist and dual-wordlist systems are up to the program writers, rather than system implementors.

Since the intent of TC has been mispresented (or, maybe, just misunderstood), changing the standard in this direction will not be considered as changing the intent, and the question of code breakage most likely will not arise. (The code that relied on the peculiarities of the single-wordlist approach has never been truly portable, even if you thought it's standard, isn't it?)

I've been asked to give an example of a problem where difference between the dual-wordlist and single-wordlist schemes is important.

In fact, any application that redefines the standard compiling words as non-immediate words will have problems when moved between single-wordlist and dual-wordlist model. On the other hand, the author of such code will definitely have problems while maintaining code where unstandard words have standard names.

The most evident example of an application that redefines standard words is a cross-assembler with Forth-like control structures. (Yes, cross-assembler is an application, it is code written by us). The Assembler's IF may be defined as 


ASSEMBLER DEFINITIONS
: IF ( cond-id -- asm-orig )
   C,              \ "condition id" is really opcode
   HERE  C,        \ room for branch destination
; ( non-immediate !!! )

\ cond-id's are defined like this:
nn CONSTANT 0<
nn CONSTANT 0=
etc.

\ .IF is the Forth IF
: .IF [ FORTH ] POSTPONE IF [ ASSEMBLER ] ; IMMEDIATE
Now, which IF will be used in the following macro? 
ASSEMBLER DEFINITIONS
: ABS_AX
  CMP AX, # 0
  0< IF       \ in assembler, IF compiles a conditional branch,
              \ it's not immediate.
              \ but which IF will be found in a dual-wordlist system?
    NEG AX
  THEN
;
In the thraditional single-wordlist model it definitely will be the assembler's IF. So it would be in Forth-83 and FIG-Forth.

It is important which IF is used: if it is assembler's IF, then a conditional branch will be compiled after CMP, and ABS_AX will be expanded to: 

  CMP AX , # 0
  JGE $+1
  NEG AX
If it is Forth's IF, the result will be: 
  CMP AX , # 0
  NEG AX
That is, there will be an error without any warning! Why? Because assembler's 0< leaves a "cond-id" on the stack; it is non-zero. Then, Forth's IF takes this value as a true flag and just continues execution of the macro. NEG AX works ok and Forth's THEN causes no problems (because it worked at run-time and successfully resolved the Forth's IF).

It may work the right way on some systems, but I do not know if a system where this works in the wrong way is also standard-compliant. (It's clear that such two systems are not compatible, but are they both compliant? I think, they will be.)

What is a dual wordlist system?
A dual wordlist system has all compile-only words in a separate wordlist. This wordlist is searched only in compilation mode and all words found in this wordlist are considered to be immediate. The word : (colon) adds this wordlist to the search order and the word ; (semi-colon) removes it from there.

During 1994-1999, we all thought that implementation of the dual-wordlist approach in a way that would be standard-compliant (=compatible with the classical single-wordlist apporach) is a complex, inadequately complex task. Now it seems that for a system to be standard-compliant, compatibility with the classical scheme is not required. And then it is absolutely unclear what functionality we the program writers can rely on.

What is a single-wordlist system?
It is a system where both immediate and non-immediate reside in the same wordlist. Immediacy of a word is determined by a flag in the header. The words : (colon) and ; (semi-colon) need not change the search order.

end of document