Terminology: a lexer is responsible for tokenisation of the input—breaking it into words, if you will, like if, camelVar, 6.02e+23, {, 'string', + and +=—while the parser is responsible for syntactic analysis, the grouping of those tokens into statements and expressions (with, admittedly, a few grey areas such as backquotes and regular expressions). It's not a logical necessity that languages be designed with this distinction, but as a practical matter almost all programming languages since the 60s seem to in one way or another. In the C family (to which JS belongs notationally, though not semantically), it's pretty crucial to understanding what's going on, and the formal concept (though not a typical implementation—and certainly not this implementation) is that tokenisation is a separate process that runs ‘before’ parsing. In any case, in JS, -- is one token and - - is two, just as xx is different from x x and == is different from = =, so in this case, the space is significant. The language processor, taken as a whole. is wrong to treat - - as --, and would be just as wrong to treat -- as - -.
Espruino does not always make this mistake—var x does not get confused with varx and the far more obscure example do x(); while (!done()); does not get misinterpreted as the (equally legal) dox(); while (!done());, even during full file interactive upload to the emulator (the context that triggers the bug I was reporting).
Espruino is a JavaScript interpreter for low-power Microcontrollers. This site is both a support community for Espruino and a place to share what you are working on.
Terminology: a lexer is responsible for tokenisation of the input—breaking it into words, if you will, like
if
,camelVar
,6.02e+23
,{
,'string'
,+
and+=
—while the parser is responsible for syntactic analysis, the grouping of those tokens into statements and expressions (with, admittedly, a few grey areas such as backquotes and regular expressions). It's not a logical necessity that languages be designed with this distinction, but as a practical matter almost all programming languages since the 60s seem to in one way or another. In the C family (to which JS belongs notationally, though not semantically), it's pretty crucial to understanding what's going on, and the formal concept (though not a typical implementation—and certainly not this implementation) is that tokenisation is a separate process that runs ‘before’ parsing. In any case, in JS,--
is one token and- -
is two, just asxx
is different fromx x
and==
is different from= =
, so in this case, the space is significant. The language processor, taken as a whole. is wrong to treat- -
as--
, and would be just as wrong to treat--
as- -
.Espruino does not always make this mistake—
var x
does not get confused withvarx
and the far more obscure exampledo x(); while (!done());
does not get misinterpreted as the (equally legal)dox(); while (!done());
, even during full file interactive upload to the emulator (the context that triggers the bug I was reporting).