-
I think I notice the improved speed - 40x is impressive!
Ensure pretokenisation is turned on in the dev app loader, that you have the latest firmware, and click Reinstall apps to update the code for every app with the pretokenised strings.
I did this. Then I compacted the storage via the settings app and the on screen message is incomplete (see attached image). I reinstalled the settings app without pretokenization and tried compacting again, this time the whole message was displayed.
I am not sure that it has to do with the new changes, I have not tried with older firmwares and pretokenization. But I thought I'd mention it here anyway.
1 Attachment
Correct - it can't easily because as you noted the string might be escaped (although I guess we could have checked for that when parsing). Either way we'd have to parse over it at load time, and then again when we wanted to use it - so while it saves RAM it might not be a clear win for performance.
Correct, yes - and that is now the case if you turn on pretokenisation in the IDE. I'm going to have to update the advice in https://www.espruino.com/Code+Style#other-suggestions !
Update
I've just pushed an updated Web IDE, BangleApps (in the dev app loader) and a new firmware (the raw strings exposed an interpreter issue that's been around for a while) and the results are really good.
Ensure pretokenisation is turned on in the dev app loader, that you have the latest firmware, and click
Reinstall apps
to update the code for every app with the pretokenised strings.Quick example with the Anton clock:
On the same build, I do
reset()
followed bytm=getTime();load(eval(require("Storage").read("antonclk.app.js")));getTime()-tm
which loads the anton clock and measures how long it takes.Bear in mind that's the whole clock, so it's a big improvement. And it pretty much all comes from loading the font at the start (even though Anton is relatively modest font-wise - other clocks will see bigger benefits).
I did a quick benchmark of just how long it takes to load the font, with a test file in Storage:
So it's around 40x faster! Font rendering will be a little slower, but it's not going to be huge compared to that boost.
The only downside at the moment is that Espruino will now make all strings in pretokenised code native strings, and I feel like we should probably change it so that if a string is pretokenised but it would fit inside one JsVar (eg it's not going to use any more RAM than the pointer to the string in flash) we should keep it in RAM.