Graphics on BangleJS2

Interesting, thanks! I can confirm it happens here too, I'll look into it. Please let me know if you notice anything else...

Ok, it turns out there's something strange with E.showMessage and flash strings.

edit: It's g.wrapString - I'll try and look at this in the next few days

// ok
E.showMessage("Compacting...\nTakes approx\n1 minute")
// problem
require("Storage").write("x","Compacting­...\nTakes approx\n1 minute")
E.showMessage(require("Storage").read("x­"))

The actual pretokenising works fine by the look of it

It's now fixed on cutting edge builds!

While it sure isn't as trivial an optimization as it seemed to me at first, the speedup (and space savings?) from your benchmarks sound great!

I've updated to the latest firmware. When I try to reinstall all the apps I get the errors in the attachment. The first time I tried, it got stuck updating the Scheduler app. I manually uninstalled/installed it, then tried again. Now it doesn't get stuck but it outputs those errors.

In the WebIDE, if I enable pretokenization on Synthwave and try to upload to RAM I get the following error:
You have more open brackets than close brackets. Please see the hints in the Editor window.
Apparently it's trying to reformat the pretokenized code and breaks.

If I don't enable pretokenization, Synthwave uploads to RAM successfully, but the C++ code no longer works, I haven't looked into what broke yet. If I enable pretokenization but upload to storage instead of RAM, it crashes. It only works if I disable pretokenization and upload to storage.

Enabling Esprima minification results in "Error: Unreachable point. logically broken.", but the code uploads anyway.

Edit:
The following works with pretokenization in Storage but not in RAM.

    let c = E.compiledC(`
    // int boop(int)
    int boop(int d) {
      return d * 42;
    }
    `);
    print('boop:', c.boop(3));

1 Attachment

Those errors are expected - the 404s are fine (it tries different URLs to find the files, sometimes they're not at the first URL). They would have happened before too.

I've seen Error: Unreachable point. logically broken. for widget_utils too - it's not related to these changes. It seems it's a continue statement which looks legit, but for some reason the minifier takes issue with it - I'll just refactor the code slightly to fix that

There's no Synthwave in the main banglejs.com/apps page - so I assume it's in your repo? Any chance you haven't updated to the latest AppLoaderCore for the core folder? That might be using old tools which are then breaking the upload?

There's no Synthwave in the main banglejs.com/apps page - so I assume it's in your repo?

The PullRequest for it (together with Warpdrive) is still open (https://github.com/espruino/BangleApps/p­ull/3156). My fork of BangleApps is only missing the last 3 commits, which don't seem to be related. Besides, I was testing on https://www.espruino.com/ide, not my own fork of the IDE.

The following works with pretokenization in Storage but not in RAM.

Oh, I would expect the opposite! With current tokenization the inline C as part of storage file could now have atob array automatically expanded and tokenized so the binary string with executable code may point to the flash memory instead of RAM.

could be fixed by inline compiler by generating something like var b="base64encodedstring";var bin=atob(b) or maybe bin=""+atob("xxx") or maybe atob("xxxx",true)

Oh, I would expect the opposite!

That snippet doesn't have static/global variables, so it should be fine anywhere. The crash is probably due to the IDE replacing unknown characters with ? when uploading.

bin=""+atob("xxx")

This would be best, the others would avoid the atob optimization.

The whole message shows for me as well now. Thanks!

New ram leak

I get a new ram leak I didn't have before.

It only shows when on cutting edge firmware (2v20.69) and pretokenization is used on upload.

So far I noticed it when fastloading from spotrem to edgeclk .

I have not looked closer on spotrem code yet to see if I could find the leak.

Since fastloading from spotrem is made via fastload utils which we explicitly state is experimental I'm aware this is maybe not a priority. But I think it makes sense to mention the change here at least.

Memory usage increasing when switching between spotrem and edgeclk, both pretokenized:

>reset();
=undefined
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v20.69 (c) 2023 G.Williams
>process.memory().usage;
=966
>Bangle.load("edgeclk.app.js");
=undefined
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v20.69 (c) 2023 G.Williams
>process.memory().usage;
=2285
>Bangle.load("spotrem.app.js");
=undefined
>process.memory().usage;
=3398
>Bangle.load("edgeclk.app.js");
=undefined
>process.memory().usage;
=3519
>Bangle.load("spotrem.app.js");
=undefined
>process.memory().usage;
=4409
>Bangle.load("edgeclk.app.js");
=undefined
>process.memory().usage;
=4495
>Bangle.load("spotrem.app.js");
=undefined
>process.memory().usage;
=5385
>Bangle.load("edgeclk.app.js");
=undefined
>process.memory().usage;
=5471
>Bangle.load("spotrem.app.js");
=undefined
>process.memory().usage;
=6361
>Bangle.load("edgeclk.app.js");
=undefined
>process.memory().usage;
=6447
> 

Memory usage increasing a lot when repeatedly fastloading just spotrem and it is pretokenized:

>reset();
=undefined
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v20.69 (c) 2023 G.Williams
>Bangle.load("spotrem.app.js");
=undefined
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v20.69 (c) 2023 G.Williams
>process.memory().usage;
=3058
>Bangle.load("spotrem.app.js");
=undefined
>process.memory().usage;
=4204
>Bangle.load("spotrem.app.js");
=undefined
>process.memory().usage;
=5177
>Bangle.load("spotrem.app.js");
=undefined
>process.memory().usage;
=6150
>Bangle.load("spotrem.app.js");
=undefined
>process.memory().usage;
=7123
> 

Memory usage increasing only slightly (sometimes decreasing) when repeatedly fastloading just spotrem and it's not pretokenized:

>reset();
=undefined
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v20.69 (c) 2023 G.Williams
>
>process.memory().usage;
=999
>Bangle.load("spotrem.app.js");
=undefined
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v20.69 (c) 2023 G.Williams
>process.memory().usage;
=3268
>Bangle.load("spotrem.app.js");
=undefined
>process.memory().usage;
=4621
>Bangle.load("spotrem.app.js");
=undefined
>process.memory().usage;
=4624
>Bangle.load("spotrem.app.js");
=undefined
>process.memory().usage;
=4627
>Bangle.load("spotrem.app.js");
=undefined
>process.memory().usage;
=4630
>Bangle.load("spotrem.app.js");
=undefined
>process.memory().usage;
=4602
>Bangle.load("spotrem.app.js");
=undefined
>process.memory().usage;
=4636
> 

Are you sure this is not a widget taking more memory or app bug?

This is fastloading between antonclkplus and taglaunch witch a button press:

process.memory().usage;
=3506
process.memory().usage;
=5217
process.memory().usage;
=3606
process.memory().usage;
=5217
process.memory().usage;
=3606
process.memory().usage;
=5217
process.memory().usage;
=3606
process.memory().usage;
=5217
process.memory().usage;
=3606
process.version
="2v20.69"

Note: So far I only have reinstalled antonclkplus and taglaunch with new pretokenization.

Above I only see one increase for antonclkplus from 3506 to 3606 after taglaunch, probably from loading a lib or similar.

I was testing on https://www.espruino.com/ide

Ahh - I just checked. Are you uploading to RAM?

I had to fix the IDE so it worked when uploading to a file, but it looks like I didn't check what happens when you upload to RAM and it seems that might be broken. I'll look into it.

Memory leak

I just tried with spotrem but without fastload (but going to/from the launcher which should be ok) and I don't see any increase in memory, so it does seem fastload related.

I'm looking into it now and I'll see if I find anything obvious. E.getSizeOf(global) still reports around 1000 which makes me think there is a proper leak somewhere

Found out what it was - it's a lock leak when using flash strings as property names in objects. It'd end up leaking memory pretty much whenever you display a menu, so it's good to have found it! Actually fixing it is proving problematic though...

And annoyingly it's nothing to do with the pretokenisation changes - it's just that they happened to pick it up.

edit: Fixed!

I was testing on https://www.espruino.com/ide

Maybe try refreshing now and see if it works.

I looked at this and without redoing the pretokenisation to use 7 bit length vars (to avoid control chars under 32) we won't be able to get pretokenised code sent to the REPL.

But I think that's fine (command history would look totally broken!) - I've just disabled sending of pretokenised code to the REPL.

Confirmed, uploading to RAM in the IDE now works.

I repeated the tests with Synthwave:
Storage + No pretokenize = OK
Storage + Pretokenize = Crash and reboot. Trying to run code directly in storage?
RAM + Pretokenize = No crash but no rendering
RAM + No pretokenize = No crash but no rendering
Flash + No pretokenize = OK
I'll do some more testing later to figure out why it doesn't render anything when running from RAM.

I guess it is not a good idea to update to bleeding edge 2v20.76?

> Uncaught SyntaxError: Got '(' expected ','
 at line 4 col 151 in locale
...i,Mi,Do,Fr,Sa".split(',')[d.getDay()+­(short?7:0)],month:(d,s...
                              ^
 at line 4 col 638 in locale
...istance:(n,dp)=>n<1000?round(n,dp)+"m­":round(n/1000,dp)+"km",...
                               ^
 at line 4 col 191 in antonclkplus.app.js
...dsWithColon?0:g.stringWidth(":")/2);v­ar dateStr2=(dateOnMain==="ISO8601"...
                                   ^
in function "draw" called from line 3 col 3714 in antonclkplus.app.js
...=false;queueMillis=60000;}draw();}els­e{if(drawTimeout)clearTimeout(d...
                                      ^
in function "updateState" called from line 4 col 1582 in antonclkplus.app.js
...gs();g.clear();updateState();Bangle.o­n("lcdPower",updateStat..

1 Attachment

I experience pretty big problems as well with 2v20.76. I'd advice against updating to it for now.

Problem is: How to go back? Flashing from app loader does not work and recovery menu does not work.

There is a couple of ways to flash firmware. I think now we have to use the method here: https://www.espruino.com/Firmware+Update­#android-ios-app

Right, DFU Flashing works (did it from the IDE)

For me, fw2v20.73 seems to work, while 2v20.74 and up don't. So I'd guess commit 5fee08ff28ae5e9b14b199a7849200cf9d7c9398­ is the culprit, since that is .74's commit.

Disconnected from Web Bluetooth, Bangle.js 92f9
Initialising...
Updating application: espruino_2v20.74_banglejs2_app.bin...
Flashing Complete
Found BANGLEJS2, 2v20.74
>
Connected to Web Bluetooth, Bangle.js 92f9
Uncaught SyntaxError: Got ')' expected EOF
 at line 1 col 203
...&&(m=!1,WIDGETS.back.remove(c&&b.back­));Bangle.btnWatches&&(...
                              ^
 at line 1 col 213
...DGETS.back.remove(c&&b.back));Bangle.­btnWatches&&(Bangle.btn...
                              ^
in function called from line 3 col 355 in antonclk.app.js
...s.prototype.setFontAnton;}});Bangle.l­oadWidgets();draw();set...
                              ^
at line 3 col 355 in antonclk.app.js
...s.prototype.setFontAnton;}});Bangle.l­oadWidgets();draw();set...
                              ^
Disconnected from Web Bluetooth, Bangle.js 92f9
Initialising...
Updating application: espruino_2v20.73_banglejs2_app.bin...
Flashing Complete

edit: Fixed!

Thanks! I just tried it on 2v20.73 and I don't see the leak anymore.

yes, the .74 broke it, I cannot even list storage files in IDE

require('Storage').list().forEach(x=>pri­nt(JSON.stringify(x)));
Uncaught ReferenceError: "x" is not defined
 at line 1 col 59
...ch(x=>print(JSON.stringify(x)));

with .73 this works fine

Sorry, my bad. I stupidly thought I had a one-line fix for https://github.com/espruino/Espruino/iss­ues/2457

Just reverted so should be back to normal now

Just retested experiments with native strings and it works as expected now.

var s;
var s2;
var s3="native string";
var s4;
function test(){
s=atob("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA­AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA");
  s2="Hello";
  s4="This is a longer string";
}
test()

after uploading this file to storage and running it s,s3,s4 are native strings pointing to internal flash. s2 is not as it is too short.

However Inline C code upload does not work, the atob call there is not parsed properly, even this line does not work

bin=atob("gAAH7pAK3+0barju53r47gB6sO5nWo­kAp+4mWgAgB+4QGrjux3rx7gBK5+4mep/tEnqw7m­dq8O5Hemfup2pn7gdaNu6lSrTuZErx7hD6D9UfKA­3cd+6nerDuRkqn7odKATB27uV6sO5EenfuhXrk53­BHAL8AAIA8AAAAAAAjELUcRgAiEEYZRv/3uv8BMk­AqBET30QEzQCvz0SBGEL0AAA==");

I get

Uncaught SyntaxError: Got [ERASED] expected EOF
 at line 7 col 21 in test.js
@*D ?[247]"@+\xF3\xD1 F\x10\xBD\0\0;" [ERASED]
                                  ^

I tried to reload https://www.espruino.com/ide/ but it does not help.

Another thing is that even if it worked the native string in flash produced by atob is not aligned so even if the code had no global variables it is still not executable as is. E.toFlatString() helps with that and moves code to RAM like it was before.

Too bad there is probably no easy way to align the native string in flash as quite a lot of inline C code don't use global variables so could in theory run directly from internal flash (when it is nrf52 internal flash, not SPI flash).