Graphics on BangleJS2

Since the code needs to be copied to RAM, I wonder if it makes sense to use heatshrink or LZ4 on it.

Maybe, but the storage is not the bottleneck, I tried it with code that was about 600 bytes and it was not worth it but maybe it would make some sense with larger code. However Heatshrink may not be good for binary code
if you would put this into declaration that comes out of inline C compiler as another method it would produce compressed declaration of the bin variable.

As for the malloc code I think it would be more reliable to allocate one big array, send pointer to it once and do the malloc/free inside C on top of that.

Also, does the GC know when it is safe to free the Uint8Array?

Yes, keeping reference is enough but it is also about defragmentation and stuff moving, I think flat arrays are not moved(?) when E.defrag() is triggered, smaller stuff is moved.

To call that from C it seems what I need is jspGetNamedVariable and jspeFunctionCall

When you start using stuff in https://github.com/gfwilliams/EspruinoCo­mpiler/blob/master/src/utils.js#L73 the output of inline compiler is fixed for specific device and FW version so cannot be distributed in binary form (it builds the code with hardcoded value from process.env.EXPTR of calling device). With slight change in the compiler it is possible to fix that so that the process.env.EXPTR is used at runtime instead of compile time. then the code from compiler looks like this

I'm actually not trying to do anything specific at the moment, I'm just poking to see what works/doesn't and what's a good idea or not. From there I'll see what I do about this engine I've been making.

I like being able to share any resulting games or watchfaces I make on the repo in a way that people don't have to compile things on their own to see it. I didn't know that having inlinedC would result in extra load to the compiler server, I just assumed there was a cache somewhere since there wouldn't be much point in compiling the same code again and again.

I had also assumed atob('string literal') was being pretokenized into a binary-safe format that doesn't need escaping, since that would be smaller/faster. It would allow displaying graphics without taking up RAM... but would it be just as fast, or is external flash slower to access?

when Bangle.js wakes up, it jumps straight into an IRQ which might try and access an XIP memory address while flash is asleep.

It could be used only for storage so it works basically as Storage module on top of internal flash. that would give you direct pointers to data and even code. However it may be tricky to implement writing to such storage when XIP is turned on.

my poor server's going to suffer if loads of people start downloading and forcing it to compile the same big project all the time

This could be part of minification so people uploading app to watch get js code with already compiled binary. Or in this case it could be part of non minified app (like this)

So we're saying that just adding -lgcc could be an option here, and doesn't appear to grow code size for 'normal' code that's completely self contained?

Yes, same for -lm.

I had also assumed atob('string literal') was being pretokenized into a binary-safe format that doesn't need escaping, since that would be smaller/faster. It would allow displaying graphics without taking up RAM... but would it be just as fast, or is external flash slower to access?

atob() is function call, makes new buffer in RAM when called, yes external flash is slower, every byte from SPI flash is read by toggling GPIOs in software, it is fast but much slower than internal RAM/Flash

atob() is function call

Yes, but so is E.compiledC. In the same way that compiledC is replaced, so can an atob call.

SPI flash is read by toggling GPIOs in software

It's bit bangging SPI? o_O

Yes, but so is E.inlinedC. In the same way that inlinedC is replaced, so can an atob call.

It is not tokenized but with direct string constants it could be, good idea.

It's bit bangging SPI? o_O

https://github.com/espruino/Espruino/blo­b/master/targets/nrf5x/jshardware.c#L249­6
Espruino is single threaded so using hardware SPI may not help much and may actually draw more power. We don't know for sure until it gets implemented :-)

It's bit bangging SPI? o_O

Yes - it sounds nuts but I have tested both hardware SPI and memory mapped QSPI. At least for the read patterns we do, it's still fastest to bit-bang using 2 bits for data, so that's what we do.

Pretokenising complete strings is an interesting idea, and could help in a lot of areas. Right now all we do is turn tokens into single bytes. I guess we could store strings like: TOKEN_STR,LEN_HI,LEN_LO,raw string bytes and could handle that pretty easily. If we spotted atob(...) we could just decode that and store the raw data as well.

Interesting! This sounds like something fun to play with. Do I need a devkit to mess with this code or can I do it safely over bluetooth without bricking my Banglejs2?

Interesting! This sounds like something fun to play with. Do I need a devkit to mess with this code or can I do it safely over bluetooth without bricking my Banglejs2?

Bluetooth is slow so updating firmware over it gets old pretty quickly but yes it should be mostly safe, there is a watchdog and bootloader so it is not that easy to brick it but it can be done. Better is to use some SWD debug dongle with the charging cable and usb female cable adapter (just cut some old cable with usb female on it). These things are < US$2 on aliexpress (search for daplink) or you can make almost anything into such dongle (like raspberry pico, blue pill board). Also STLINK V2 clones (like this one https://aliexpress.com/item/100500162162­6894.html - $1.60 including shipping for me) work - either as is or reflashed with daplink firmware. I have tons of them so when I need it I often take another one instead of disconnecting it from something :-)

I happen to have one of those STLINK V2 clones, so all I need is the usb female cable adapter.
Time to cut some old USB extension. :)

Time to cut some old USB extension. :)

https://www.espruino.com/Bangle.js2+Tech­nical#swd
Don't know right now which of green and white USB wires is SWD data vs clock, just try both ways, it won't hurt, red, black should be 5V, GND as usual but you can of course test against the connector (and the table in the link above). STLINK has 5V pin so you can connect it for charging too.

With ST firmware it should work in openocd like this openocd -d2 -f interface/stlink-v2.cfg -f target/nrf52.cfg

I should add that you can usually compile Espruino for Linux (or Windows WSL) by just cloning the repo and typing make

While you can't test the C compilation there, things like pretokenisation can be tested fine - so it's a much faster, less painful way of developing

My first try at getting this compiling didn't go so well... on a Mac. I'll try again on Arch Linux.

May not help much in your case but this works in fresh ubuntu 22.04 in WSL or booted from live CD/USB

sudo apt-get update ; sudo apt-get install python-is-python3 python3-pip git curl build-essential
git clone https://github.com/espruino/Espruino ; cd Espruino/ ; . scripts/provision.sh BANGLEJS2
make BOARD=BANGLEJS2 RELEASE=1 clean ; time make -j BOARD=BANGLEJS2 RELEASE=1

I use the final build as a simple non scientific CPU benchmark for random devices :-) Quite long part is last linking phase which is single threaded with gcc 8 so this tries the turbo frequency too.

Just to add that I'm looking into the raw strings code at the moment - I already have a proof of concept and it looks pretty easy - under 50 lines of changes.

I may have to do something a bit more complicated to get the pretokeniser to detect atob("...") but even that doesn't look too painful

Just committed so it'll be in cutting edge builds. Handles normal strings and atob("...")

The real improvement will be if the pretokeniser in the Web IDE / App Loader will do it automatically (when it knows the Bangle firmware supports it!), which I'll try next week. It should be a massive boost for clock faces that load their own fonts/images

One thought about this - currently we store the raw string as 0xD1, len_lo, len_hi, string_data

But for a bunch of strings they are less than 255 chars so only need one byte of length. Maybe we should consider doing a bit like MQTT does with variable message length, which IIRC is:

• 0xD1, len, string_data for <128
  
• 0xD1, 128 | (len&127), len>>7, string_data
  

This sounds a bit like UTF8. Would there happen to be code in the firmware for encoding/decoding that already?

Would there happen to be code in the firmware for encoding/decoding that already?

There is, but UTF8 encodes the value in the bottom 5/6 bits of the char code (I can't remember exactly) - it's slight overkill I think.

But I had an idea over the weekend - I'll just have two raw string types - 0xD1 for 16 bit and 0xD2 for 8 bit length

I wonder how it will work when you convert it back to readable data, like using edit(f) directly on device, also when you mix quotes or double quotes to include the other quote. Currently function f() {var a='aaa"aaa';print(a);} remembers the type of quotes when using edit(f). So these tokenised ones will simply print always quoted with " and the binary data or ' escaped(?)

Just committed...

I wonder how it will work when you convert it back to readable data

It's smart enough to know:

>E.setFlags({pretokenise:true})
>function f() {var a='aaa"aaa';print(a);}
>edit(f) 
>function f() {var a="aaa\"aaa";print(a);}

It's not perfect, but then if you've enabled pretokenisation you have to expect it won't be able to put everything back exactly as it was

There is one more thing about strings I did not know.
When you read via var t=require("Storage").read("test.js") then t is native string = it points to flash memory, not RAM
when you load/execute this test.js file

var s
var s2
var s3="native string"
function test(){
s=atob("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA­AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA")
  s2="Hello"
}

via load("test.js") then it can be seen via trace()

 #18[r1,l2] Name String [1 blocks] "s3"    #19[r1,l0] String [2 blocks] "native string"
  #21[r1,l2] Name String [1 blocks] "test"    #22[r1,l0] Function {
      #24[r1,l2] Name String [1 blocks] "\xFFcod"        #23[r1,l0] NativeString [1 blocks] "s=atob(\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA­AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\"­)s2=\"Hello\""
    }

that the function body is still native string and points to flash however the s3 string is normal string in RAM, also when you execute the test function the s2 also is a string in RAM. This I did not know.

I was expecting that like test function body and stuff read via Storage.read those s2,s3 strings would be native strings pointing to flash not taking any RAM. However it is not like that and never was(?) Works like that before and after this string tokenisation and also tested wit some older versions like 2.16.

If such optimization was there I was wondering what could happen with such strings when parsing or execution of code with these new features just added here but as it is not there there is no issue :-)

I guess this optimization I thought was already there could be added too? At least when the string is there full in verbatim and does not contain escape characters e.g. 'aaa"aaa' could point into native part aaa"aaa but it won't work for "aaa\"aaa"

Then in future case of WebIDE replacing atob() with big binary strings which would be stored directly inside the file in flash, those strings would also be in flash but without that they would be in RAM like shown here(?)

I tested this in device with no SPI flash so Storage is native nrf52 flash so maybe for BANGLE2 it may be different?

EDIT: I retested with Bangle 2 and the only difference is that function body is "FlashString" not "NativeString" but s2,s3 are in RAM too. But anyway with pretokenized strings this feature may be easier to implement and when done by IDE also the tokenized "aaa\"aaa" would be simple string that could be Flash/NativeString (when parsed from code that is also such string = storage file).

However it is not like that and never was(?)

Correct - it can't easily because as you noted the string might be escaped (although I guess we could have checked for that when parsing). Either way we'd have to parse over it at load time, and then again when we wanted to use it - so while it saves RAM it might not be a clear win for performance.

Then in future case of WebIDE replacing atob() with big binary strings which would be stored directly inside the file in flash, those strings would also be in flash

Correct, yes - and that is now the case if you turn on pretokenisation in the IDE. I'm going to have to update the advice in https://www.espruino.com/Code+Style#othe­r-suggestions !

Update

I've just pushed an updated Web IDE, BangleApps (in the dev app loader) and a new firmware (the raw strings exposed an interpreter issue that's been around for a while) and the results are really good.

Ensure pretokenisation is turned on in the dev app loader, that you have the latest firmware, and click Reinstall apps to update the code for every app with the pretokenised strings.

Quick example with the Anton clock:

On the same build, I do reset() followed by tm=getTime();load(eval(require("Storage"­).read("antonclk.app.js")));getTime()-tm­ which loads the anton clock and measures how long it takes.

• Not prtokenised 256ms
  
• pretokenised(old) 254ms
  
• pretokenised with strings 194ms
  
  

Bear in mind that's the whole clock, so it's a big improvement. And it pretty much all comes from loading the font at the start (even though Anton is relatively modest font-wise - other clocks will see bigger benefits).

I did a quick benchmark of just how long it takes to load the font, with a test file in Storage:

setFontAnton = function(scale) {
g.setFontCustom(atob("...."), 78 + (scale << 8) + (1 << 16));
};

t = getTime();for (var i=0;i<1000;i++) setFontAnton();print(getTime()-t,"ms")

• Not pretokenised: 53ms per call
  
• pretokenised with strings: 1.37ms
  
  

So it's around 40x faster! Font rendering will be a little slower, but it's not going to be huge compared to that boost.

The only downside at the moment is that Espruino will now make all strings in pretokenised code native strings, and I feel like we should probably change it so that if a string is pretokenised but it would fit inside one JsVar (eg it's not going to use any more RAM than the pointer to the string in flash) we should keep it in RAM.

I think I notice the improved speed - 40x is impressive!

Ensure pretokenisation is turned on in the dev app loader, that you have the latest firmware, and click Reinstall apps to update the code for every app with the pretokenised strings.

I did this. Then I compacted the storage via the settings app and the on screen message is incomplete (see attached image). I reinstalled the settings app without pretokenization and tried compacting again, this time the whole message was displayed.

I am not sure that it has to do with the new changes, I have not tried with older firmwares and pretokenization. But I thought I'd mention it here anyway.

1 Attachment