Graphics on BangleJS2

Because JavaScript uses doubles you need to be able to convert to float and back.

This could be done before entering your inlineC code if float type was supported directly. And there is the Float32Array - this is part of javascript standard so javascript can already convert it for you to float type.

I don't know what the implication is to support float directly, but it does sound like a good alternative to doing the conversion in C. The conversion needs to happen somewhere. Is it better to have it in Espruino, where it will take up space even if you don't use compiledC, or is it better to have in each compiledC block that uses doubles?

Personally, I don't really need to pass floats/doubles to/from C. Just being able to work with floats is enough for me. I merely replied with a working version of the code that was asked for.

For some reason the following compiles fine on my local EspruinoCompiler, but fails on the online one:

let c = E.compiledC(`
// int boop(double)

int boop(double d) {
  return float(d) * 3;
}
`);

print('boop:', c.boop(3.5));

Maybe it's a different GCC version? arm-none-eabi-gcc -v gives me:
gcc version 10.3.1 20210824 (release) (GNU Arm Embedded Toolchain 10.3-2021.10)

I think fixing it on the online one is just a matter of adding cflags += "-lgcc "; but I can't test that here. It doesn't increase the binary size unnecessarily and is much better than messing with inline assembly.

Is it better to have it in Espruino, where it will take up space even if you don't use compiledC, or is it better to have in each compiledC block that uses doubles?

Well, the conversion code already takes that space in Espruino due to Float32Array. I just tried to implement float type support for native calls in espruino and with this small patch (also attached) it works

diff --git a/src/jsnative.c b/src/jsnative.c
index fb1f20a22..201a158a7 100644
--- a/src/jsnative.c
+++ b/src/jsnative.c
@@ -145,6 +145,12 @@ JsVar *jsnCallFunction(void *function, JsnArgumentType argumentSpecifier, JsVar
     case JSWAT_INT32: // 32 bit int
       argData[argCount++] = (uint32_t)jsvGetInteger(param);
       break;
+    case JSWAT_FLOAT32: { // 32 bit float
+      union {uint32_t i; float f;} v; // with softfp convention float values are passed in integer registers
+      v.f = (float) jsvGetFloat(param); // truncate from double
+      argData[argCount++] = v.i;
+      break;
+    }
 [#ifndef](https://forum.espruino.com/search/?q=%23ifndef) ESPR_EMBED
     case JSWAT_PIN: // 16 bit int
       argData[argCount++] = (uint32_t)jshGetPinFromVar(param);
@@ -265,6 +271,11 @@ JsVar *jsnCallFunction(void *function, JsnArgumentType argumentSpecifier, JsVar
     return jsvNewFromInteger((JsVarInt)result);
   case JSWAT_JSVARFLOAT: // 64 bit float
     return jsvNewFromFloat(*(JsVarFloat*)&result);
+  case JSWAT_FLOAT32:  {// 32 bit float
+    union {uint32_t i; float f;} v; // with softfp convention float values are passed in integer registers
+    v.i = (uint32_t)result;
+    return jsvNewFromFloat((JsVarFloat)v.f);
+  }
   default:
     assert(0);
     return 0;
diff --git a/src/jswrap_espruino.c b/src/jswrap_espruino.c
index 1d6ef5cba..80ccf61f7 100644
--- a/src/jswrap_espruino.c
+++ b/src/jswrap_espruino.c
@@ -196,6 +211,7 @@ int nativeCallGetCType() {
     if (strcmp(name,"bool")==0) t=JSWAT_BOOL;
     if (strcmp(name,"Pin")==0) t=JSWAT_PIN;
     if (strcmp(name,"JsVar")==0) t=JSWAT_JSVAR;
+    if (strcmp(name,"float")==0) t=JSWAT_FLOAT32;
     jslMatch(LEX_ID);
     return t;
   }
diff --git a/src/jswrapper.h b/src/jswrapper.h
index 99866f0dd..c60a55069 100644
--- a/src/jswrapper.h
+++ b/src/jswrapper.h
@@ -43,6 +43,7 @@ typedef enum {
   JSWAT_BOOL, ///< boolean
   JSWAT_INT32, ///< 32 bit int
   JSWAT_PIN, ///< A pin
+  JSWAT_FLOAT32, ///< 32 bit float
   JSWAT_JSVARFLOAT, ///< 64 bit float
   JSWAT__LAST = JSWAT_JSVARFLOAT,
   JSWAT_MASK = NEXT_POWER_2(JSWAT__LAST)-1,

Tested with

var c = E.compiledC(`
// float fadd(float,float)
// float fmul(float,float)

float fadd(float f1,float f2){
return f1+f2;
}
float fmul(float f1,float f2){
return f1*f2;
}`);
print(c.fmul(1.5,-1.5));
print(c.fadd(1.5,-0.25));

and I get

>-2.25
>1.25

EDIT: after comparing output of
echo 'int main(){return 0;}' | arm-none-eabi-gcc -mfloat-abi=softfp -mfpu=fpv4-sp-d16 -xc - -dM -E | sort
for different calling conventions it looks like the code in jsnative.c and jswrap_espruino.c could be #ifdef-ed by __ARM_PCS so that it is there only for ARM soft/softfp builds.

1 Attachment

• FLOAT32.patch

@fanoush that's neat! I think I'd been put off doing it because I was concerned there were only 3 bits for the argument type, and we already had 7 arg types - but we do have one free so we might as well use it :)

I've just merged that in...

it looks like the code in jsnative.c and jswrap_espruino.c could be ifdef-ed by __ARM_PCS

Which bit?

@FManga thanks for the d2f implementation - it's a bit odd it's not getting picked up by the linker automatically if it's in the file though. There's something odd going on there as for Espruino it appears to work fine for us: https://github.com/espruino/Espruino/blob/master/src/jswrap_math.c#L33

Which bit?

well the code with union of int32 and float will work only on arm with soft or softfp convention so I am not sure what to do when building for other platforms (like linux or esp32) so for those I was thinking about keeping just the enum but ifdef everything out including the nativeCallGetCType "float" line. Or keep it in nativeCallGetCType but fail at runtime when someone calls E.nativeCall on such platforms with float argument in signature - then the ifdef can be only for the 'case JSWAT_FLOAT32:' bits. however for RELEASE=1 it will fail silently. I am not actually sure for which platforms the Inline C and/or E.nativeCall is implemented. So maybe it is Ok to keep it as is. I don't know :-)

@FManga thanks for the d2f implementation - it's a bit odd it's not getting picked up by the linker automatically if it's in the file

can be C vs C++ name mangling, Inline C is actually compiled as C++ source (unlike espruino jswrap_math.c) so something like wrapping it into extern "C" { } could work. EDIT: oh it is already there!! so I don't know

Ahh, ok - Well, while nativeCall is there on most platforms, I think it's only really used on ARM - so I wouldn't worry too much. I guess if people start to use it for built-in functions it could become a problem though...

It is the -flto, once I remove it, it links fine. Also when I compile __aeabi_d2f.c alone into separate .o but without -flto it works when the rest uses -flto. However when I add -flto to that separate compile into __aeabi_d2f.o it does not help and does not work.
So somehow gcc expects the __aeabi_d2f to be placed in object file which is not compiled with lto.

Thanks for checking - that's really strange... All of Espruino is compiled with flto and it seems to work in that case... I wonder what's different!

it needs __attribute__((used)), it builds when used like that

extern "C" __attribute__((used)) float __aeabi_d2f(double d){
return 1.0f;
}

Wouldn't that also force it into the resulting binary even if it's not used?

Unfortunately yes, I just tried and for otherwise unused function it makes it bigger with this. But without this it probably gets optimized out because it is not directly called so LTO removes it(?). Or it is built in a way that linker does not expect later when searching for it - LTO may add some additional flags or mangling(?) that this method in standard library is not expected to have.

EDIT: some info is in https://gcc.gnu.org/wiki/LinkTimeOptimizationFAQ that's where I got the __attribute__((used)) idea from

However when I add -flto to that separate compile into __aeabi_d2f.o it does not help and does not work.

without -flto

$ arm-none-eabi-gcc -c -fno-common -fno-exceptions -fdata-sections -ffunction-sections -fpermissive -Os -fpic -fpie -mfloat-abi=softfp -mfpu=fpv4-sp-d16 -fsingle-precision-constant -Wdouble-promotion -Wfloat-conversion  -fno-fat-lto-objects __aeabi_d2f.cpp -o __aeabi_d2f.o
$ arm-none-eabi-objdump -d __aeabi_d2f.o

__aeabi_d2f.o:     file format elf32-littlearm


Disassembly of section .text.__aeabi_d2f:

00000000 <__aeabi_d2f>:
   0:   e3a005fe        mov     r0, #1065353216 ; 0x3f800000
   4:   e12fff1e        bx      lr

However with -flto the object file is in fact empty (with tons of other LTO stuff still in it :-)

$ arm-none-eabi-gcc -c -fno-common -fno-exceptions -fdata-sections -ffunction-sections -fpermissive -Os -fpic -fpie -mfloat-abi=softfp -mfpu=fpv4-sp-d16 -fsingle-precision-constant -Wdouble-promotion -Wfloat-conversion  -flto -fno-fat-lt
o-objects __aeabi_d2f.cpp -o __aeabi_d2f.o
$ arm-none-eabi-objdump -d __aeabi_d2f.o

__aeabi_d2f.o:     file format elf32-littlearm

Is there any reason why we don't simply use -lgcc?
If so, I can prepare the opposite f2d conversion, we compile both without lto then link to that instead. That would solve all the problems, right?

Is there any reason why we don't simply use -lgcc?

https://github.com/gfwilliams/EspruinoCompiler/blob/master/inc/linker.ld#L39
Probably to not to link in lot of bloat? You can try without that line how big the binary will be, the code is loaded into RAM so should be small, there is not that much RAM available.

The linker will still discard anything that isn't actually used so it shouldn't introduce any bloat.
When I cast a double to float the only thing that gets added to the binary is the expected __aeabi_d2f function.

Indeed, just tested, and the DISCARD linker script does not work for this as it gets added anyway when I add "-lgcc".

I see some suspicious extra .debug_frame section listed by objdump when -lgcc is there but this does not get into the binary as adding it into DISCARD works and the btoa string is still the same even when not seeing it in the output.

Not sure if everything from libgcc.a is harmless like this but looks like for this case it works fine indeed.

the code is loaded into RAM

I hoped all the compiled code wasn't in RAM. Is this because it's in a string or does the chip not support XiP to run directly from flash?

I hoped all the compiled code wasn't in RAM. Is this because it's in a string or does the chip not support XiP to run directly from flash?

It is meant for small time critical things, otherwise you are supposed to code mainly in JavaScript :-)
It is linked so that data segment with writable variables is in same area together with code (for simplicity of loading). Also yes, Bangle2 SPI flash is not setup for XIP (hardware can do it). There is support for two flash storage areas, one being internal nrf52 flash which is executable, but it is currently not enabled (?), I have this enabled in some older builds for other devices and it works, if you would write the flat string there and don't use any global variable then it would probably work. Also another answer is that if you need lot of C code added you make your own build of Espruino with your own module compiled in.

EDIT: you can try require("Flash").getFree() if it gives you something sensible on Bangle2, then you could write the code to free area of internal flash directly to test how/if it works

EDIT2:
require("Flash").getFree()
=[
{ addr: 675840, length: 249856 }
]

250KB is quite a lot, with some JS code that would manage loading code into that, it could be usable.

BTW as the floating point hardware can do only basic operations like +-*/ I tried to add -lm and use sinf() and it pulls quite a lot of code and the binary grows to 4KB. If I used double type and sin() instead the same code is almost 8KB.

Ah, with data and code being together it makes sense.
I would prefer not to require my own build of Espruino. I'd like to make my graphics/game engine in C++, the game logic in JavaScript, then be able to share it with others on the apps repo.

By using Flash module it is doable. If you would allocate writable data separately in JS and pass it as pointers to code stored to flash it needs no changes. some js code to manage free internal flash blocks so you would know where the code is loaded shouldn't be hard. the code is position independent. Also you can use your local EspruinoCompiler and distribute just the compiled output as part of your app (with sources somewhere and guide how to recompile it).

For some reason I don't need to add -lm for my local EspruinoCompiler to find sinf, just like I didn't need -lgcc before.
It would be nice if each compile/upload in the IDE would give some stats (amount of space taken up by strings, total space taken up in flash, free RAM on the connected device). As it is, I have no idea if 4KB is a lot or if that's acceptable.
Since the code needs to be copied to RAM, I wonder if it makes sense to use heatshrink or LZ4 on it.

If you would allocate writable data separately in JS and pass it as pointers to code stored to flash it needs no changes.

Something like this?

function malloc(size) {
  return E.getAddressOf(new Uint8Array(size), true);
}

Can that be called from C? Also, does the GC know when it is safe to free the Uint8Array?
Would I need to keep references to the Uint8Array in JS to keep it from being collected?

const blocks = [];
function malloc(size) {
  let block = new Uint8Array(size);
  blocks.push(block);
  return E.getAddressOf(block, true);
}
function free(addr) {
  for (let i = 0; i < blocks.length; ++i) {
    if (addr == E.getAddressOf(blocks[i], true)) {
      let last = blocks.pop();
      if (blocks.length != i) blocks[i] = last;
      return;
    }
  }
}

Edit: To call that from C it seems what I need is jspGetNamedVariable and jspeFunctionCall.

Edit2: It might make sense to have a libespruino that implements malloc/free/sin/cos/sinf/cosf/etc by calling the JS implementations.

So we're saying that just adding -lgcc could be an option here, and doesn't appear to grow code size for 'normal' code that's completely self contained?

RAM/Flash

I have tried enabling XIP, but there are some big issues with it - for instance the flash has to be put to sleep when sleeping but then when Bangle.js wakes up, it jumps straight into an IRQ which might try and access an XIP memory address while flash is asleep.

It's a nightmare - what you can do is enable a second flash filesystem in internal flash that by default JS files go in - I was going to start producing a second build for it as it's a lot faster, but even that seems to have some problems at the moment (I'm not 100% sure why).

Would I need to keep references to the Uint8Array in JS to keep it from being collected?

Yes, exactly - definitely don't do E.getAddressOf(new Uint8Array(size), true); or GC will free the area you just got the pointer to as soon as getAddressOf exits!

To call that from C it seems what I need is jspGetNamedVariable and jspeFunctionCall

Yes - although I can't say enough how bad an idea I think trying to dynamically allocate data from C is. Can you not just statically allocate everything you need? In which case it's all handled for you by E.compiledC.

It might make sense to have a libespruino that implements malloc/free/sin/cos/sinf/cosf/etc by calling the JS implementations.

I just feel like you're trying to do something that this isn't designed for at all. As @fanoush says it was really only designed for the odd function, and even if you did get your project working and release it, my poor server's going to suffer if loads of people start downloading and forcing it to compile the same big project all the time.

I wonder whether really what you need is almost a separate project with a Makefile so you can make it locally, and then when it's done just upload the base64 of the compiled code into the app and link to your repo?

There are definitely things you can do, for instance when starting up you could call peek32(E.getAddressOf(Math.sin)) and so on from JS, which will get you the address of the underlying functions, which you could then pass into C to do what you need.