CODE AND STORAGE OVERLAP while building firmware for NRF52

Hi,

I tried to modify (and add) functionality to the espruino bluetooth library. Then I tried to compile the new firmware, but the "check_elf_size.py" script exited with the above error. I guess I have to update the 'saved_code' segment of the chip inside the board definition of the MDBT42Q module. However, I'm a bit puzzled on how exactly I have to do that. Can someone help me?

Thanks!

Should be enough to reduce number 10 on three lines (65,67,68) here
https://github.com/espruino/Espruino/blob/master/boards/MDBT42Q.py#L65

So:

'address' : ((118 - 10 - 10) * 4096), 
'pages' : 10 - 10,
'flash_available' : 512 - ((31 + 8 + 2 + 10)*4) - 10

???

Not sure how you mean it.

You want no saved code at all? really? so just replace those three '10' values by '0' but then you don't need e.g. Flash or Storage module. I'd keep at least 2 pages like this

'address' : ((118 - 2) * 4096), 
'pages' : 2,
'flash_available' : 512 - ((31 + 8 + 2 + 2)*4) 

this will give you 32KB more for code.

Or... Maybe just comment some lines like https://github.com/espruino/Espruino/blob/master/boards/MDBT42Q.py#L35 in order to remove functionality that you don't need?

If you modify the saved_code section then that'll reduce the amount of flash memory available for you to save JS to, which might not be what you want.

I see. What I did was to add the possibility to modify the MTU size which is something I need. I still need some storage, but not that much in general so reducing the flash memory sounds like a good option to me.

Yeah no, I still want to have a little space for my code ;) Thanks!

It'd be great if you could share your changes for the MTU size - it's something that would be worth pulling into normal Espruino builds.

I'm surprised that pushes the firmware size up though - could be you're using a different compiler? Older ones tend to produce bigger firmwares.

When talking about this, one guy hacked up SDK11 based espruino running on DS-D6 tracker to support 'long writes' , the code that worked for him is here
https://gist.github.com/sairon/788564c0c756ccbe26f3f7dd0ec1c439
As I understand it, long writes feature is some sort of transaction over multiple writes so you either get all data or nothing, would you be interested in that? He needed it because he tried to emulate another device (some game controller?) that supported it.

As I understand it to support long writes you need to handle request from softdevice to allocate memory as this feature is done by batching data and then returning you the result when it is done.
EDIT:
However the memory handling is pretty strange there (static m_buffer and also the malloc and free in BLE_EVT_USER_MEM_RELEASE) and some changes seems unrelated?

Nice - I'd be interested to find out if there's anything else that uses the long writes? I'm not sure I'd want to merge that exactly as-is, but I reckon it could be tweaked such that the memory buffer would be allocated on demand out of JsVars.

The malloc thing should really be allocated on the stack I guess (since I don't really support malloc in Espruino - I'm amazed that worked). Although honestly I don't think that's even needed - the event could be posted and then the data read directly out of the buffer outside of the IRQ.

I had a quick play with the MTU stuff and did this: https://github.com/espruino/Espruino/tree/increased_mtu

But it takes a huge chunk out of available RAM. Perhaps an increased (but not huge) MTU would still be an idea?

I haven't had a chance to test it out properly, but my hope is that I can enable it for the bootloader to try and improve the speed of flashing (but then the flasher will have to know about the MTU).

Would it be possible to create a library function that takes a maximum MTU size as parameter, and allocates the memory from jsVars? For example NRF.setMaxMTU(xxx) or something like that?
I think right now this is could be an "expert" feature, so the user either knows the needed MTU size, or can spend some time guessing experimenting. Can the devices auto-negotiate?
But if available, that could open up some new opportunities. For example my "data streaming" thingy might benefit from higher MTU size & higher bandwidth. Ok, should go and actually complete that :)

Long writes have a big disadvantage compared to a bigger MTU:

1. A long write is a normal write distributed over multiple connection intervals which is a waste of time if you have long connection intervals
   
2. They need to be supported by a given characteristic and do not apply to the whole device
   
   

If both connected devices are BLE 4.2 or higher I'd always prefer to use a bigger MTU size. This is way faster and applies to all service/characteristics which is quite handy.

iOS will automatically set MTU to 185 bytes. I think max. 200 bytes is a good size. This would be very handy if we are using the HTTP Proxy example.

Apart from that, is it correct that we would have to switch SDK to a newer version to support bigger MTU sizes? I feel like Espruino still runs on SDK 12.

They do automatically negotiate. I think though that the max MTU has to be set before initializing the softdevice so I'm unsure if you are able to change this in runtime. That's why the

NRF.requestMTU(byte size)

function would be way more useful.

Sure bigger MTU is better if it is available. Long writes look just like a workaround for small MTU so you can have characteristics over 20 bytes in length even without it. Maybe same memory buffer could be used for both since in reality they may be mutually exclusive? Having both features means that you can have same code logic if at least one (biger MTU or long writes) is supported by client.

When googling, link in this post https://devzone.nordicsemi.com/f/nordic-q-a/28388/android-sending-data-20-bytes-by-ble would suggest long writes are supported on Android and Nordic is using this feature in nRF ToolBox

Yeah long writes were the go-to feature in BLE 4.0 and 4.1, since bigger MTU sizes were not available before 4.2.

You could also combine a bigger MTU with a long write enabling you sizes up to kilobytes which is huge for a LE transport. For my special use-case bigger MTUs however would be more appealing because it would reduce the lag between a http request and its response.

is it correct that we would have to switch SDK to a newer version to support bigger MTU sizes? I feel like Espruino still runs on SDK 12.

SoftDevice 3.0 used with SDK12 should support larger MTUs
https://devzone.nordicsemi.com/nordic/nordic-blog/b/blog/posts/s132-v30-production-version-now-available-from-nor

I'll give it another spin. Maybe I also made a mistake in the implementation. Whenever I set a bigger MTU size of 23 the SDK returned me an error 7 which is INVALID_PARAM. In the docs they say it's due to the fact that the maximum MTU size is smaller than the requested one, but I did update all the maximum MTU size definitions.

That branch I posted earlier should help with the bigger MTU - unfortunately you have to request it when initialising the softdevice, and the SD then needs extra RAM. Realistically that means it's not something you can do at runtime because all variables would need to be relocated.

Even so, I think it would be worth losing a bit of RAM permanently for - maybe not to max out the MTU, but at least to go for something significantly higher than 20.

and the SD then needs extra RAM. Realistically that means it's not something you can do at runtime because all variables would need to be relocated.

as for C globals that are determined at link time those cannot be moved at runtime easily but can javascript variables be in two areas? Because it sounds interesting to restart softdevice with different parameters at runtime - like more concurrent client or server connections or MTU change.

EDIT:
Or maybe C globals could be above or right below the stack? so JS variables would be right above softdevice and could grow shrink from top to the bottom according to soft device requirements? Basically this https://github.com/espruino/Espruino/blob/master/targetlibs/nrf5x_12/nrf5x_linkers/linker_nrf52_ble_espruino.ld#L9 would start below reserved stack so all data/bss would fit and space for JS variables would be below and not part of this.

Even so, I think it would be worth losing a bit of RAM permanently for - maybe not to max out the MTU, but at least to go for something significantly higher than 20.

I second that. It would allow more flexibility.

Potentially you could do something... Maybe swapping the variables and stack so the variables started right at the top of memory and then the stack grew downwards towards the softdevice.

Sounds like one hell of a hack though - at some point it's just more sane to provide two firmware images :)

@Gordon why not have different firmware versions? Some people may not need more than 23 bytes, but some others will need it. You could have a BLE4 and a BLE5 firmware, with different features active.

Yes, that's what I was saying above. There are all kinds of things that can be added/removed.

There's already @JumJum's excellent online firmware builder at http://forum.espruino.com/conversations/334202 so even if it wasn't included by default it'd be easy to build a firmware with it enabled.

That's great! However the firmware builder seems to be offline.