Debugging c code tips?

I can agree with that. I'm currently using my pico as a program because i found it easy to set up and more reliable then my fake JTAG programmer.

I'm surprised you hit issues with --overlap=replace - my understanding was that's only really needed if the hex files overlap - and if they're overlapping generally that's probably going to cause you some problems as something is too big?

On the debug front, it's a nightmare with nRF52 because the softdevice tries to restart if it realises the CPU is halted for much time. There is a wdt_off GDB macro at https://github.com/espruino/Espruino/blo­b/master/.gdbinit#L31-L33 that you can call the very first time you break execution after the softdevice is started and I believe that will stop the reboot, but bluetooth connections will still completely break as soon as you start breakpointing stuff.

There is a wdt_off GDB macro at https://github.com/espruino/Espruino/blo­b/master/.gdbinit#L31-L33 that you can call the very first time you break execution after the softdevice is started

Well that is useful if one has watchdog started but is not so much related to softdevice
https://infocenter.nordicsemi.com/topic/­com.nordic.infocenter.nrf52832.ps.v1.1/w­dt.html?cp=5_2_0_39_3_6#register.CONFIG - there is a bit that pauses watchdog when being debugged. EDIT: however I am not sure what writing value 1 done by wdt_off there can fix, from description clearing both bits 0 and 3 could help?

Found the PRIMASK stuff, discussed here https://devzone.nordicsemi.com/f/nordic-­q-a/477/can-i-debug-ble-program/2498 with some gdb hooks EDIT: oh i see now that it is already there in .gdbinit

I'm surprised you hit issues with --overlap=replace

overlap=METHOD
What to do when data in files overlapped.
replace -- use data from last file that contains data at overlapped address

That does sound very bad :/

I should probably look into that.

So commenting out a few parts i managed to get to jsiInit in the main.c file.

I'm still trying to figure this out, but this isn't were it laod the banglejs_storage is it?

Currently i'm working on modifying the bangle code project.

jsiInit calls jsiSemiInit which in turn does load from flash...

I think jsfIsStorageValid is the first use - it tries to read from storage and if it seems corrupt in some way it tries to pop up a recovery menu

Ok so i don't think i'm running into a "error" --overlap. It seems like it part of the build script?

So there seems to be two types of defines

• BOOTLOADER
  
• USE_BOOTLOADER
  
  

And in my python board file i have 'bootloader' : 1. This one is sets USE_BOOTLOADER to true.

As for BOOTLOADER

• BOOTLOADER=1 make the bootloader (not Espruino)
  
  

link to code

$(PROJ_NAME).hex: $(PROJ_NAME).app_hex
 ifdef USE_BOOTLOADER
  ifdef BOOTLOADER
	@echo Bootloader - leaving hex file as-is
	@mv $(PROJ_NAME).app_hex $(PROJ_NAME).hex
	# for testing:
	[#python](https://forum.espruino.com/sea­rch/?q=%23python) scripts/hexmerge.py --overlap=replace $(SOFTDEVICE) $(PROJ_NAME).app_hex -o $(PROJ_NAME).hex
  else
	@echo Merging SoftDevice and Bootloader
	@# build a DFU settings file we can merge in... family can be NRF52840 or NRF52
	nrfutil settings generate --family $(BOOTLOADER_SETTINGS_FAMILY) --application $(PROJ_NAME).app_hex --app-boot-validation VALIDATE_GENERATED_CRC --application-version 0xff --bootloader-version 0xff --bl-settings-version 2 $(OBJDIR)/dfu_settings.hex
	@echo FIXME - had to set --overlap=replace
	python scripts/hexmerge.py --overlap=replace $(SOFTDEVICE) $(NRF_BOOTLOADER) $(PROJ_NAME).app_hex $(OBJDIR)/dfu_settings.hex -o $(PROJ_NAME).hex
  endif
 else
	@echo Merging SoftDevice
	python scripts/hexmerge.py $(SOFTDEVICE) $(PROJ_NAME).app_hex -o $(PROJ_NAME).hex
 endif # USE_BOOTLOADER

```

So i tried to set the bootloader to stop the merging hex files.
make BOARD=P8-SDK12-SPIFLASH -j8 BOOTLOADER=1

But then i get a overflow issue region RAM overflowed with stack
:/

what are you trying to achieve?

When I want to save time and flash wear by skipping the softdevice and bootlader in the hex file I copy paste the mergehex line when the build ends and delete the softdevice and bootloader hex from that (and often change destination) so it becomes
python scripts/hexmerge.py --overlap=replace bin/espruino_2v19.695_banglejs2.app_hex obj/dfu_settings.hex -o /mnt/c/pathtoopenocd/espruino_2v19.695_b­anglejs2.hex (as I build inside WSL but run openocd in windows)

So i tried to set the bootloader to stop the merging hex files.
make BOARD=P8-SDK12-SPIFLASH -j8 BOOTLOADER=1

that should build just the bootloader, you can't use that to avoid merging hex files for the application

If it helps:

• 'bootloader' : 1 sets USE_BOOTLOADER as you noted - this means that the firmware is arranged such that a bootloader exists at the end of flash memory as well. When building, both a bootloader and the firmware will be built and then merged into a hex at the end (unless you do DFU_UPDATE_BUILD=1` in which case it just creates an update zip file
  
• BOOTLOADER=1 forces it to build just the bootloader hex file
  

Sorry for being very vague and just asking single ended questions. My plan was to get something working then make a formal right up about what I'm trying to do, to see if it worth the effort. But like everything its always harder then it seems

Primary Goal

I'm trying to use @fanoush pinetime python build file and make it a "real" fake bangleOS watch.

Bigger picture

Make a way to port watches easier to bangleOS

Why

Related this topic. My thought would be if all the watches used the same bangleOS then it would be easier to make apps works across devices. There seems to be many devices that can use Espruino but are not using Bangle primarily because its hyper optimized to work with the watches being sold to actual fund this project. Which it completely fine.

How

So my thought is to have 2 builds that are function compatible with bangleOS. So the official one and the secondary one that is more like a "c compiled time driver version" that's unofficial. So there would be a layer of abstraction on each type of hardware device and at compile time you would selected the types of hardware from the configuration file. What this looks like now is just a copy of jswrap_bangle.c just stripped to the function calls and I've started to make a few stubs for the display driver.

Problem

I'm having a hard time getting to the point were i start actual trying to my idea's. Currently i can't seem to even build it. This leads the above problem i had were if i had the "BOOTLOADER=1" for some reason i get a error saying the I'm overflowing the RAM. I tried to remove just about everything and even faking the ram size in the python file, but that didn't help. So I'm thinking of starting from the top and trying it again and see what i did wrong.

if i had the "BOOTLOADER=1" for some reason i get a error saying the I'm overflowing the RAM.

That is odd because this is just a bootloader = separate small app, by default mostly just Nordic code
https://github.com/espruino/Espruino/tre­e/master/targets/nrf5x_dfu
Well, unless you add display configuration with display type that is supported by bootloader or additional flags (like BANGLEJS) I don't see why bootloader RAM would not fit.
I mean stuff like this
https://github.com/espruino/Espruino/blo­b/master/boards/BANGLEJS.py#L109
https://github.com/espruino/Espruino/blo­b/master/boards/BANGLEJS.py#L39

the RAM size in board file is just for info, what is important is the linker files, for bootloader itself the RAM you are overflowing is defined here
https://github.com/espruino/Espruino/blo­b/master/targetlibs/nrf5x_12/nrf5x_linke­rs/secure_dfu_gcc_nrf52.ld#L17
for main app with bootloader:1 in board file it is here
https://github.com/espruino/Espruino/blo­b/master/targetlibs/nrf5x_12/nrf5x_linke­rs/linker_nrf52_ble_espruino_bootloader.­ld#L13
This is just for info, there is nothing needed to change there :-)
If you use SWD exclusively you may put bootloader:0 for now there for simplicity to skip building it, this will however let espruino extend to areas where bootloader normally is and possibly break it. Could work if you mass erase the device to clear bootloader address in UICR or if you modify the non bootloader espruino linker file
https://github.com/espruino/Espruino/blo­b/master/targetlibs/nrf5x_12/nrf5x_linke­rs/linker_nrf52_ble_espruino.ld#L12
to end like the one for bootloader.However maybe it is bettter to keep things as they are and figure out why it does not work.

It may help to run provision file . scripts/provision.sh BANGLEJS to get verified gcc compiler and use that one, older or newer versions produce larger code that may not fit (into FLASH, not RAM though)

As for the big picture, well, that's a great goal :-)

Good news

I was able to get this build to actual work.
So I'm using the bootloader : 0 for now. I'll deal with that problem at another time.

So know i'm sorry i don't understand why this work but it seems to work. I was getting random crashes at place and i wasn't allowed to use any of the bluetooth function and these problems seemed to all go away when i did this.

@echo "load bin/espruino_2v19.88_p8_SDK12_SD30_SPIFL­ASH.hex" >> gdbinit
@echo "load bin/espruino_2v19.88_p8_SDK12_SD30_SPIFL­ASH.elf" >> gdbinit

If i used just one there would be problems in different spots.

When you flash it to the board you get mostly different things flashed?

hex

Loading section .sec1, size 0x964 lma 0x0
Loading section .sec2, size 0xf000 lma 0x1000
Loading section .sec3, size 0xebf0 lma 0x10000
Loading section .sec4, size 0x1000 lma 0x1f000
Loading section .sec5, size 0x10000 lma 0x20000
Loading section .sec6, size 0x10000 lma 0x30000
Loading section .sec7, size 0x10000 lma 0x40000
Loading section .sec8, size 0x10000 lma 0x50000
Loading section .sec9, size 0x77d4 lma 0x60000
Start address 0x1f7bc, load size 421160

Elf

Transfer rate: 19 KB/sec, 13161 bytes/write.
Loading section .text, size 0x4871c lma 0x1f000
Loading section .ARM.exidx, size 0x8 lma 0x6771c
Loading section .data, size 0xa0 lma 0x67724
Loading section .fs_data, size 0x10 lma 0x677c4
Start address 0x1f7bc, load size 296916

But now i can actually bluetooth to the board. So i call that a huge win.

Hi - well, that sounds like a great start. I'm not sure if it helps, but I just manually flash then use this in gdb:

file bin/bootloader_espruino_2v19....elf

Rather than load - maybe load actually causes all the flash to be cleared, so since the elf doesn't include the softdevice it crashes?

The end result you're after sounds great, but personally I feel like you're maybe making this into a huge bit of work when it may not have to be - and making vast changes to jswrap_bangle.c that won't be in the official version is basically going to result in the two builds getting being forked which almost feels worse than the current situation.

So am I right in thinking that you're trying to port to pinetime/p8? Isn't it the case that basically all the hardware (LCD, touchscreen, accelerometer, etc) is already supported by the drivers that are built in to jswrap_bangle?

So if you defined BANGLEJS and BANGLEJS_P8 in the board.py file, and all the device entries there are filled in properly, and you add an init section to jswrap_banglejs_hwinit you should be a pretty long way towards having it working?

I know having the code for all devices in the one file isn't ideal, but you may be able to add P8 support by adding less than 100 lines to jwrap_bangle, which I'd be happy to upstream.

At least at that point you should have a pretty clear idea what had to change, and so what would be sensible in terms of moving things about to make it easier.

Personally I would like to move device drivers each to their own files (for instance Puck.js/Microbit could maybe share accelerometer code with Bangle.js), but it's quite a hard problem to be able to do that in a way that is efficient and also doesn't break anything when sometimes devices work in subtly different ways.

First i just want to say, it think @Gordon approach is the best open source watch platform by a long shot. These thoughts are just my attempt at making it more universal.

2 repo's worse then a single one

So this is true that there would be some copying from the main repo to the generic.

But to help mitigate this what i would do is move all the JS wrapper functions calls to separate file and they would just call a single c function. If you remove all the code and just leave the JS wrapper calls its around like 1800 lines of code that would be shared. Since it's a single call it will be optimized away.

Also there's a good amount of device specific code that wouldn't need to be migrated over. So its only code thats completely generic.

Code placement

So in a perfect world. Where i would like this is.

Espruino/libs/banglejs/alt_banglejs

This folder would be a separate github repo. So it wouldn't mess with the main code. But it could benefit from all the changes from the general espruino improvements.

In this folder you could also put the pythonBoard files

Hard problem of devices working differently and being efficient

So the approach i'm thinking of is going to 100% reduce some efficiency's. But all abstractions will cause this. But the lack of abstraction is probably part of the reason why not many watches have been ported to banglejs. Know i don't know if its worth it, But i think its worth a look into how it might be done. That's why i'm trying to make a little test bed and play around with it.

I know having the code for all devices in the one file isn't ideal, but you may be able to add P8 support by adding less than 100 lines to jwrap_bangle, which I'd be happy to upstream.

At least at that point you should have a pretty clear idea what had to change, and so what would be sensible in terms of moving things about to make it easier.

Might be interesting to see, personally I am a bit skeptical about this approach but still it may be worth trying :-)

IMO there are already too many watches and devices supported in jwrap_bangle so in theory there should be plenty of material already there for the 'clear idea'. I fear that adding yet another one in the same way won't help anything. OTOH trying to add another one in new, different, modular way can change the trend.

But the lack of abstraction is probably part of the reason why not many watches have been ported to banglejs.

For me it is one of the reasons.

Another one is that I think there is too much stuff hardcoded in C. I came to Espruino because I did not like this way of developing in the first place :-) I'd prefer to have more stuff done in JS and if that is slow then improving the engine or moving only the speed critical (and/or as generic as possible) code into the C side. So all the pin numbers, LCD initialization data, memory allocations (like framebuffer) could be called from some js outside of the main binary. In linux kernel they use device tree (dtb files) or kernel modules, here small pieces of js/json could do the same thing with configuration or even basic management of the hardware.

However this way is worth it when there are many similar devices and the flexibility is actually wanted. When you are selling a product then you probably want to limit ways how it can be 'broken' by your customers or 3rd parties (=applications). And when you are targeting end users who are not developers then any such flexibility may backfire or is simply not appreciated.

So there are different goals. The way other watches are currently working is IMO fine in principle. The bigger issue is focus, if you are doing it for fun as a hobby in spare time then it is not so polished. Also there is no reliable long term source of these watches so stuff gets outdated and abandoned. Also in my case I see the issue that most of these watches are fun for tinkering but I would actually not like to wear them daily so one less reason for polishing :-) I like ones with always on screen, preferably small light so that's Q3/Bangle 2. The only other watch I liked and used to wear is DK08 and I can imagine wearing G5 too (even if it is a bit bigger).

i think its worth a look into how it might be done. That's why i'm trying to make a little test bed and play around with it.

That's great!

I think there is too much stuff hardcoded in C

Yes, it's a hard one - when developing something like Bangle.js I start out making it work in JS too, and then end up having to port to C. It's really just trying to get to a point where you have some device that is almost impossible to brick regardless of what you do in JS - but also even if JS could be made much much faster it's still going to struggle with things the the LCD that want to be fast (and perhaps even transfer data in the background like we do for Bangle.js)

In an ideal world I'd really like the BOARD.py file to be able to describe the entire device, and to then have the relevant driver files pulled in and made to work without ifdefs everywhere - but I think in many cases these watches to strange stuff (like sharing I2C and SPI wires!) that might just make that impossible.

move all the JS wrapper functions calls to separate file and they would just call a single c function

Obviously you're welcome to do whatever you want, but I wouldn't be happy about pulling something that stubbed every function back into the main repo - while yes it'll probably get compiled out, it just makes everything harder for developers who then have to try and follow a series of stubs to get to the actual code. This has been done in the ESP32 port and it just makes trying to work on it a nightmare: https://github.com/espruino/Espruino/blo­b/master/targets/esp32/bluetooth.c#L258

For the actual JS functions, we have the ability to add a JSON header that overrides previous ones, and we do this on DICKENS (the Starfield Collectors Edition watch) so that could be an option for some stuff:
https://github.com/espruino/Espruino/blo­b/master/libs/dickens/jswrap_dickens.c#L­30-L34

Personally, I think that stuff to do with the actual JS interface (like Bangle.setOptions/etc) that's going to be mostly common should always stay in jswrap_bangle.c but if it were possible to pull out the actual hardware-related code into separate files, that would be good.

But I'm very worried about copying stuff out in such a way that you then end up duplicating loads of code, because then it makes maintenance a nightmare. What if I want to change how buzz works but now there are 10 identical jswrap_banglejs_buzz implementations spread across the Espruino and alt_watch repos?

In many ways just forking the repo and changing jswrap_bangle.c for the new watches is preferable over duplicating code over multiple different files - at least then if there's a fix for something in the main repository you stand a fighting chance of being able to merge that in - but if the code end up duplicated all over the place it's going to make keeping everything (especially if it's in a separate repo) up to date almost impossible.

Once again i appreciate the amount of time people take to actual respond to this. So I'll try not to waste to much more of your time, since i think it will probably be more meaningful discussion once i have at least a working display.

But hopeful these responses will make you slightly less worried. Granted your still rightful to be skeptical of it working. I'll probably find some big bugs once i get into this more.

But I'm very worried about copying stuff out in such a way that you then end up duplicating loads of code, because then it makes maintenance a nightmare. What if I want to change how buzz works but now there are 10 identical jswrap_banglejs_buzz implementations spread across the Espruino and alt_watch repos?

My "goal" is that it wouldn't really matter all that much how bangleJS worked. I'm trying to encapsulate hardware. So if you imagined jswrap_bangle.c without most of the defines. That's what i would like. Granted defines will still probably be used just in a slightly different way. I just want to separate the larger state machine of bangleJS from the unique features of each device.

So for you example about buzzer, i haven't tried to make it but from the states it might look like this.

• buzzer_beep()
  
• buzzer_buzz()
  
• buzzer_vibrate()
  
  

These functions would be overridable if they needed to be. Granted a thing like a IMU will probably need a more advance abstraction because it's more complicated.

So yes i also wouldn't accept the idea of if you changed the Buzzer, backlight or other hardware that it would cause a huge backlog of devices to not be useable. I understand that nobody would ever want that level of burden. Especially a open source project.

while yes it'll probably get compiled out, it just makes everything harder for developers who then have to try and follow a series of stubs to get to the actual code. This has been done in the ESP32 port and it just makes trying to work on it a nightmare

Your probably right on this. How would you feel about moving the true pure javascript stuff out.
Ex.
here to here

hereto here

How would you feel about moving the true pure javascript stuff out...

You mean the JSON comments that describe what's in Bangle.js? Well, for every other bit of Espruino we have jswrap_X.c files that contain the JSON descriptions of the methods/fields of X.

So I feel like in a way it'd be a shame to change that.

I guess I think the file's pretty much always going to be so big you need to use your code editor's search and Ctrl-click functionality to jump to declarations, so I'm not really sure if that has a huge benefit?

But yes, I think we're basically on the same page - I'd like jswrap_bangle to be tidier, but to implement the code that implements the functionality of the Bangle object. Then when stuff can be pulled out without duplicating code we could do that.

IMO even keeping as-is and splitting the drivers for various bits of hardware (like accelerometer/pressure/etc) into their own files would be a big benefit.

Ideally just specifying an accelerometer type in BOARD.py would be enough to compile in the relevant driver.c file and use it, but that feels a way off!

This conversation can continue here