Development Questions/Comments

I didn't know the best forum to post this - so feel free move it if a better place.

Short background. I am a full time programmer. For the last 2/3 years I have been leading a team using ReactJs (modern javascript framework) with Typescipt to build large scale multi platform apps.

I have spent the last 4 months as a fulltime parent and have been looking for some projects to do around that. I have worked with dev boards like the esp32 with both Arduino and more recently Micropython for a long time so although I don't do this type of development for a living I am very familiar with it.

I bought the bangle 2 watch a few weeks ago. The Javascript programming framework looked interesting. I liked the idea of open source software I can improve - mostly around improving the existing apps for hiking and navigation.

I have some comments around the dev framework which may well be ignorance, any criticism is meant as constructive rather than an attack on the framework and community. Any help or feedback would be grateful.

One of the starting points is the documentation. I am finding navigating the documentation confusing, often getting circular links from one page to another. Some of the questions below may be answered in the documentation but I haven't been able to find it (it also doesn't help when a small baby keeps interrupting)

1. Its not clear to me which version of Javascript espruino is supporting. es5, es6 .. etc? Or some cut down version supporting some of the functionality of one of these (as with micropython). I would be planning to use typescript and smaller single files of typescript in an offline ide, which could be built down to a single or multiple javascript files, if the compiler is custom javascript the use of typescript is limited.

2. Coding styles. There is a small section on suggested naming conventions, syntax structure etc. Has anyone considered having a defined linting file for people to use? You can even run forced linting when code is submitted to github. With an extreme implementation rejecting files not parsing certain linting rules.

3. Folder structure. Having boilerplate folder structure for new applications is really useful. Especially for beginner programmers. I can even make a cli in javascript to do this if there is demand. Although I am guessing the web ide makes chucking everything in a single file easier.

This questions come from looking at the documentation and existing code library. Having looked at existing navigation apps within the github repo. There is alot of very poorly built code. Long single files of functions (or in some cases not even split into functions). Lack of object orientated code (not sure if classes supported in the compiler). Use of timers instead of async functions. All of which is understandable for any hobby developer and are one of the issues with scripting languages over ones like c# which force better code by design.

The issue for me is that I can't extend any of the existing applications. I would need to rewrite the base code into a structure which would be maintainable and upgradable in the future. Without the use of linting and type safe code (typescript) building any reasonable size application is difficult. If I was rewriting one of these apps the first thing I would do is split the code into much smaller object orientated files. Splitting any generic code related to the hardware to a separate library project.

Was the original intention that espruino dev boards/watches will always just be for simple hobby type projects. Or was there a hope that atleast on the watch side users would build up libraries of really useful everyday apps? If the later is true, nudging people towards object orientated approaches, code split into smaller files, supporting additional layers like typescript becomes hugely beneficial. Third party function libraries you can just import with specific functions around the hardware sensors etc would allow quicker development and code reuse.

Let me try to answer at least some of your questions:

1. try https://www.espruino.com/Features (In short no spread operator or optional chaining)
   
2. Boiler plate: When I add a new app I often copy from pages like http://www.espruino.com/Bangle.js+App+Se­ttings
   However I do not find it useful to automatically have a settings-file in every new app I create. If the settings-file is unused it just creates useless overhead. While there is some annoying overhead when creating a new app (metadata, icon add about 5 minutes of work) I currently do not see much potential of improvement here.
   
   

Long single files of functions (or in some cases not even split into functions)

While I can not really argue against refactoring code into extra functions: Please note that every function you add generates some overhead = slower program. In the (I think it was the) calendar app I once inlined the code of a function I previously extracted because it slowed the execution by 100ms (It was called multiple times).

Lack of object orientated code

Needs more memory, Bangle.js 2 has 256kB of RAM, Bangle.js 1 has much less and we already are at the limit the memory can handle, e.g. the run app on Bangle.js 1.

Third party function libraries you can just import with specific functions around the hardware sensors

What would you change e.g. for GPS? You have setGPSPower(), isGPSOn(), getGPSFix() and the GPS event on the Bangle-Object. What would you make different? There are also some extra modules like modules/buzz.js

Lets take the Anton clock as an agument against, there are two versions of it: Anton Clock and Anton Clock Plus. The latter includes some extra code that you notice when starting the clock (feel free to try it out).

Do you have a specific (small) example, e.g. an app where you would refactor something? Or maybe comment on an existing pull request on what you would make different? (However I can not speak for all authors if they want your comments)

Disclaimer: I'm not an espruino expert, I'm just someone who made one or two changes to Bangle.js apps.

I see your points, I'm just skeptical that every refactoring someone would make in a Browser/Node.js environment is a good idea in a resource constraint one.

Hi,

I might not give the "best" answers but I'll chime in with some anyway.

One of the starting points is the documentation. I am finding navigating the documentation confusing, often getting circular links from one page to another.

Maybe not exactly what you're talking about, but there's this issue for improving the docs: https://github.com/espruino/Espruino/iss­ues/1498

Its not clear to me which version of Javascript espruino is supporting. es5, es6 .. etc?

This issue at the espruino repo might me somewhat useful: https://github.com/espruino/Espruino/iss­ues/1302

Folder structure. Having boilerplate folder structure for new applications is really useful. Especially for beginner programmers. I can even make a cli in javascript to do this if there is demand. Although I am guessing the web ide makes chucking everything in a single file easier.

There are these example folders:

• https://github.com/espruino/BangleApps/t­ree/master/apps/_example_app
  
• https://github.com/espruino/BangleApps/t­ree/master/apps/_example_clkinfo
  
• https://github.com/espruino/BangleApps/t­ree/master/apps/_example_widget
  
  

Not sure myself where those are mentioned in the docs. EDIT: Mentioned in the BangleApps repo readme.

Use of timers instead of async functions.

Maybe this is relevant (I'm not sure): https://github.com/espruino/Espruino/iss­ues/1302#issuecomment-356257549

Cheers!

Edit - bonus links (I guess you already saw these):

• http://www.espruino.com/Performance
  
• http://www.espruino.com/Bangle.js+Guidel­ines
  
• http://www.espruino.com/Tutorials#inform­ation
  
• http://www.espruino.com/FAQ#is-there-a-c­oding-style-i-should-use-
  

Besides the performance and RAM reasons for keeping the code "flat" there should also be taken into consideration that there are users who are technical novices just starting out to code using the espruino environment. Too much rules and focus on "best practice" could maybe deter those from even trying. There are however some apps already using typescript, so I do not see a problem in trying to get better code quality for some apps while keeping in mind the resources (especially in Bangle.js 1) and the beginners just starting out.

Thanks for the long response, its really helpful. Re timers - its not in the documentation, just some of the apps using them. It was one of my less constructive comments as unless the bangle js compiler doesn't support async functions, its an understandable coding mistake, as async stuff is often harder to get head around.

Appreciate you taking the time for this response.

While I can not really argue against refactoring code into extra
functions: Please note that every function you add generates some
overhead = slower program. In the (I think it was the) calendar app I
once inlined the code of a function I previously extracted because it
slowed the execution by 100ms (It was called multiple times).

While this might be true in alot of real world projects, there isn't any direct correlation between oo approach and slower code. Javascript was never designed for it, which makes it harder to do well.

What would you change e.g. for GPS? You have setGPSPower(), isGPSOn(),
getGPSFix() and the GPS event on the Bangle-Object.

I was thinking higher level. I can give good examples once I look at the source code more and start making an app - which will almost certainly result in a side library for gps stuff.

Those examples you give are of the core functions. So firstly on those. If you call getGPSFix() and gps power has not been set on, what happens? I wouldn't suggest changing the core function to auto turn it on, as may be a reason you call getGPSFix without turning on GPS. However in most cases I imagine turning GPS on first is a requirement. So if create a higher level object like a GPS manager, calling myGpsManager.getFix() would check if GPS is currently on, if not set it on first.

At even higher level, its likely alot of apps actually want to do the same thing once they have GPS location, like overlay position on an openstreetmap tile. So having pure helper libraries which are not apps themselves, just code to import can massively speed up new projects.

I see your points, I'm just skeptical that every refactoring someone
would make in a Browser/Node.js environment is a good idea in a
resource constrain

Will come back with examples once I have started building an app. I think some of the stuff is that I am coming from a world of large scale apps, typescript with webpack or similar to tree shake and remove unused stuff, minify if needed, flatten to single file if needed.

Besides the performance and RAM reasons for keeping the code "flat" there should also be taken into consideration that there are users who are technical novices just starting out to code using the espruino environment. Too much rules and focus on "best practice" could maybe deter those from even trying. There are however some apps already using typescript, so I do not see a problem in trying to get better code quality for some apps while keeping in mind the resources (especially in Bangle.js 1) and the beginners just starting out.

Some of this depends on if you using typescript/webpack etc to turn your files into production code. It also depends on which version of javascript as it is shifting slowly towards only functions being used from a file being imported without needing the likes of webpack to do this.

Even without any of that, there shouldn't be any difference in RAM usage between code split into multiple files and shoved in a single file.

Those examples you give are of the core functions. So firstly on those. If you call getGPSFix() and gps power has not been set on, what happens? I wouldn't suggest changing the core function to auto turn it on, as may be a reason you call getGPSFix without turning on GPS. However in most cases I imagine turning GPS on first is a requirement. So if create a higher level object like a GPS manager, calling myGpsManager.getFix() would check if GPS is currently on, if not set it on first.

Well for the GPS example: If on GPS will drain the battery in ~4 hours. Also after turning it on it needs ~30 seconds (can be much more, up to nearly 15 minutes or a bit less, simplified for this example) until you receive valid data. So what a user usually does not want is

• Have GPS powered on randomly
  
• Enable GPS, read data, disable GPS
  
  

Instead I want to control when to power on GPS or not. When reading the position continously I just want to power on GPS on start and never disable it again (until the app closes). If the use case is to get a position every hour or so the myGpsManager.getFix() (as promise I assume?) might be ok, but if needed can be written a less then 5 minutes, no need to import a big library with maybe 100 functions that are unused.

At even higher level, its likely alot of apps actually want to do the same thing once they have GPS location, like overlay position on an openstreetmap tile.

Currently the only use case I see for overlaying the current position on a map is the openstmap-app and it already does that. Well you could maybe combine "GPS Navigation" or "GPS Trekking" with "openstmap" e.g. overlaying current waypoint and your position on a map (your turn @halemmerich) but you will quickly run into memory constraints, at least on Bangle.js 1.

Will come back with examples once I have started building an app.

Cool every PR brings the ecosystem forward.

There are functions in the GPS context that would do well and be useful in a module, for example compass noise filtering, handling access to track data more encapsulated, "sensor fusion" between compass and GPS and lots more of smaller things that could be useful outside of their apps. Some refactorings in this spirit I would like to do to GPS Trekking.
When developing for the Bangle, there are some tradeoffs to make. One big part of the execution time is actually parsing the javascript text. So the "form" of your code directly influences execution speed. I have written Imageclock and learned the hard way how expensive function calls and verbose code are. In there I started with recursively parsing JSON and rendering that as a watchface. That was so slow that watchfaces with seconds just weren't possible. "Flattening" that tree of watchface components in the browser and generating the rendering code for the Bangle got that several times faster for some watchfaces. It was calling exactly the same rendering code for the elements, just no longer recursively traversing the JSON saved that much execution time.
Another example would be the Layout module. It's great for easy creation of layouts, but if you need to refresh often or are doing memory intensive stuff otherwise you will probably run into problems with either speed or RAM usage. That's not to say Layout is not a great module, I try to use it whenever possible. But it shows that on a platform this constrained there won't be a way to make every code nice and tidy, sometimes getting to the edges of available resources means ugly code or hacks to even get it to work at all.

Instead I want to control when to power on GPS or not. When reading
the position continously I just want to power on GPS on start and
never disable it again (until the app closes).

If it really takes up to 15mins to get a fix its not really usable for much anyway. I assume its quicker via the android integration app. If it is draining the power quickly you kinda do want some sort of intelligent power off. Like if the person hasn't moved in a while getting a continuous fix doesn't make sense. Either way you could supply options to the manager when you initiate it.

If the use case is to
get a position every hour or so the myGpsManager.getFix() (as promise
I assume?) might be ok, but if needed can be written a less then 5
minutes, no need to import a big library with maybe 100 functions that
are unused.

This is why you use tree shaking and don't import the entire library. The idea that because a function takes 5mins to write you shouldn't split it into a helper library for reuse makes little sense. If you are designing good code most functions shouldn't take a long time to write, something which becomes very apparent when writing unit tests. Having helper libraries allows you to have an initial implementation covered by unit tests which can be updated in future, with all projects using that lib getting the benefits of the update.

But it shows that on a platform this constrained there won't be a way
to make every code nice and tidy, sometimes getting to the edges of
available resources means ugly code or hacks to even get it to work at
all.

At this stage I am probably underestimating the memory constraints of the system. I write evolutionary algorithms (type of AI) for scheduling software for my day job. Some functions get called millions of times during the evolutionary process. Sometimes having to sacrifice nice code for speed so I understand why you would.

Given the hardware constraints I am starting to see why javascript might not be the best language for watch development.

I did stumble upon a project in the forums where someone had been trying to run reactjs apps in bangle. It doesn't look like the project code has been updated in 4 years. Did they basically give up at some point?

I'm on holiday at the moment so I don't have much time to read into your questions fully and answer in detail, but it feels like others have done quite a good job.

I just want to add that Bangle.js is very intentionally designed to keep things simple, to cram as much as possible into what is quite a constrained device. We add features and APIs that allow you to do new things, but don't generally spend time on adding features that you could have just implemented in JS.

there shouldn't be any difference in RAM usage

Even if you know for certain this is the case on Node.js on desktop with gigabytes of RAM, it may not work that way on Bangle.js - or 'not a big difference' on desktop (eg 200kB extra memory) may translate to quite a big issue on Bangle.js with more limited resources.

Some of the differences with a desktop JS engine are documented here but honestly the best bet is to have a play, do some benchmarking, and figure out whether things really can really be improved by doing things a different way.

At this stage I am probably underestimating the memory constraints of the system.

quite likely, and maybe also CPU speed and I/O speed, check this https://www.espruino.com/Performance to get better picture

Given the hardware constraints I am starting to see why javascript might not be the best language for watch development.

I don't see anything better that offers the simplicity, flexibility and extensibility while still having low memory requirements. Also the asynchronous nature of js make things easier than using languages like micropython or lua.

To give some indications for speed/RAM:

1. Allocation of one full screen 4 bit buffer on Bangle.js 2 would be about 10% of the RAM. On Bangle.js 1 probably more like about a quarter.

2. 1000 iterations of an empty for loop take nearly 230ms

3. Assigning a variable on each iteration costs 136ms extra

4. Commenting out that assignment still costs 24ms over the base loop

As for javascript as the language of choice I'm with @fanoush. You could use C/C++ like on other micros, but that will cost a lot of convenience and simplicity. Nevertheless you can even include native/compiled code in the JS if you need every last bit of speed the processor can muster.

I don't see anything better that offers the simplicity, flexibility and extensibility while still having low memory requirements. Also the asynchronous nature of js make things easier than using languages like micropython or lua.

Not sure I agree with the async part, I think the introduction of async await layer to wrap promises helps, but promises are a fairly hard concept to get head around if new to programming (and the code gets very messy if still use promise.then syntax), something which is apparent in user app code when timeouts are being used instead of async functions, and I suspect are causing some of freezing issues I have seen briefly seen in the forum)...but I am a python fanboy so I will defend it against all reason.

I'm on holiday at the moment so I don't have much time to read into your questions fully and answer in detail, but it feels like others have done quite a good job.

They have, you have a great community.

I just want to add that Bangle.js is very intentionally designed to
keep things simple, to cram as much as possible into what is quite a
constrained device.

I think you have done an amazing job and anything which opens up programming to a wider audience is great.

Some of the differences with a desktop JS engine are documented here
but honestly the best bet is to have a play, do some benchmarking, and
figure out whether things really can really be improved by doing
things a different way.

Yeh I am surprised assoc arrays are implemented via linked lists and not hashmaps. I have a few performance questions, but maybe best in a private message or separate thread...and like you said if I play about with it and do some benchmarking I will have better idea.

but I am a python fanboy so I will defend it against all reason.

then maybe look into wasp-os https://github.com/wasp-os/wasp-os it is an attempt to do something similar with micropython

... there is also a circuitpython port for Bangle.js if you want to go that route, but as far as I know the framework needed for apps/etc isn't there yet - so you'll be writing pretty much everything from scratch.

I am surprised assoc arrays are implemented via linked lists and not hashmaps.

Since we use fixed-size memory elements (to avoid delays from malloc/etc and fragmentation) hashmaps don't fit that well - not to mention keys usually being added to objects incrementally. I did have plans for a red/black tree (which would still work) but the JS interpreter was designed for 64kB RAM and under, and then you're looking at the most you could ever have in an object being 3000 items (because that would fill all memory), which you can actually iterate over pretty fast.

In reality most code is written with some level of tree-ness (even if that's only nested scopes) so there aren't a great deal of elements in each object and a tree or hashmap wouldn't provide a massive pay-off.

PRs to the interpreter are always welcome if you find a way to hugely improve things though ;)

It's easy to look at decisions in Espruino from the outside and pick holes in it. It's a reason the Performance page is a bit defensive - it's been routinely used for Espruino-bashing online, but at the end of the day Espruino does manage to provide a reasonably convincing watch experience on Bangle.js - it may not be how most people would do it.

I have now spent some time developing my hiking app on the platform so I have alot more understanding of how everything fits together. Its worth noting I bought a bangle 2 so any memory references are related to this rather than the first one. I will publish it to the app store once more complete.

See screenshot for how I structured the project. Everything is typescript and packed with webpack into a single app.js minified file. When working with limited mem this has the advantage that things like tests and mocks are not included in the prod build as well as any unused files or functions.

The app provides a vector drawn route relative to current location and bearing, direction needed to hit next waypoint (via arrow pointer), direction at next waypoint (relative to current bearing). Distances to next waypoint/end etc (Screenshot is from the emulator which is not showing the light coloured objects).

I took some ideas from the gipy project (which is the best attempt of oo I have seen) and nav compass. I read the gpx files directly from storage. I know gipy creates its own format, but I am not sure what calculations you need to make prior to running as vector maths is super quick on most processors. It also allows you to use gadget bridge to put a gpx file on the bangle without going though the app loader every time.

Observations:

Documentation is definitely a weak point especially with built in modules and which javascript spec the compiler is following (like 'let' and 'const' are implemented but regex named capture groups is not). Most of what I now know is from trial and error and declaration files.

I only hit memory issues if stored alot of information in long running variables - like entire location history.

Assuming the garbage collection is working as would expect I would definitely use the documentation to nudge people to use small modular functions over single large code chunks. Always using "let" over "var" so variables take up memory for the smallest time possible. Keep global variables to min. I think you can do this in the documentation and tutorials in a way which doesn't over complicate things for beginners and doesn't force it.

2 Attachments

It's easy to look at decisions in Espruino from the outside and pick holes in it. It's a reason the Performance page is a bit defensive

Yes I agree, I think constructive criticism is really important for improving and opens discussion. Bashing for the sake of bashing doesn't help anyone. Anytime I question something its with this aim of improving the product or my own understanding. Something I often fail to bring across in my wording.

Its really hard to not be defensive when you spend so much time making something so this is totally understandable. Especially when alot of comments have no positive intent and often lack context on why you made a certain decision (like with the hashmaps).

I would still implement assocs as hashmaps so they are future proofed for larger mem sizes. But I have no experience in creating compilers and I probably use assocs more as a dictionary than others might, coming from python and c#. You clearly have time constraints on this stuff as well and is a difference between perfect and good enough for moment.

Thanks - that app's looking like it's coming together well!

Documentation is definitely a weak point

Again, it's a matter of lacking time. I think the actual built-in function reference is ok (maybe not the rendering, but the data itself) - and we create that from comments in the code so it's always up to date.

But if you see somewhere where documentation could be improved, please do create a PR for it. Espruino only exists in the state it is at the moment because many people contribute to it - and documentation is something that almost anyone can do, and it is hugely helpful for everyone.

I think right now there's also a big problem of actually finding information - there's quite a lot of documentation, but making it so that users find it and read it when they need to is a big problem.

But...

I would definitely use the documentation to nudge people to use small modular functions over single large code chunks

It's more complicated than that. GC does work as you'd expect, but function code (unless you explicitly mark it not to be) is kept in flash and is executed direct from there. The thing that takes RAM is the function declaration (which references arguments and where in storage the code is).

So actually having bigger functions, and defining the functions you need inside those is probably preferable, as it reduces RAM but also the amount of vars/functions that are defined at one time.

So it's important not to point people into doing what you believe is correct/optimal, without actually having tested it first.

as vector maths is super quick on most processors

there are no vector instructions in Cortex M4 (if you mean something like https://en.wikipedia.org/wiki/Vector_pro­cessor ). Even all the floating point math is done in software because Javascript uses Double type and Cortex M4 only has single precision floats. There is really nothing that could be called super quick here.

I would still implement assocs as hashmaps so they are future proofed for larger mem sizes.

Are you aware there is even no dynamic heap (malloc()/free() C api calls) available? Where would that hashmap and its data be stored?

I think you still did not grasp fully the CPU and memory limitations. Which is OTOH a testament to all the work that Gordon did to make it work so well on such limited devices. Your comments would be more fitting if we had 10 times the memory and CPU speed. Then it would be comparable to e.g. 486 or Pentium machines from nineties running Windows 3.11 with 2MB of RAM (did javascript run on that?)

BTW Espruino also works on Microbit 1 which only has 16KB of RAM and it works there including Bluetooth LE stack (!) which had to be disabled in Micropython implementation because Micropython is really not designed for limited devices like this (that's why there is also Snek).

As for 'future-proofing' it might be a bit more complicated than adding hashmaps ;-) Only recently we got typed arrays over 64KB which made no sense few years ago as typical espruino device had less than that in total.

I think constructive criticism is really important for improving and opens discussion. ..... Anytime I question something its with this aim of improving the product or my own understanding.

And BTW my replies are not meant to discourage or silence you, it is just that it still looks like your technical suggestions are missing the context a bit.

There are some other javascript engines for embedded devices that target a bit higher hardware requirements that made different design decisions. Maybe thanks to that they are more future proof or feature complete regarding JS specs but they do not run so well on the hardware we have here.

Two interesting ones I've seen sometime ago
https://kalumajs.org/ based on JerryScript
https://www.moddable.com/ - discussion https://forums.raspberrypi.com/viewtopic­.php?t=304964
Did not actually try to do something with them as they target Raspberry Pico and/or ESP32/ESP8266 which are not low power platforms like nrf52.