-
Sun 2021.10.10
'I've noticed that after a while . . . '
@TTBangler Approx how many upload attempts?
'the Web IDE stops working'
What version of IDE and is the online one in use or the native app?
Please post
process.envfor current flashed version please.'I'm seeing this with a 250 line, ~8k file'
FWIW I have been successful at over a 30K ~1000+ line file (which would minimize to 9K with comments removed) upload to a Pico. While doing this I kept an eye on available memory.
process.memory()Once Free memory started getting below 150 JSVars, I would experience similar issues. I don't have a Bangle handy to check available memory, would you also post that. Were you checking free memory as left hand console snippets were uploaded?
-
Thank you very much for your reply Robin!
"Send to Espruino" was working for me fine until I believe my code reached a certain size. After that, it no longer worked no matter how many times I hit "Send to Espruino".
I'm using v 0.75.11 of the Web IDE (sorry for neglecting to include that earlier).
I entered process.env into the left-hand side of the IDE after running my program from storage and here's what it returned:
>process.env ={ VERSION: "2v10.90", GIT_COMMIT: "e60188501", BOARD: "EMSCRIPTEN", FLASH: 4194304, SPIFLASH: 4194304, HWVERSION: 1, STORAGE: 4194304, RAM: 0, SERIAL: "00000000-addeadde-addeadde", CONSOLE: "USB", MODULES: "Flash,Storage,hea" ... "tensorflow,locale", EXPTR: 94208 } >Hopefully that is the info you wanted to see, please let me know if you wanted something else (I obviously still have lots to learn about Espruino).
In the last line of my program, I printed process.memory() to the console and this is what was printed:
{ "free": 2271, "usage": 229, "total": 2500, "history": 0, "gc": 0, "gctime": 0, "blocksize": 13 } >My program consists of 4 functions and "Send to Espruino" works fine if I load the first two functions but when I add in the third--and by that I mean add it to the right-hand source code pane--then "Send to Espruino" no longer does anything. After loading the second function in the source code pane, and doing "Send to Espruino" that should only have loaded the first two functions (and a few global variables) into memory, process.memory() returned:
>process.memory() ={ free: 2426, usage: 74, total: 2500, history: 1, gc: 0, gctime: 1, blocksize: 13 } >If I paste nothing else but the third function into the source code pane and do "Send to Espruino", it works and the IDE tells me "No errors. Minified 6259 bytes to 2802 bytes."
Sorry but I don't really understand how to interpret the numbers returned by process.memory() or trace() for that matter . When I load my full program into my Bangle.js storage, run it and log process.memory() to the console at the end, it returns:
{ "free": 2278, "usage": 306, "total": 2584, "history": 0, "gc": 0, "gctime": 4.66918945312, "blocksize": 13, "stackEndAddress": 536925152, "flash_start": 0, "flash_binary_end": 450188, "flash_code_start": 1610612736, "flash_length": 524288 } >Thank you for offering to help me figure out what's going on. My full program (of which these four functions are only a part) is ~2.5K lines/~80K. That runs fine on my Bangle.js from storage, but of course "Send to Espruino" doesn't work for that large program either. I've had to limit some features of my full program due to running out of memory so if I could get the IDE to minify my programs that would be great.
-
Hi! It looks from the information you gave like you're actually running on the Emulator rather than a real Bangle? Was that intentional?
When the upload fails, can you still interact using the left-hand pane of the IDE?
You could also try clicking the down-arrow next to the upload button, selecting 'storage' and giving a filename there? That should have basically the same effect as doing the manual upload.
It's possible that there's something in your code that is causing the IDE to error while converting it for Espruino. Are you able to paste up your code here?
-
Thank you very much for your reply Gordon, and congratulations on another successful Kickstarter campaign!
I was indeed uploading my code to the emulator, but whether I try to upload it to the emulator or my Bangle.js the result is the same (i.e., nothing happens). After clicking the upload ("Send to Espruino") button, the left-hand pane of the IDE flashes briefly, but I can still interact with it normally. If I click the down-arrow and enter a storage file, again nothing happens, and I can still interact with the left-hand pane as usual.
I would not be at all surprised if there's something in my code causing the problem, however as I noted previously if I cut-and-paste each function alone into the right-hand pane, each one is uploaded successfully without any EMSCRIPTEN errors. The problem only occurs when I try to upload all the functions at once, leading me to believe I'd exceeded some size limit.
Anyway, I have attached a file that reproduces the problem in the emulator and my Bangle.js. I'm not sure it adheres completely to the recommended coding guidelines but I think it's pretty close. Thank you in advance for investigating the issue and pointing out any obvious missteps in my code (I'm still learning how to write code for my Bangle.js).
Best of luck with Bangle.js v2, I can't wait to receive mine and start updating my apps for it!
1 Attachment
-
Mon 2021.10.11
I took that source code file verbatim and loaded into notepad++ to determine it's size. I then stripped the comments as that is what occurs during the upload. Should the leading whitespace chars and blank lines be removed, we'd get down to the ~400 JSVars that Espruino finally unpacks.
9077 / 16 = 567
7961 / 16 = 497Bytes / Bytes per JSVar = JSvars
So plenty of Free memory remaining.
>process.memory() ={ free: 1207, usage: 393, total: 1600, history: 5,
Some techy stuff that might pique your interest:which I conveniently retrieved from a thread I created:
I ran into a similar issue five years ago and ideas here might provide some insight:
Note that my code file was three times larger!!
-
Mon 2021.10.11
@TTBangler while I'm not up to speed using the graphics class on Bangle.js that code block does produce errors for me. I don't have a Bangle to test/troubleshoot and I'm not entirely sure whether the user is required to define a container for it.
The entire code block takes roughtly 20-30 seconds to upload using the native app and there is plenty of free space remaining.
>process.memory() ={ free: 1207, usage: 393, total: 1600, history: 5, gc: 0, gctime: 1.572 } >These errors occured using the native app as I don't have a Bangle connected, and no Emulator access within that app.
>Uncaught ReferenceError: "g" is not defined at line 275 col 2 g.clear(); ^ Uncaught ReferenceError: "g" is not defined at line 130 col 2 g.clear();if(options.title!=undefined){g.setFont("Vector",30);g.setCol... ^ in function "drawChoices" called from line 13 col 132 ...choices;drawChoices(options);if(options.choices==2){return new Promise(... ^ in function "getChoice" called from line 308 col 8 }).then((v) => { ^ { "free": 1232, "usage": 368, "total": 1600, "history": 672, "gc": 0, "gctime": 1.853 } Uncaught ReferenceError: "Bangle" is not defined at line 323 col 7 Bangle.on('swipe', swipeMemoryLogger); ^ > -
Using the online IDE, upload takes about three seconds using the Emulator:
>{ "free": 1456, "usage": 1044, "total": 2500, "history": 67, "gc": 0, "gctime": 2, "blocksize": 13 } >It is interesting to note that as Espruino does it's housekeeping, some blocks are freed as I typed this ten seconds after the render above:
>process.memory() ={ free: 1584, usage: 916, total: 2500, history: 103, gc: 0, gctime: 2, blocksize: 13 } > -
Mon 2021.10.11
Reply to post #3
post #3 'Sorry but I don't really understand how to interpret the numbers returned by process.memory() or trace() for that matter'
Glance over as previously provided
Simple answer, Total - Usage = Free
For me, when Free gets to below ~150 it's time to take a break and figure out what to take out.
History is what occurs as one uploads from the L-Hand console side. Espruino will retain until it determines possible memory issues then do some housekeeping to clear. See my link in post #6 on one way to clear if desired. Although I wondered what might go awry, that value may be ignored for most development.
-
Reply post #3
'and by that I mean add it to the right-hand source code pane--then "Send to Espruino" no longer does anything.'
As explained in post #3 I'm cut-n-paste in both the R-Hand editor side and upload, along with just the first two functions followed by the rest in the L-Hand console side.
What browser and PC OS are you running on?
I'm not having issues with
ttchoice.app.jsusing Chrome on Windows10 -
Wow Robin, thank you so much for all the helpful info and links about how memory is handled in Espruino! And thank you also for thinking to ask what browser/OS I use and for actually running my code. I have been using Chrome on Ubuntu and the Web IDE has been working so well that it never occurred to me that my OS might be relevant. Well I just tried uploading my code with Chrome/Windows 10 and it was successfully minified and uploaded, thank you! (And this surprised me because my Windows machine has half as much memory as by Ubuntu box, 8G vs 16G).
Unfortunately, the code I uploaded here is just a small subset of my app. If I put my app's full source code into the right-hand pane of the IDE on Windows and do "Send to Espruino"/RAM then I get the crash-like behavior I think Gordon was alluding to. Specifically, the message "Execution Interrupted" is written many times to the console followed by "Prompt not detected - upload failed. Trying to recover...". If I do "Send to Espruino"/Storage as Gordon mentioned I am able to upload my full source code but it isn't minified. So I guess I will have to employ some of the tricks you've kindly referenced to reduce the size of my app.
Well hopefully this post will be of some help to others who run into problems with large apps.
Thanks again Robin and Gordon for your gracious help with my issue!
-
Oh, the mystery deepens! I see the same behavior on Ubuntu regardless of which minification setting I use, even when it's turned off. And sorry for not doing this sooner but when I looked at Chrome's debug console I noticed "'JSCODE' exceeded the quota" errors when doing "Send to Espruino". From what I could learn by searching for the error it appears that Chrome ran out of storage, but as shown in the attached screenshot it still seems to have plenty available.
If whoever develops the Web IDE has any suggestions on what to try next I'd be happy to do more investigation as I can reproduce the issue consistently on my machine.
1 Attachment
-
-
Wow, that is extremely strange... I guess it comes from this line:
window.localStorage.setItem("JSCODE", Espruino.Core.EditorJavaScript.getCode());In the console, please could you paste in
Espruino.Core.EditorJavaScript.getCode().lengthand see what it says? It seems super unlikely you'd have it anything like the quota just with some JS code.You could also try just running
window.localStorage.clear();in case somehow some other variable has used up the local storage. Is it possible you'd enabled offline mode? I guess that could have used a bunch of storage. -
Thanks Gordon for following up on this!
Below I've attached the result of what happened after I pasted what you requested into the Chrome/Ubuntu console. I'm sorry I don't understand what you mean by "offline mode", however the good news is that after entering window.localStorage.clear() into the Chrome console I was able to upload and minify my source code. I also tried pasting my full app into the source code pane and tried to upload it, and got the same "Execution interrupted" errors as I did on Windows. If there's anything else you'd like me to do to help track down what's happening, please let me know.
In the meantime, I believe I read somewhere in the Espruino documentation about Linux command-line development tools so I'll try to see if those will allow me to minify my full app (~2.5K lines, ~80K).
1 Attachment
-
-
It's kind of strange - really not sure what to suggest as 9k of data shouldn't be anywhere near enough to cause quote issues!
However, I've added a 'try' around the localStorage save. It's not live on the main site yet but if you try the beta Web IDE at https://espruino.github.io/EspruinoWebIDE/ then hopefully even if localStorage breaks, you'll be able to keep using the IDE just fine
-
Thr 2021.10.14
post #16 'really not sure what to suggest as 9k of data shouldn't be anywhere near enough to cause quote issues!'
To @Gordon, I have a feeling that my evaluation of the source provided
post #6 '9077 / 16 = 567'
and measured usage after removing comments and whitespace
usage: 393
was for the file ttchoice.app.js initially uploaded in post #5 and not the actual code ref post #13 and #15
I'm wondering if in fact that QUOTA limit is being reached as the actual app is now being uploaded and not the subset ttchoice.app.js as I'm guessing is what you may have felt was the source being used as was I.
post #3 'My full program (of which these four functions are only a part) is ~2.5K lines/~80K'
post #11 'Unfortunately, the code I uploaded here is just a small subset of my app.'For comparison, my very large ongoing app MVLOA pushing the limits I know, comprised of four modules that totaled nearly 40K with comments with a combination of online and hand minification while attempting to upload, nearly filled the ~2200 JSVars the Pico has. While I don't know the capacity of Bangle or the Emulator, should the the actual app be what is in use, it seems very likely that even with minification, there is likely to be some capacity issues here.
Super rough estimate here: as Espruino requires half the space to unpack (the reason I used several small modules to get around - uploading the largest first allowed nearly 30K max for me) should a Pico be used, 2200 / 2 = 1100 * 16 ~= 17K of minified source. Much more above that is eeking out the limits I'm afraid. What is the capacity est for Bangle and emulator?
To @TTBangler is this in fact the case here? pp. I'm wondering if
-
Hello Robin, Gordon,
Sorry if I caused any confusion. Just to clarify, the app I uploaded to this post (that is a subset of a much larger app) results in the 'JSCODE' exceeded error for me on Chrome/Ubuntu with the non-beta Web IDE Bangle.js v1 Emulator. I've just confirmed that with the change Gordon made, the (subset) app does now successfully minify/upload for me on the same Chrome/Ubuntu system using the beta Web IDE Bangle.js v1 Emulator. Also, by entering 'window.localStorage.clear()' after getting the 'JSCODE' error in the non-beta IDE, the (subset) app can also be successfully uploaded/minified.
My full app (that I have not posted here) results in an "Execution Interrupted"/"Prompt not detected - upload failed" IDE crash even with the beta Web IDE Bangle.js v1 Emulator on both Windows and Ubuntu. However--and this is great news--my full app does successfully upload/minify with either the non-beta or beta Bangle.js v2 Web IDE Emulator on Ubuntu. Moreover, I just discovered that I can download the minified source code from the Bangle.js v2 emulator and upload it into storage in the Bangle.js v1 emulator, and the minified version of the source reduces the process.memory() usage from 1239 down to 1215. I had to make a couple of minor changes to the minified source code to get the app to run but it seems to run fine. Oh, and I forgot to mention that the message I receive when uploading my full app to the v2 Emulator is "No errors. Minified 74751 bytes to 37349 bytes."
Hope that clarifies things--thank you both very much again for your help!
-
Great - glad that sorted it!
@Robin the quote messages have nothing to do with Espruino at all - that's purely the IDE on the Web Browser. However the quota should really be in the megabytes range (and appears to be for everyone else) - so I'm not sure why it's getting exceeded on this particular we browser.
-
Fri 2021.10.15
Hi @TTBangler a couple of points as numbers didn't seem to add up.
'Minified 74751 bytes to 37349 bytes.'
'process.memory() usage from 1239 down to 1215'Note: 37349 / 16 = 2334
I'm not so sure the value returned from the emulator is representative of the physical Bangle hardware.
As Espruino will require some of the free JsVars for housekeeping, stack, vars, function returns, etc., depending on what your code is doing, my target minimum remaining would leave around 1900 JsVars available for your code. Not sure what the capacity of Bangle V2 is but,
The results of my Bangle V1
>process.env ={ VERSION: "2v04.218", GIT_COMMIT: "5439331a", BOARD: "BANGLEJS", FLASH: 524288, RAM: 65536, SERIAL: "330b0ad8-0c11713d", CONSOLE: "Bluetooth", MODULES: "Flash,Storage,hea" ... "ow,graphical_menu", EXPTR: 536883676 } >process.memory() ={ free: 2057, usage: 43, total: 2100, history: 13, gc: 0, gctime: 3.20434570312, "stackEndAddress": 536923072, flash_start: 0, "flash_binary_end": 471492, "flash_code_start": 1073741824, flash_length: 524288 } > -
Hi @Robin,
Thanks, I tend to do most development using the IDE Emulator (since it's faster than the actual hardware) and I just discovered that my full app code I minified for Bangle.js v2 that works for the Bangle.js v1 emulator gets a strange error and doesn't run on my actual Bangle.js v1 hardware.
By the way, I believe version 2v10 of the firmware somehow increased the amount of usable memory so here are the numbers I get on my real Bangle.js v1 after connecting and before loading anything:
>process.env ={ VERSION: "2v10", GIT_COMMIT: "63dafbf9f", BOARD: "BANGLEJS", FLASH: 524288, SPIFLASH: 4194304, HWVERSION: 1, STORAGE: 4194304, RAM: 65536, SERIAL: "8614df71-b4fb9fc0", CONSOLE: "Bluetooth", MODULES: "Flash,Storage,hea" ... "tensorflow,locale", EXPTR: 536882372 } >process.memory() ={ free: 1928, usage: 656, total: 2584, history: 11, gc: 27, gctime: 5.79833984375, blocksize: 13, stackEndAddress: 536925152, flash_start: 0, flash_binary_end: 450188, flash_code_start: 1610612736, flash_length: 524288 } >That's a good point about Espruino needing space for housekeeping. I've actually been very pleasantly surprised by how much I was able to do before I started running into out-of-memory problems at runtime. And I believe the Bangle.js v2 has 4x more memory so can't wait to receive mine!
-
using emulator and also my own cross development environment in browser (as used to build the https://www.espruino.com/ui (w/ screenshots: Modular and extensible UI framework and ui elements.), I think I ran into similar issues: in the end it was overloading the Espruino interpreter buffer on upload. Upload means in practically all environments just transporting bytes to another device and storing them there. In Espruino context that is less than half the truth: code is executed on upload, and if stored directly in flash, the same happens when 'loading' (derivates of parts of) it into RAM for execution. Make sure that an 'atomic unit of code becomes not to big before the parser of the interpreter comes to a 'touch down' for execution and tackle the next thing. Im not sure if the emulator has the same constraints in regard of memory as the 'real hard wares' have.... after all, I can emulate not just Bangle things (more so in my emulator, where I emulate also devices connected to Espruino, such as displays, and pins, and...). So it is less the number of total lines but the lines that the parser has to go thru before being able to execute the parsed things and move on to the next thing.
-
Sat 2021.10.16
This L14
blocksize: 13raises a red flag for me. Don't know the answer, but I've only seen 16 or 12 on older boards.Was this somehow forced? @TTBangler or an anomaly that's popped up using the emulator?
I'm also somewhat concerned about the actual char count in relation to the number of JsVars that are being reported in use. I've noticed that what I thought I understood (project years ago) and what seems to be occurring/reported now doesn't add up to what I believed at that time. In your case, file size is around twice what the actual JsVar usage appears to be. Will continue to monitor and realize the need to do more size/reservation calculations, Trace, debug, dump, flat string length, etc. in order to cement what is going on in memory, before the dreaded 'LOW_MEMORY' flag appears. -
I wasn't aware of your UI framework--it looks very impressive, thanks for bringing it to my attention! I'll definitely look more into your framework once I get my Bangle.js v2 (since it has a full touchscreen).
Thanks also for the explanation about what happens during an upload. Overloading the Espruino interpreter buffer on upload could indeed be what happened in my case. The IDE crash behavior was the same when uploading my very large app to either the emulator or my Bangle.js.
Robin
Gordon
allObjects-
Post a reply
About
IDE "Send to Espruino" stops working--size limit exceeded?
Posted by
@TTBangler
Hello,
I've noticed that after a while, "Send to Espruino" in the Web IDE stops working. It seems to happen once the code exceeds a certain size. In my case, I'm seeing this with a 250 line, ~8k file. I don't see any errors listed in the IDE's console (or Javascript errors flagged in my code), clicking "Send to Espruino" simply appears not to do anything anymore. I can work around this by uploading the file to storage and running it from there, but then I lose the memory benefit of the IDE's automatic minification. I also set the pretokenise flag at the top of my file, but haven't seen any obvious memory reduction.
Well if anybody can offer any hints on what's going on and how I can get "Send to Espruino" working again, I would greatly appreciate it. I'm having a great time building apps for my Bangle.js and can't wait to receive the latest version!