-
-
Sorry, didn't see this reply earlier (I didn't refresh the page oops). No, the filename is correct. To verify the glitch, have a Bangle watch with bare minimum apps (Bootloader + Default Launcher + Battery Widget + Morph Clock App). Now run the following code (same as above without the erase all and dummy sync)
function repro() { for(var i = 0; i < 1; i++){ print("Writing test file, attempt " + (i+1)); var file = require("Storage").open("testfile.log", "a"); try { while (true){ var sentence = "Long string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberish\n"; file.write(sentence); } } catch(ex){ console.log(ex); console.log("File size after write error: " + file.getLength().toString()); file = null; } print("Files before Erase " + (i+1) + "\n" + JSON.stringify(require("Storage").list())); print("Storage.getFree Before Erase " + (i+1) + " " + require("Storage").getFree().toString()); require("Storage").open("testfile.log", "w").erase(); require("Storage").compact(); print("Storage.getFree After Erase and compact " + (i+1) + " " + require("Storage").getFree().toString()); print("Files after Erase " + (i+1) + "\n" + JSON.stringify(require("Storage").list())); } } repro();
You will see
getFree
before and aftererase
doesn't make a difference even though thetestfile.log
file gets removed from thelist()
. -
Okay @Gordon... I have finally got the reason why my App code is failing and sample code is working. I decided to read the
compact
documentation more closely to realise it fails if there isn't enough RAM. Digging through the code showed me it needs atleastallocated + 1024
to continue withcompact
. I haven't traced it as far as how much isallocated
but when I have my app loaded and I runprocess.memory()
I get this:process.memory(); ={ free: 1036, usage: 1064, total: 2100, history: 7, gc: 0, gctime: 3.72314453125, blocksize: 16, "stackEndAddress": 536923400, flash_start: 0, "flash_binary_end": 483876, "flash_code_start": 1073741824, flash_length: 524288 }
Even if allocated is 126 or 256, we can see the amount of free memory (1036) isn't enough for compact to continue.
However, the reason my sample above kept working is because on a Bangle with everything erased, if I run
process.memory()
I get the following:>process.memory(); ={ free: 2056, usage: 44, total: 2100, history: 7, gc: 0, gctime: 2.99072265625, blocksize: 16, "stackEndAddress": 536923400, flash_start: 0, "flash_binary_end": 483876, "flash_code_start": 1073741824, flash_length: 524288 } >
This means,
require("Storage").compact()
is ineffective if you have Bootloader + Default Launcher + Battery Widget + BT Widget + Settings and 1 clock App.How we resolve this I am not sure, maybe after erase
compact
is called without the check for minimum RAM? Will that work? I have to dig deeper, but essentially we have an issue where we are unable to realise the full potential of Bangle and the storage it has. -
Okay... I used the WebIDE to delete the new file and that didn't help either
>require("Storage").getFree(); =2686684 >require("Storage").open("fitclock.log","w").erase(); =undefined >require("Storage").getFree(); =2686684 >require("Storage").compact(); =undefined >require("Storage").getFree(); =2686684 >
Next I'll try to set the file handle to null before deleting it.
-
Yup, my code doesn't seem to work (even if I try to delete the file before it has errored out). The watch was not on my wrist so it wasn't trying to record any movement.
After dowloading the log I wanted to clear the log,
Bangle.Helper.size("fitclock.log"); 869585 =undefined >require("Storage").getFree(); =3292892 >Bangle.AppLog.clearLog(); =undefined >require("Storage").getFree(); =3292892 >require("Storage").compact(); =undefined >require("Storage").getFree(); =3292892
Here's the minified->uploaded->downloaded->beautified code for
clearLog
Bangle.AppLog = { currentFile: null, lock: !1, shardCount: 0, currentFileName: "", error: !1, diskFull: !1, init: e => { Bangle.AppLog.currentFileName = e; try { null != e && "" != e ? (Bangle.AppLog.currentFile = require("Storage").open(e, "a"), console.log("File created: " + e)) : console.error("Log file name not provided") } catch (e) { console.log("Failed to create file", e) } }, write: e => { if (1 != Bangle.AppLog.lock && 1 != Bangle.AppLog.error) try { Bangle.AppLog.currentFile.write(e) } catch (e) { console.log(e), Bangle.AppLog.currentFile = null, Bangle.AppLog.error = !0 } }, clearLog: () => { Bangle.AppLog.lock = !0, Bangle.AppLog.currentFile = null, require("Storage").open(Bangle.AppLog.currentFileName, "w").erase(), require("Storage").compact(), Bangle.AppLog.lock = !1, Bangle.AppLog.error = !1 }, beginSync: () => { for (var e = require("Storage").open(Bangle.AppLog.currentFileName, "r"), t = ""; null != t && -1 == t.indexOf("ÿ") && null != (t = e.readLine());) print(t); print("\x3c!-- finished sync --\x3e"), e = null } }
I am not sure if my minification using
gulp-terser
is mangling code in some way. I'll test with directrequire("Storage").open('...','w').erase()
from console tomorrow morning. -
Okay, @Gordon I think
compact
does the trick. The following code breaks if you comment out thecompact()
call (line 34 below).*** WARNING Does a Storage.eraseAll() ***
function repro() { require("Storage").eraseAll(); for(var i = 0; i < 5; i++){ print("Writing test file, attempt " + (i+1)); var file = require("Storage").open("testfile.log", "a"); try { while (true){ var sentence = "Long string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberish\n"; file.write(sentence); } } catch(ex){ console.log(ex); console.log("File size after write error: " + file.getLength().toString()); file = null; } var f = require('Storage').open("testfile.log", 'r'); var line = ''; print("Faking a sync please wait"); while (line != null && line.indexOf('\xFF') == -1){ line = f.readLine(); if(line != null){ //print(line); } else{ break; } } print("<!-- finished sync -->"); print("Files before Erase " + (i+1) + "\n" + JSON.stringify(require("Storage").list())); print("Storage.getFree Before Erase " + (i+1) + " " + require("Storage").getFree().toString()); require("Storage").open("testfile.log", "w").erase(); require("Storage").compact(); print("Storage.getFree After Erase and compact" + (i+1) + " " + require("Storage").getFree().toString()); print("Files after Erase " + (i+1) + "\n" + JSON.stringify(require("Storage").list())); } } repro();
That's strange because I am sure I have
compact
in myclearLog
function now but that doesn't seem to work.
I will do more testing and report back.process.env ={ VERSION: "2v04.398", GIT_COMMIT: "292f7d0d", BOARD: "BANGLEJS", FLASH: 524288, SPIFLASH: 4194304, STORAGE: 4194304, RAM: 65536, SERIAL: "a0cf0da6-7b6c234d", CONSOLE: "Bluetooth", MODULES: "Flash,Storage,hea" ... "tensorflow,locale", EXPTR: 536883676 } >
-
-
Okay, apologies in advance for putting out a post without clearly reproducible code. But maybe someone has seen this before and can help.
I am able to write and delete to Bangle JS using the
StorageFile
api.Currently my code displays
ERROR
on screen when my file has tried to write over an MB and failed. Thereafter it doesn't try to write anymore until the file has been dowloaded and the old file deleted. Following are bits of code that do these thingsBangle.AppLog = { currentFile: null, shardCount: 0, currentFileName: "", error: false, init: (filename) => { Bangle.AppLog.currentFileName = filename; try{ if(filename != null && filename != ''){ Bangle.AppLog.currentFile = require("Storage").open(filename, "a"); console.log("File created: " + filename); } else{ console.error("Log file name not provided"); } } catch(ex){ console.log("Failed to create file", ex); } }, write : (sentence) => { if(Bangle.AppLog.error != true){ try{ Bangle.AppLog.currentFile.write(sentence); } catch (ex){ console.log(ex); Bangle.AppLog.currentFile = null; Bangle.AppLog.error = true; } } }, clearLog: () => { Bangle.AppLog.currentFile = null; require("Storage").open(Bangle.AppLog.currentFileName, "w").erase(); Bangle.AppLog.error = false; }, beginSync: () => { var f = require('Storage').open(Bangle.AppLog.currentFileName, 'r'); var line = ''; while (line != null && line.indexOf('\xFF') == -1){ line = f.readLine(); if(line != null){ print(line); } else{ break; } } print("<!-- finished sync -->"); f=null; } };
Every so often (every 1 out of 3 tries) after I have deleted the old file by calling
Bangle.AppLog.clearLog()
from the WebIDE - the app will try to (re)create the file, (it doesn't throw any error when callingopen(...)
), but fails when trying to write to the file.>Bangle.Helper.size("fitclock.log"); 0 =undefined Trying to create new file:fitclock.log File created: fitclock.log Error: Error: Unable to find or create file Trying to create new file:fitclock.log File created: fitclock.log
As seen from the above log entries to the console,
init()
doesn't throw an error but the firstwrite
does.Unfortunately, once the storage gets into this state, I cannot get Bangle to write to any file with any other name. Rebooting using "Button 1 & 2 press" doesn't help. I have to to
Storage.eraseAll()
to get back ability to write to file system again. All of them throw the same "Unable to find of create file" error.> > > >var f = require("Storage").open("test", "a"); =StorageFile: { name: "test", chunk: 1, offset: 0, addr: 0, mode: 97 } >f.write("Testing"); Uncaught Error: Unable to find or create file at line 1 col 18 f.write("Testing"); ^ >
It is worth noting, even though
StorageFile.open(...)
did not throw an error, the file doesn't exist or is not recognised in the file WebIDE's file browser (ref to attached image).I looked through the entire repository and there is only one instance of the error message
Unable to find or create file
and it seems to be only present hereThat led me to the open issue... can't tell for sure but these two might be related?
P.S. Interesting side note... there seems to be very little disk space left for real or so Bangle thinks... So the files probably didn't get deleted and their space reclaimed? That would explain the 1/3 frequency. That certainly explains why things work after
eraseAll
... Hmm... what am I doing wrong with myclearLog
function?>require("Storage").getFree(); =3144
-
@PlaidFox I am in a similar boat, I was planning to start with the Gesture Test app in the Bangle repository
Also there are some links posted by @Gordon in this comment that might come in handy.
Do share your successes and learnings :D
-
Ah... silly me. I took long (ish) file names an indication of more than 255 pages being possible.
I quite like the
StorageFile
api so I think I'll just build a rudimentary roll-over system where a new file is created on reaching max-size, until I've run out of all space available. Still hoping to never use all that space :-).Thanks again.
-
There are two ways to do this,
- First is to create a widget with a tiny UI on the top to indicate it is running
- Take any of the clock apps there is an add functionality to the app.
1 and 2
If you build an App, you have the option of handling the
kill
event and saving app state to a file. When you app is restarted you can continue where you left off. You'll need to wire up the event handlers again though.3
If you build a widget I believe it auto starts on load (I haven't built one yet). If you build an app, and it is the only clock app on the watch it will start automatically on load. If you have multiple clock apps, you have the option of specifying which one is your default clock app.
4
Should be doable, all the code is out there but I would explore all other options first :-)
Have a poke at the existing apps in the Bangle Apps repository on Github it is fairly (Espruino) beginner friendly :-)
- First is to create a widget with a tiny UI on the top to indicate it is running
-
If I run this code on a cleaned up Bangle watch, it seems to throw a
file too big error
eventually and the file size reported is ~ 1Mb. Am I doing something wrong to not be able to write on beyond 1Mb ?(** BEWARE: starts with Storage.eraseAll() ** )
function repro() { require("Storage").eraseAll(); var file = require("Storage").open("testfile.log", "a"); try { while (true){ var sentence = "Long string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberishLong string of gibberish\n"; file.write(sentence); } } catch(ex){ console.log(ex); console.log("File size after write error: " + file.getLength()); file = null; } } repro();
>Error: Error: File too big! File size after write error: 1036320
Following are my env details
>process.env ={ VERSION: "2v04.375", GIT_COMMIT: "9b362663", BOARD: "BANGLEJS", FLASH: 524288, SPIFLASH: 4194304, STORAGE: 4194304, RAM: 65536, SERIAL: "a0cf0da6-7b6c234d", CONSOLE: "Bluetooth", MODULES: "Flash,Storage,hea" ... "le,graphical_menu", EXPTR: 536883676 } >
My fork was last synced with upstream on March 1, so I am wee bit behind. Apologies if this is causing the issue.
Update: Just to clarify, I was expecting the File write to error out after ~4Mb which is the max capacity of the BangleJS.
-
Thanks for looking into it @Gordon. I just did a "Clean cache and hard reload" on the WebIDE and tried to download the
test
file. It gave me the following error in console.index.js:93593 Uncaught TypeError: Cannot read property 'length' of undefined at Object.isASCII (index.js:93593) at showViewFileDialog (index.js:131556) at index.js:131654 at index.js:131329 at nextStep (index.js:93250) at onTimeout (index.js:93281)
I guess this is the timeout you are referring to in #11. I mistook that as an issue with the
length
call. So all is well now. Marking this one as resolved. -
@Gordon Apologies, for not providing code reference.
My code for
readline
is now working as expected after I added the\n
back to the code. Don't remember how it went missing but my bad.I don't have a
read
in my code. When I was mistakenly writing logs without linebreaks, the WebIDE functionality of downloading a file was also failing.My question should have been framed, why was
read
function used by WebIDE failing to download a large file.var i = 10000; var file = require("Storage").open("test", "a"); while (i > 0){ var sentence = "Long string of gibberish"; file.write(sentence); i--; } file = null;
If you create a file using the above code and try to download the file using WebIDE, the download fails.
Thank you very much for your stellar support and responses so far. Did not mean to annoy you.
-
@Gordon I understand why the
readline
is failing but why is theread()
failing? Is the file too big forlength
? -
-
Looks like there is an issue with
readLine()
inStorageFile
I am getting this error:
Execution Interrupted at line 2 col 21 line = f.readLine(); ^ New interpreter error: LOW_MEMORY,MEMORY
WebIDE also fails with this error:
index.js:93593 Uncaught TypeError: Cannot read property 'length' of undefined at Object.isASCII (index.js:93593) at showViewFileDialog (index.js:131556) at index.js:131654 at index.js:131329 at nextStep (index.js:93250) at onTimeout (index.js:93281)
I get a massive blurb output in the IDE (check image of the debug window attached, got the same the output in the WebIDE console).
Update: The
ftclog
log file is '80 shards long'.Cheers.
-
Thanks @Gordon. The WebIDE also failed with similar error.
I failed to download the large file and ended up doing an
eraseAll
. Bangle had nearly frozen when the file was that big.I am currently on firmware 4.375 and have turned on HRM and hopefully will get about 10-15 minutes of GPS info as well by the time I get home. It will be interesting to see how big a file I can push, now that the whole of 4Mb is available. I'll probably do an upper limit of an Mb or so.
I currently have two statuses, one is
diskfull
and other iserror
. Funnily enough I had hitError
today notdiskfull
. Looking forward to redoing that bit using the newlength
call.Sorry for pushing the boundaries and keeping things on the edge for you :-)
-
My Python script to download
StorageFile
s seems to have gone bust completely. It now times out waiting for first byte. I changed the previous timeout from 10 seconds to upto 30 seconds and it still didn't kick off. In the last attempt I got this in the file instead of actual data.Execution Interrupted at line 2 col 21 line = f.readLine(); ^ New interpreter error: MEMORY_BUSY
The command sent to the Bangle (via Python/Bleak) is
getFileCommand = b"\x03\x10\ var f = require('Storage').open('ftclog', 'r');\n\x10\ var line = '';\n\x10\ while ((line != null && line != undefined) && (line.indexOf('\xFF') == -1)){\n\x10\ line = f.readLine();\n\x10\ print(line);\n\x10\ }\n\x10\ "
I also tried using the WebIDE and got similar error. At the time I wasn't sure if it was because I had initiated another connect on a different browser tab.
getFileCommand = b"\x03\x10\ var f = require('Storage').open('ftclog', 'r');\n\x10\ var line = '';\n\x10\ while ((line != null && line != undefined) && (line.indexOf('\xFF') == -1)){\n\x10\ line = f.readLine();\n\x10\ print(line);\n\x10\ }\n\x10\ "
In the WebIDE, atleast it spent good 10 minutes downloading the blurb before failing.
I will try with a longer timeout in the evening in my script.
Firmware details:
{ VERSION: "2v04.375", GIT_COMMIT: "9b362663", BOARD: "BANGLEJS", FLASH: 524288, SPIFLASH: 4194304, STORAGE: 4194304, RAM: 65536, SERIAL: "a0cf0da6-7b6c234d", CONSOLE: "Bluetooth", MODULES: "Flash,Storage,hea" ... "le,graphical_menu", EXPTR: 536883676 }
P.S. This may not be related to the firmware refactor. I started having this issue on Friday when I was running
2v04.99
Full Python script on Github
-
after upgrading the firmware. The watch is stuck on the boot screen with the version. Not able to find anything in the troubleshooting. Help is appreciated.
Have you tried uploading any apps from the apps store after the upgrade? If not, install the
Bootloader
,Default Launcher
and any of the* Clock
apps to get started.I had the same boot screen. I am not sure if the firmware comes with any default apps.
-
-
Just got ver 0.06 of Bootloader app after merging from upstream and this issue is gone. So marking it as resolved. Thanks @Gordon.
-
I think that's something that could definitely be improved on! 1 megabyte in 10 hours is around 30 bytes every second! That's insanely detailed - like you're storing 30 bytes of data for every heartbeat!
There's the enterprise dev (in me) kicking in again... I was extra cautious to have correct CSV format, so every field in every line was wrapped in an opening quote and a closing quote #faceplam...
- just getting rid of the quotes from the file reduced it by 100K. I am actually saving all the
accel
values sox
,y
,z
,g
anddelta
. - Changed row header form
HRM
toHR
andACCL
toAL
, another 10K savings - My Date/Time format is a mess, I am going to change that up to save some more.
The file currently has ~12K lines of data of which
- 7364 lines are accelerometer values (saved at every
step
event). - 396 GPS entries (Loads of NAN that can be trimmed)
- 4728 lines of Heart Rate entries
- Logging started at about 09:07AM and stopped at 16:19PM.
I am saving full resolution of the floating point data I get so yeah each line is pretty detailed.
Looks like I can't really save full day of heart rate + accelerometer + GPS settings in current format. I'll see what further optimisations I can do or what data I can live without.
There is nothing on the desktop/analysis side yet apart from a prototype to map GPS points on a map using MapBox. But I want a closer look at accelerometer readings and heart rate and find co-relations between low movement + heart rate = sleep quality and stuff like that. Also correlations between heart rate and GPS values when cycling etc. Basically mimic as much of the Fitbit functionality as possible.
I'll keep working on optimising data collection until we have access to the rest of the memory via
StorageFile
:-). Also I'll aim to sync atleast twice a day. So it is just a matter of managing entries for 8 hours.Update:
Looks like it clocked off at about 2:15 pm today... that too at 675KB only... hmm... very curious... I'll keep digging. First I need to fix this need foreraseAll
after it has errored out! - just getting rid of the quotes from the file reduced it by 100K. I am actually saving all the
-
Oh... thanks for both the tips. I'll remove the redundant lock.
@Gordon reference to your comment about the global
AppLog
namespace making python script easier, absolutely. I am planning to move the code forsync
into another function in AppLog namespace. That way I'll have a little more control when I introduce compression and send back data uncompressed. I'll wait for your suggestions on compression.
@chrisotherwise are you uploading Apps from the official BangleApps repository or a custom App you built yourself. If it is a custom app and an open URL could you share the URL with us :-)