-
Memory is getting written over. In 'C' it's the dreaded missing null line terminator. Used to malloc heapfree etc.
I guarantee this is not what's happening in JS.
what is the proper technique to discover a possible memory overrun
http://www.espruino.com/Reference#l_E_errorFlag
It'll get called when Espruino's had some difficulty allocating memory. It may not have 100% failed at that point, but it's got close.
However, if you have problems even without that and you HAVEN'T seen any
MEMORYerror on the console, it's not an error to do with memory getting used up -
Fri 2020.03.27
Thank you @Gordon for the reply.
'I guarantee this is not what's happening in JS'
I can assure you that after running a setInterval that updated once a second, calling a log to console method, printing the contents of around twenty vars, worked for at least five minutes, then multiple errors such as the one in post #1 indicating 'undefined' started popping up. I attempted to print those vars from the command line Left-hand console side, same error.
Doing a dump, and the defined var that is clearly seen on the Right-hand editor side, and the var is clearly there, that was uploaded. Doing a search both in the source using Notepad++ and within the WebIDE editor match. But now attempting to run a wrapper function that calls the console.log statements now won't even run, also providing the same error. So Espruino is informing us that it can't find the previously declared var either, otherwise it would just return =undefined when not assigned a value.
I even performed a sanity check on the L.H. console side to see what error is seen when an attempt to view a known non declared var is entered and the same errror occurs.
I never defined rrr so this is correct - and the same for the now missing var
>rrr Uncaught ReferenceError: "rrr" is not defined at line 1 col 1 rrr >intervalID =undefined
I scrolled through the dump contents and the var that is defined in the code that was uploaded is no longer there.I'll copy the contents of the console to a text file and take a fresh look at both the dump and the output, along with running more tests tomorrow. I'm chicken to power down/try again, in fear of not being able to have this repeatable situation, be reproduceable.
If your statement above is accurate, what other ideas might make a var disappear?
I'll read over the errorFlag event tonight and play with it tomorrow. Maybe that will shed some light.EDIT: This is what is shown just after the undefined errors start to occur:
>E.getErrorFlags() =[ "FIFO_FULL", "LOW_MEMORY" ] >
The LOW_MEMORY is obvious, and the FIFO_FULL most likely occurs as the logging function throws the error, bypassing or delaying the ability to turn off a pulse train created by analogWrite and the corresponding setWatch fills quickly. -
Sat 2020.03.26
'please share build version and a code snippet'
Build version is provided at the end of post #1 code file at post #12 link here
Thank you @MaBe for taking the time to read through this post, and your enthusiasm and curiosity to assist here.
I purposely wrote the content and selected the title requesting a 'Technique' as I realized debugging this will be too time consuming to put anyone through. I also realized Gordon would understand that, as he had been assisting here in another thread I started a month ago, and wouldn't be available, (see post #9 there, below) until after the Bangle K.S. shipments. I kept the request terse, knowing that I could move forward with a simple technique as reducing the demand on memory, may and most likely allow me to see this through to completion.
post #12 Tutorial example output anomaly using low level access PPI on MDBT42Q breakout board
To 'whet your whistle' I discovered some cool stuff using PPI that would make for a great multi-part tutorial to put the Javascript haters at bay, demonstrating near one-for-one match with that other 'C' programmed ubiquitous device.
The last code file in post #12 above worked as I described above in post #1 here, great until I started adding functions that called multiple console.log statements rendering var contents to the WebIDE console. I knew I was running short on memory in the other thread, and continuing was going to be a challenge.
I've also toyed with whether I'd be better off attempting to teach others to use this tutorial in order to find a solution, compared with just pluggin' along knowing it's likely a memory issue. My realization is that I would spend more time attempting to convey concepts and use others time that will be needed with Bangle, than just continuing performing tests with Gordon's new suggestion. The code file is over 1500 lines of code and comments, and is using 90%+ of available memory.
To move forward, the six bulleted items at the end of post #12 need more needed attention at the moment, in order for me to complete this tutorial. Adding Gordon's suggestion, and seeking solutions to pare down the existing file size is trivial at this point. Providing assistance in solving those items would be a more prudent use of your time, should you be able to spare it, and thank you.
Should I not find a suitable solution within a week, I'll post the entire code file as it sits.
-
Sat 2020.03.28
post #2 'However, if you have problems even without that and you HAVEN'T seen any MEMORY error on the console, it's not an error to do with memory getting used up'
If I understand the above, I don't appear to have problems as nothing was being displayed, and I 'hadn't' seen a
MEMORYerror while reducing the file size, but I do now see aNew interpreter error: FIFO_FULLerror, so not sure if that is inclusive for the above statement.Okay, worked all day on this. While I have been extra careful over the last two years to make sure all code is inside separate functions so no execution occurs during upload, I've double checked and verified this is the case, along with making sure nothing other than several vars are initialized, only a function call to display a shortcut list of function call names is rendered. I ripped out many tens of lines of code to get the file size waaay down, so that there were around 600-700 JSVARS available, so as not to run into the ~100 JSVAR minimum that I have read elsewhere that must not be exceeded.
I issue a
reset(1)within the WebIDE. I clear the console, button upper left. Even with now a much smaller code file, now when I just upload that file, a new error appears, but not theMEMORYerror mention in the response above.I even unplugged the MDBT42Q, in case something corrupt remained, but is consistent, despite nothing I am intentionally doing would be causing a 'FIFO_FULL' error:
New interpreter error: FIFO_FULL >This error is consitent with what I witnessed yesterday, post #3 with the significantly larger file but during runtime. So, maybe that 'FIFO_FULL' flag has always been there, as I wasn't aware to even detect that until the suggestion in post #2. What is really puzzling is the now 'LOW_MEMORY' flag, despite only using 70% of memory. This actually ran several days ago with only ~200 JSVARS remaining, so with an additional 400 free now, shouldn't be an issue.
I upload the code file from the Right-hand editor side, and the following is observed, and entering the command, flags can be seen that are set:
New interpreter error: FIFO_FULL > >E.getErrorFlags() [ "FIFO_FULL", "LOW_MEMORY" ] =undefined >While creating the content for this post, I entered
dump()and re-ran a memory check, and now the flags are reset!{ "free": 667, "usage": 1833, "total": 2500, "history": 602, "gc": 0, "gctime": 3.90625, "stackEndAddress": 536928592, "flash_start": 0, "flash_binary_end": 428148, "flash_code_start": 442368, "flash_length": 524288 } =undefined >E.getErrorFlags() [ ] =undefined >
The new question, what is causing theFIFO_FULLerror when just uploading code? I'm not seeing theMEMORYerror and there are 600+ JSVARS available, so there shouldn't be a memory issue, correct?http://www.espruino.com/Reference#l_E_getErrorFlags
'FIFO_FULL': The receive FIFO filled up and data was lost. This could be state transitions for setWatch, or received characters.
'LOW_MEMORY': Memory is running low - Espruino had to run a garbage collection pass or remove some of the command history
'MEMORY': Espruino ran out of memory and was unable to allocate some data that it needed.What is the trip point for low memory as I've stripped 30%+ and still get the flag on upload:
{ "free": 871, "usage": 1629, "total": 2500, "history": 726,Note that running a forced garbage collection
process.memorydoes clear the flag. -
Sun 2020.03.29
The following is 100% reproduceable, just uploading the code.
Uncaught ReferenceError: "dpu" is not definedI performed the following three times as I couldn't believe my luck finding a repeatable condition.
Out of all the situations I've come across this past week, this is the easiest to reproduce.I've gutted nearly 1/2 of code file lines, primarily external comment blocks, embedded comment lines within functions, and any code snippets that weren't needed to run a simple test.
It shouldn't be necessary to wire external pins as code no longer functions, although I haven't tested.
To get to this final file, the goal of the project no longer works, and many features can no longer be tested. That's okay at this point, as I've found a 100% reporoduceable situation. It appears that whatever table mechanism that references internal functions by name, becomes corrupt and therefore unaccessible, just by uploading!
3x I've powered off/on the device. I've closed/opened the WebIDE. I haven't rebooted Windows10 yet, but will do soon as I also search for, and try other lint tools.
I added some simple function wrappers to avoid having to type long commands.
Display Memory Flags Error >dmf() Process Memory shortcut >pm() Run Garbage Collection >rgc()I load the code file from disk. I upload to the MDBT42Q.
It takes twenty seconds to settle down before the second > prompt is visible.
At this point, an attempt to run function dpu() results in:
Uncaught ReferenceError: "dpu" is not defined at line 1265 col 3 dpu();Typing dump() or searching within the WebIDE editor will find that function at L200
I've run every combination of the following, and ther results are approximately the same. Running a garbage collection does clear the 'LOW_MEMORY' flag and frees up 6 JSVARS. Repeatable.
Here is a sample:
Even with 30%+ free JSVARS, no MEMORY error, but a hidden LOW_MEMORY flag is set:
>dmf() [ "LOW_MEMORY" ] =undefined >pm() { "free": 816, "usage": 1684, "total": 2500, "history": 732, "gc": 0, "gctime": 4.0283203125, "stackEndAddress": 536928592, "flash_start": 0, "flash_binary_end": 428148, "flash_code_start": 442368, "flash_length": 524288 } =undefined >rgc() [ ] { "free": 811, "usage": 1689, "total": 2500, "history": 734, "gc": 0, "gctime": 4.11987304687, "stackEndAddress": 536928592, "flash_start": 0, "flash_binary_end": 428148, "flash_code_start": 442368, "flash_length": 524288 } =undefined >dmf() [ ] =undefined >pm() { "free": 816, "usage": 1684, "total": 2500, "history": 736, "gc": 0, "gctime": 4.0283203125, "stackEndAddress": 536928592, "flash_start": 0, "flash_binary_end": 428148, "flash_code_start": 442368, "flash_length": 524288 } =undefinedAttempting to run a function that is visible on the code side and within a dump() results in:
>dpu() Uncaught ReferenceError: "dpu" is not defined at line 1 col 1 dpu() ^ >r() counter: 0 counterPulses: 0 Uncaught ReferenceError: "dpu" is not defined at line 1265 col 3 dpu()Performing a dump() will clearly show the function listed in tact L367 around the first quarter of the dump, as everything does not match the source file chronological order.
I'm posting this here now, as this ran a week ago before I started adding more comments and additional accessor functions. I'll search for other lint checkers, and continue testing tomorrow with a fresh start.
Windows 10, version 1909 64bit Web IDE version 0.70.6 { "VERSION": "2v03", "GIT_COMMIT": "e77d74f6", "BOARD": "MDBT42Q",Butchered code file to get the repeatable error, so withold the comments on format and style. ;-)
L367 in dump20200328p0807dpuTEST3.txt file is the dump showing function exists
3 Attachments
-
Sun 2020.03.29
Fresh start. Power on MDBT42Q. Start WebIDE. Same code file where I left off.
This is starting to appear to be a parsing issue, or when data chunks are sent to the MDBT42Q, they are not being received as they were sent. Now a new error appears:
Uncaught SyntaxError: Got UNFINISHED STRING expected EOF at line 934 col 27 console.log( " " ); " );dump() shows that line L7 is there, fifth inside the help() function:
function h() {help();} function help() { console.log( " " ); console.log( " " ); console.log( "Tutorial PPI - Programmable Peripheral Interconnect" ); console.log( " for the MDBT42Q Breakout board" ); console.log( " " ); " ); console.log( "Single letter command line helper functions" ); console.log( " " ); console.log( " " ); console.log( " h() help " );
If one inspects around L730 in the source file, it can be seen the correct syntax is present. The dump is showing that [console.log( "] is missing L8 from what was originally there.function h() { help(); } function help() { console.log( " " ); console.log( " " ); console.log( "Tutorial PPI - Programmable Peripheral Interconnect" ); console.log( " for the MDBT42Q Breakout board" ); console.log( " " ); console.log( " " ); console.log( "Single letter command line helper functions" ); console.log( " " );When I attempt to view the console output:
WebIDE :: Settings >> Console
BT> Sending "\n// External button " BT> Sending "press sends one puls" BT> Sending "e\u001b\nswbp();\u001b\n\u001b\n\u001b\n\u001b\n\u001b\n" BT> Sending " // Sync flag with " BT> Sending "pin state\u001b\n dp();\u001b\n" BT> Sending " sync();\u001b\n \u001b\n}\n\u0010\u001b["
I note that data is sent by the WebIDE in small chunks. Presumeably the device sends back similar manageable small chunks in response to the dump() request. I've tried to capture the outbound file, but just not quick enough to open the console output, before it is pushed off the top.As we (viewers) see the errors within the WebIDE, and this device is wireless, it might be that Espruino running on the device detects an issue in memory, and then reports that to the WebIDE. This would mean that memory on the device is corrupt, and not a translation issue sending chars to the WebIDE that might become corrupt within the WebIDE memory space, during the transfer process.
Does Espruino perform a checksum test on each line of chars that are sent over BT to validate what is received is what was sent? If yes, then this is more likely a parsing/re-assembly issue.
-
Sun 2020.03.29
two hours later
This time a possible parsing issue.
I stripped out more comments, now making 800+ JSVARS available. I put back in a function that just prints to the console, but on startup am now faced with a 'FIFO_FULL' error. I commented out that L1248 df() function call, and a new error popped up, but that made sense, as I had stripped the setWatch and inlineC section from Test3, under the assumption that maybe there was a parsing issue with the 'C' code. See Test4
New interpreter error: FIFO_FULL >dmf() [ "FIFO_FULL", "LOW_MEMORY" ] =undefined >pm() { "free": 838, "usage": 1662, "total": 2500, "history": 728,
So, I inserted the setWatch and inlineC back in. L225 thru L 244 in Test4aNow the error is cleared. Note that this pushes the entire balance of the file downward, by around twenty lines, as the file now has more code lines. On upload, the error in post #8 is now gone!
Performing a dump also confirms this. L996 dump20200329a1109Test4aFORUM.txt
This now points to a parsing issue.
3 Attachments
-
Ok, so taking a very quick look at your code, what you're doing is:
- Sending out a 2kHz square wave
- Using
setWatchon this square wave which is gated with something.
Given Espruino's execution speed this, all by itself, is going to be quite a stretch. When you add a watch, the events generated when a pin changes state go into the FIFO (the same one used for uploading code). If you don't handle those quickly enough then you get FIFO_FULL.
So... If that code ever gets enabled while you are uploading then it'll interfere with the code upload.
If you had happened to save the code to flash with 'always run at boot' then it'd really mess things up. Or there's a possibility that if it is running, the upload of new code will be affected
Also, just so you know, if you're trying to save memory then adding 'help' functions like this really won't help:
function help() { console.log( " " ); console.log( " " ); console.log( "Tutorial PPI - Programmable Peripheral Interconnect" ); console.log( " for the MDBT42Q Breakout board" ); console.log( " " ); console.log( " " ); console.log( "Single letter command line helper functions" ); // ...If you absolutely feel you need them then you could use templated Strings, which will be a lot easier to write, and will use less memory:
function help() { console.log( ` Tutorial PPI - Programmable Peripheral Interconnect for the MDBT42Q Breakout board Single letter command line helper functions ... `); - Sending out a 2kHz square wave
-
Mon 2020.03.30
This narrative is in response to this answer, but is included here as it contains follow up example output pertinent to the ongoing issue:
'I have never come across a microcontroller with flaky RAM'
Forum post on Sun 29th, March 2020 Is there an exhaustive test to exercise RAM to inspect for flaky cells
I haven't seen a flaky memory cell in twenty years, so I thought unlikely either. But, I had a situation yesterday that nearly identically mirrored the one that caused me to post this thread in the first place.Remember I had a working code file ~2000+ lines for over a week, running with around 200-300 JsVars free. It was close to being stamped 'done' but needed some minor code cleanup, and the addition of console help.
In post #9 I mentioned I inserted back in a small snippet, expanding the file size, and miraculously the dump errors ceased. So I happily continued for around four hours. Then, Whammo! a similar error response then appeared.
Uncaught ReferenceError: "pulseMarkFreq" is not defined at line 422 col 39 analogWrite( pinWaveform, 0, { freq : pulseMarkFreq } );which is eerily similar to the original
Uncaught ReferenceError: "argsStartup" is not defined at line 1145 col 51 console.log( " STARTUP: update: " + argsStartup.update );which could not have been true, as I just uploaded the code file with the var definition there.
I now understand the reason I didn't locate the cause, and why I started this thread was that the varargsStartupdidn't appear in the dump using a visual scan of the console. A search in notepad didn't find it either, as I was looking for the entire string as was the notepad find feature. It would take several days to learn I would find it with a sub-string search.var argsStartup = {};Now doing a sub-string search I was able to locate why Espruino is reporting the error, as that var IS NO LONGER THERE!
dump()var ergsStartup = { "gate": 500, "freq": 200, "update": 1000 };After initialization, not only does the content of the var location change, the name of the var changes also!!
Using:
I looked up the offending char
a 97 61 e 101 65but the insight doesn't occur until converted to it's binary equivalent:
a 0110 0001 e 0110 0101I opened this thread. Then when on a hunch I wondered if RAM/hardware was a problem, I posted the link above for an exhaustive test routine.
From post #9 I felt I had a stable situation and moved on. Until the big surprise:Uncaught ReferenceError: "pulseMarkFreq" is not defined at line 422 col 39 analogWrite( pinWaveform, 0, { freq : pulseMarkFreq } );and found on two separate occassions, but only archive the dump for one:
var pulseMarkFreq = 200;dump
var rulseMarkFreq = 4;and looking those up
p 112 70 r 114 72converted to it's binary equivalent:
p 0111 0000 r 0111 0010It is interesting to note that the changing char is in the fifth location in from the margin, and each discovered case, one and only one bit (but not the same one) is flipping.
I inquired about how dump() actually works at the end of post #8, and have re-inserted it here:
last two pp of post #8
As we (viewers) see the errors within the WebIDE, and this device is wireless, it might be that Espruino running on the device detects an issue in memory, and then reports that to the WebIDE. This would mean that memory on the device is corrupt, and not a translation issue sending chars to the WebIDE that might become corrupt within the WebIDE memory space, during the transfer process.
Does Espruino perform a checksum test on each line of chars that are sent over BT to validate what is received is what was sent? If yes, then this is more likely a parsing/re-assembly issue.
Knowing how dump works is the clue to solving this predicament. Memory is changing at run time, as I have inspected (accurate) a dump, just after upload, checked available memory, observed the var from the command line, started the setInterval and then observed the error and re-verifying the var is subsequently changing. I now take issue with the statement from post #2'I guarantee this is not what's happening in JS'
as here are now three concrete examples.
-
Mon 2020.03.30
response to post #10
Thanks for taking a quick peek @Gordon.
'If you don't handle those quickly enough then you get FIFO_FULL.'
Yes understood from a previous post that suggested a limit of ~100 and I'm well below that with sending out just six pulses at this point.
'So... If that code ever gets enabled while you are uploading then it'll interfere with the code upload.'
This one might agree with an observation I made with PPI that peripherals stay enabled, and a means to disable would be nice.
Tutorial example output anomaly using low level access PPI on MDBT42Q breakout board
'to save the code to flash with 'always run at boot' then it'd really mess things up'
Thanks for the tip/reminder. Purposely not using save() and not using minification to rule those out.
'then adding 'help' functions'
I understand, but mostly needed to assist in debug at this point as I'm not able to recall quickly all the variants that are needed. I tried the Template Literals approach, and would be great if it worked as in that suggestion, but it isn't possible to vertically space for legibility, even using an embedded newline char
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals
console.log( ` Tutorial PPI - Programmable Peripheral Interconnect for the MDBT42Q Breakout board \n \n \n \n Single letter command line helper functions ... `);becomes just:
Tutorial PPI - Programmable Peripheral Interconnect for the MDBT42Q Breakout board Single letter command line helper functions ...and not what we intended. Had to resort back to the tried and true for now to be able to read legibly.
Perusing the forums for the last year regarding setWatch, I have noticed that most are finding trouble attempting to impliment their trials. IMO there is conflict with the channel concept with how setWatch interacts with the underlying PPI. see March 5th post link belowAlthough @Oscar indicated some success, there wasn't a definitive clue that setWatch was mastered
Posted on Thu 2nd, January 2020 Problem with setWatch on MDBT42Q
@barry_b_benson hasn't responded as to the success with a 300msec watch
Posted on Sun 7th, July 2019 Piezo doesn't behave well with setWatch()
I had issues that weren't fully resolved six months ago
Posted on Fri 2nd, August 2019 Proper technique needed to nest a setWatch() within a separate setWatch()
and then again with the issues with PPI and this thread
Posted on Thu 5th, March 2020 Tutorial example output anomaly using low level access PPI on MDBT42Q breakout board
Posted on Fri 27th, March 2020 Technique needed to notify of low memory condition during runtimeI only found one reference to PPI and like myself, @TomWS unsuccessful in a solution
Posted on Sun 3rd, March 2019 Getting very random results using OneWire on MDBT42Q
While the simple examples presented seem to open the door, in practical use, not many have explored this feature, so those issues that I have uncovered most likely hadn't been discovered yet.
-
but it isn't possible to vertically space for legibility
I have no idea what you're doing, but:
console.log( ` Tutorial PPI - Programmable Peripheral Interconnect for the MDBT42Q Breakout board Single letter command line helper functions ... `);prints as:
Tutorial PPI - Programmable Peripheral Interconnect for the MDBT42Q Breakout board Single letter command line helper functions ...as you'd expect. Just tried it and it's perfect.
Thousands of people use setWatch every day - for buttons - and it's fine. So you're obviously doing something different here...
I'm well below that with sending out just six pulses at this point.
If you're using
analogWrite( pinWaveform, 0.2, { freq : pulseMarkFreq } );that is outputting a continuous waveform - not 6 pulses. -
Tue 2020.03.31
'I have no idea what you're doing, but'
Could it be a notepad++ to WebIDE import that strips out the newline? What is it supposed to be?
\nor\r\nor even just\rCould it be a Windows10 clipboard copy issue? That MDN link indicates should be\nbut that isn't even showing in the forum post as that literal.While that block works here in the forum, it isn't within the WebIDE. I copy-n-pasted directly from the section from #10 post in both the editor and console sides. I'll break out the hex editor today.
'So you're obviously doing something different here'
And the reason I stated in the other PPI thread that as there aren't many uisng MDBT42Q that the error(s) is a result of setWatch and underlying channel PPI stuff.
'that is outputting a continuous waveform - not 6 pulses'
If that were the case, I'd be seing 'FIFO_FULL' error flag, which is not occurring. While the tutorial will show all this, the other gated pulse is used to turn on/off that write output. I expect by duration calculation and can see and have verified six on a scope. One reason I slowed the pulses down when the errors started to occur, to make sure pulse edges caught by setWatch were not going over the ~100 detection limit.
-
If that were the case, I'd be seing 'FIFO_FULL' error flag, which is not occurring
Post #9:
New interpreter error: FIFO_FULL >dmf() [ "FIFO_FULL", "LOW_MEMORY" ]Sorry, I'm super busy at the moment and I don't have time to go further with this at the moment. Maybe someone else can jump in
-
Tue 2020.03.31
ref to 'FIFO_FULL' snippet in post #15
The reference to post #9 did occur there, but that was three days and more than ten new file cleanup attempts. In my mind, I was thinking 'CALLBACK' flag and knew I had posted, but as can be seen, not in this thread. It was in fact at:
How to determine which setWatch callback function triggers the E.getErrorFlags() CALLBACK error
So I now understand the response as I hadn't placed that content here, although I knew I had posted it.
'Maybe someone else can jump in'
Yes please, any assistance would be greatly appreciated. Priority at this moment:
Resolution to vertical spacing (blank lines) with Template Literals - confirm w/ file below
EDIT: Thr 20200402 - (blank lines) above To be worked on see post #2:
http://forum.espruino.com/comments/15186872/JsVars calculation analysis in post #3 at:
Posted on Tue 31st, March 2020 How to determine which setWatch callback function triggers the E.getErrorFlags() CALLBACK error
Someone with low level PPI skills to confirm observations left off at:
Posted on Thu 5th, March 2020 Tutorial example output anomaly using low level access PPI on MDBT42Q breakout board
In the mean time, I'll attempt to rip out ALL comments and manage two files, as painful as that will be, and I'll look into uploading not changing code to flash, run that from there and still edit using RAM to continue.
From post #13'prints as as you'd expect. Just tried it and it's perfect'
works on your PC ;-)
So, which WebIDE and OS was this done? Four tests below for me on Windows10 fail.
EDIT: Thr 20200402 - To be worked on see post #2:
http://forum.espruino.com/comments/15186872/Template Literal example post #10 and post #13 is not rendering as such with the
\nnewline stripped and appears like this to me in four separate tests:2v03 (c) 2018 G.Williams Espruino is Open Source. Our work is supported only by sales of official boards and donations: http://espruino.com/Donate > Tutorial PPI - Programmable Peripheral Interconnect for the MDBT42Q Breakout board Single letter command line helper functions ... >dump() function t() { console.log(` Tutorial PPI - Programmable Peripheral Interconnect for the MDBT42Q Breakout board Line 1 Line 2 Single letter command line helper functions ... `); } >process.env ={ VERSION: "2v03", GIT_COMMIT: "e77d74f6", BOARD: "MDBT42Q",It can be observed from the dump, that the newline char isn't preserved within the function itself, and thus when attempted to then render, doesn't include the blank line as desired.
Not seeing blank lines as in that post #13 suggestion:
Tutorial PPI - Programmable Peripheral Interconnect for the MDBT42Q Breakout board Single letter command line helper functions ...
Tested with both a nRF52 MDBT42Q and can see the newline char is sent:Found a prompt... great! Splitting for reset(), delay 250 BT> Sending "\u0010reset();\n" BT> Sending "\u0010\n\u0010\u001b[26dfunction t()" BT> Sending " {\u001b\n\u001b\nconsole.log(`\n" BT> Sending "Tutorial PPI - Progr" BT> Sending "ammable Peripheral I" BT> Sending "nterconnect\n\n for" BT> Sending " the MDBT42Q Breakou" BT> Sending "t board\n\nLine 1\n\nLin" BT> Sending "e 2\n\n\n\nSingle letter" BT> Sending " command line helper" BT> Sending " functions\n...\n`);\u001b\n" BT> Sending "\u001b\n\u001b\n}\n\u0010\u001b[48dconsole." BT> Sending "log(`\nTutorial PPI -" BT> Sending " Programmable Periph" BT> Sending "eral Interconnect\n\n " BT> Sending " for the MDBT42Q B" BT> Sending "reakout board\n\n\n\n\n\nS" BT> Sending "ingle letter command" BT> Sending " line helper functio" BT> Sending "ns\n...\n`);\n\n"and on a STM32 Pico with identical results. Also tested with both the native app and the online IDE
No minification
Test using native app Web IDE version 0.70.6
Test using online Web IDE version 0.73.3EDIT: Thr 20200402 - (blank lines) above To be worked on see post #2:
http://forum.espruino.com/comments/15186872/
1 Attachment
-
Fri 2020.04.03
Finally have made some progress! I have discovered 'WHAT' is going on, but it isn't quite clear 'WHY' the output is behaving the way it is. At the moment, it appears to be how Javascript works, and doesn't appear to be related to the number of pulses seen by setWatch. On a discovery made while using PPI, a hunch I have is to do with the prescaler, as certain frequencies produce very bizarre output data values.
I'll be spending the next few days collecting data ouptut, then creating functions to present that data for analysis. We'll then have the means to test known values and observe both hardware and software output.
In that discovery process, I learned a very valuable lesson with garbage collection. While I understood that Espruino performs garbage collection during idle times and usingprocess.memory()causes a modified garbage collection, I finally understand why free memory was getting chewed up at an alarming rate. Knowing the above, I kept monitoring free JsVars, perform a run, then again observe. I would then cut-n-paste into the Left-hand console side of the WebIDE, Wash, Lather, Rinse, Repeat What wasn't intuitive was why I was forced to re-upload from the Right-hand editor side, always when the 'FIFO_FULL' flag was set. As I had been paying attention to the free JsVars, then making calculations as to the amount of memory that would have been used by collecting data, that never added up, I should have been paying closer attentiton to the number of JsVars used as well. On occassion, a garbage collection by Espruino freed up gobs of space, and it never dawned on me why. That is, until, . . . I observed that the history value would go down. Upon re-visiting the notes beneath the process reference would clarify, and it then was understood that as the cut-n-paste process was gobbling up precious JsVars, then with certain test data, the program couldn't complete, tripping the 'FIFO_FULL' error. lesson learned
Will be busy over the next few days collecting data. . . . stay tuned -
Mon 2020.04.06
Sidebar: flag 'blocksize : Size of a block (variable) in bytes' doesn't appear in results or returns 'undefined' if direct access is attempted: process.memory().blocksize
Also, the 'gc' flag is always zero, and always returns 'undefined' process.memory().gc
Three simple answer questions, no code analysis required. . . .There appears to be an anomaly issuing a gabage collection command.
>pm() { "free": 461, "usage": 2039, "total": 2500, "history": 352, "gc": 0, "gctime": 4.30297851562, "stackEndAddress": 536928592, "flash_start": 0, "flash_binary_end": 428148, "flash_code_start": 442368, "flash_length": 524288 } =undefined
At this point, if I attempt to copy-n-paste a function into the console Left-hand side of the WebIDE, one of two conditions typically occur. The first is that upload is successful, but then are greeted with:=undefined >process.memory() Execution Interrupted >
and no further keystrokes can be entered, or worst case, the function isn't completly copied, thereby locking up the WebIDE.
Reviewing the flag detail, link abovegc : Memory freed during the GC pass
free : Memory that is available to be used (in blocks)
history : Memory used for command history - that is freed if memory is low. Note that this is INCLUDED in the figure for 'free'
Knowing that JsVars are maintained by a table of linked references, and realizing I am uploading thirty lines of dense console.log output, I used the following to understand exactly what the copy-n-paste was doing, regarding memory usage:>E.getSizeOf( function( . . . . ... :} ) =105
Upon upload complete, I'm greeted with 'Execution Interrupted' and no more keystrokes may be entered. This means that the history content is not getting removed before lockup. Only left with: Disconnect device, power down, power up, reconnect and re-upload. Annoying.
Q1: As the 'History' flag indicates that it's value is included in the 'Free' JsVars count, does this mean that the total 'Free' value, is in fact not a true memory available indicator, e.g. 'Memory that is available to be used' is in fact not what the user nor what Espruino actually has available for operations, but must be adjusted by subtracting out the history value? Should that be true, maybe an option 'Avail' or 'Usable' representing 'Free' minus 'History' which is more intuitive? Is there a way to run the garbage collector to just remove the history?While analyzing the gabage collection detail, I discovered another quirk that is also gobbling up memory.
My code file is 28K in size, and takes about 30 seconds to upload over BLE. Once complete, I can inspect and see that 2000 JsVars are being used. As we know:2000 x 16 = 32,000
BT sends out chunks of around 20 chars: WebIDE >> Settings >> ConsoleBT> Sending "gmf();\n console.log"
BT> Sending "( \" L912 free :"
BT> Sending "\" + j );\n\n var len "
BT> Sending "= E.getSizeOf( pulse"
BT> Sending "Array );\n console.l"which shows that around 22 chars max are transfered on each push.
Knowing it takes about 30 seconds to upload, we are transfering roughly 1000 chars / sec
At 8 bits per char = 8000 bits/sec Feq 1 / 8000 = 0.000 125 or 125usec per bit and 1msec per char - this is close to but not the same 9600 BUAD USART rate controllable by the WebIDEThis seems rather slow, 8000bps, one tenth of achievable.
A rather exhaustive and informative read on how complex BLE really is!!
'throughput may theoretically reach the limit of ~230 kbps, but actual applications analyzed in this review show throughputs limited to ~100 kbps; the maximum reachable range is strictly dependent on the radio power,'
Ahhh, in reading over that page, found Section 3 Table 1 that payload is 20 bytes!
Q2: What is the transfer speed of BLE packets uisng the WebIDE?As I realized that console.log statements are eating up memory, and while the WebIDE Template Literal backburner fix is underway, I decided to re-write those functions writing to memory for later analysis.
But a curious thing still happened, yep, still running out of memory. This time though, for a completely different reason. I was attempting to write a JSON name:value string to each memory location, but that was still failing. So I stripped down to something easily understood.
var ary = new Array( 16 );Using E.getSizeOf() and ary.length continued to verify actual usage. Using a known value, in this case from getTime() we know we have a float, and a float takes eight bytes. Three for the JsVar ID and those eight should take 1 JsVar, which it does. I write exactly four floats to that array. Using the above, I am able to iterate over the array, and the length stays at 16.
Now here is where it gets interesting. When I place the same code inside a setWatch(), under some conditions, unkown at this point the array mysteriously jumps in size, despite I never use an indexer that is out of range. I'm only iterating over the first four slots in the array. Some times, the array jumps by ten, sometimes it quadruples. I've even placed conditionals inside, along with console.assert() to absolutely make sure I'm not changing the indexer into that array. Yet, the array increases in size. It is almost as if the underlying PPI hardware continues to fire the setWatch, or the setWatch is responding to the underlying PPI and not the actual state at the pin as the scope sees it.
I tried searching W3C, MDN, W3Schools and other sites to see if I could learn on the criteria for arrays. Years ago, it used to be an additional ten slots were added, but I can't locate under what conditions.
Reading over:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/length
and specifically a suggested section:
Looking over the source:
https://github.com/espruino/Espruino/blob/master/src/jswrap_array.c#L34
I see several slice, join and length references.
Q3: What is the criteria for Espruino to expand an array, and what increase % is used to make that determination? Where in source?
Gordon
MaBe-
Post a reply
About
Technique needed to notify of low memory condition during runtime
Posted by
@Robin
Fri 2020.03.27
I know this is a remaining JSVAR memory issue. I have a rather large app 2300 JSVARs with loads of
helper logging output.
It ran great until I started altering data sets. I started to notice intermittent errors, so started smattering code with try/catch blocks to see if I could catch errors. Minimal luck.
Today, I have a repeatable situation after an execution of a helper function that writes out ~20 consol.log() statements. A single manual invocation works great. Inside a setInterval however, started getting bizarre errors that are never caught by the try/catch blocks. After performing a dump() I can validate that this error does in fact show that previously defined vars are no longer there.
Memory is getting written over. In 'C' it's the dreaded missing null line terminator. Used to malloc heapfree etc.
Using process.memory().free I can determine the amount of JSVARs available before I execute a function, but what is the proper technique to discover a possible memory overrun before it occurs at runtime? Is there a method to know whether a function call is going to require massive memory overhead before the call?
Using a MDBT42Q breakout board