Technique needed to notify of low memory condition during runtime

Mon 2020.04.06

Sidebar: flag 'blocksize : Size of a block (variable) in bytes' doesn't appear in results or returns 'undefined' if direct access is attempted:  process.memory().blocksize
Also, the 'gc' flag is always zero, and always returns 'undefined'  process.memory().gc

http://www.espruino.com/Reference#process

Three simple answer questions, no code analysis required. . . .

There appears to be an anomaly issuing a gabage collection command.

>pm()
{ "free": 461, "usage": 2039, "total": 2500, "history": 352,
  "gc": 0, "gctime": 4.30297851562, "stackEndAddress": 536928592, "flash_start": 0, "flash_binary_end": 428148,
  "flash_code_start": 442368, "flash_length": 524288 }
=undefined

At this point, if I attempt to copy-n-paste a function into the console Left-hand side of the WebIDE, one of two conditions typically occur. The first is that upload is successful, but then are greeted with:

=undefined
>process.memory()
Execution Interrupted
> 

and no further keystrokes can be entered, or worst case, the function isn't completly copied, thereby locking up the WebIDE.

Reviewing the flag detail, link above

gc : Memory freed during the GC pass
free : Memory that is available to be used (in blocks)
history : Memory used for command history - that is freed if memory is low. Note that this is INCLUDED in the figure for 'free'

Knowing that JsVars are maintained by a table of linked references, and realizing I am uploading thirty lines of dense console.log output, I used the following to understand exactly what the copy-n-paste was doing, regarding memory usage:

>E.getSizeOf( function( . . . . 

    ...

:} )
=105

Upon upload complete, I'm greeted with 'Execution Interrupted' and no more keystrokes may be entered. This means that the history content is not getting removed before lockup. Only left with: Disconnect device, power down, power up, reconnect and re-upload. Annoying.

Q1: As the 'History' flag indicates that it's value is included in the 'Free' JsVars count, does this mean that the total 'Free' value, is in fact not a true memory available indicator, e.g. 'Memory that is available to be used' is in fact not what the user nor what Espruino actually has available for operations, but must be adjusted by subtracting out the history value? Should that be true, maybe an option 'Avail' or 'Usable' representing 'Free' minus 'History' which is more intuitive? Is there a way to run the garbage collector to just remove the history?

While analyzing the gabage collection detail, I discovered another quirk that is also gobbling up memory.

My code file is 28K in size, and takes about 30 seconds to upload over BLE. Once complete, I can inspect and see that 2000 JsVars are being used. As we know:

2000 x 16 = 32,000

BT sends out chunks of around 20 chars: WebIDE >> Settings >> Console

BT> Sending "gmf();\n console.log"
BT> Sending "( \" L912 free :"
BT> Sending "\" + j );\n\n var len "
BT> Sending "= E.getSizeOf( pulse"
BT> Sending "Array );\n console.l"

which shows that around 22 chars max are transfered on each push.

Knowing it takes about 30 seconds to upload, we are transfering roughly 1000 chars / sec
At 8 bits per char = 8000 bits/sec   Feq 1 / 8000 = 0.000 125 or 125usec per bit and 1msec per char - this is close to but not the same 9600 BUAD USART rate controllable by the WebIDE

This seems rather slow, 8000bps, one tenth of achievable.

A rather exhaustive and informative read on how complex BLE really is!!

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5751532/

'throughput may theoretically reach the limit of ~230 kbps, but actual applications analyzed in this review show throughputs limited to ~100 kbps; the maximum reachable range is strictly dependent on the radio power,'

Ahhh, in reading over that page, found Section 3 Table 1 that payload is 20 bytes!

Q2: What is the transfer speed of BLE packets uisng the WebIDE?

As I realized that console.log statements are eating up memory, and while the WebIDE Template Literal backburner fix is underway, I decided to re-write those functions writing to memory for later analysis.

But a curious thing still happened, yep, still running out of memory. This time though, for a completely different reason. I was attempting to write a JSON name:value string to each memory location, but that was still failing. So I stripped down to something easily understood.

var ary = new Array( 16 );

Using E.getSizeOf() and ary.length continued to verify actual usage. Using a known value, in this case from getTime() we know we have a float, and a float takes eight bytes. Three for the JsVar ID and those eight should take 1 JsVar, which it does. I write exactly four floats to that array. Using the above, I am able to iterate over the array, and the length stays at 16.

Now here is where it gets interesting. When I place the same code inside a setWatch(), under some conditions, unkown at this point the array mysteriously jumps in size, despite I never use an indexer that is out of range. I'm only iterating over the first four slots in the array. Some times, the array jumps by ten, sometimes it quadruples. I've even placed conditionals inside, along with console.assert() to absolutely make sure I'm not changing the indexer into that array. Yet, the array increases in size. It is almost as if the underlying PPI hardware continues to fire the setWatch, or the setWatch is responding to the underlying PPI and not the actual state at the pin as the scope sees it.

I tried searching W3C, MDN, W3Schools and other sites to see if I could learn on the criteria for arrays. Years ago, it used to be an additional ten slots were added, but I can't locate under what conditions.

Reading over:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/length

and specifically a suggested section:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array#Relationship_between_length_and_numerical_properties

Looking over the source:

https://github.com/espruino/Espruino/blob/master/src/jswrap_array.c#L34

I see several slice, join and length references.

Q3: What is the criteria for Espruino to expand an array, and what increase % is used to make that determination? Where in source?