-
'I could modify dumpArray() to manage the large buffer, but I already know what I'm going to get'
Nothing glaringly obvious in the code snippet, no mathematical limit stands out in the 18,796 value. Don't have the same hardware in order to test on this end. Any chance to humor me (us) and reveal the amount of free memory before and after dumpArray() as in the try/catch above? Are any errors shown then?
Although I didn't spend a ton of time studying the source:
Clicking on the right facing arrow adjacent to the heading at:http://www.espruino.com/Reference#l_E_toUint8Array
hyperslinks to the source:
https://github.com/espruino/Espruino/blob/master/src/jswrap_espruino.c#L826Was wondering if there are constants there that might clue us in.
'I'm not seeing buffer overflows, out-of-memory errors, etc'
It's possible that errors are being thrown, bubbling up and not caught. I find that wrapping every code block within a function, sometimes reveals tid-bits, such as 'undefined' and 'null' which otherwise escape viewing.
EDIT:
Opps, looks like out post edits crossed.'process.memory().free is 1942 at the end of the receipt of data'
Then does the memory error occur when attempting to reveal with console.log() at that point? What remains *should* be able to be viewed; 18796 / 16 = ~1175 which should leave enough ~600 for Espruino housekeeping.
'until it encounters a truncated NRPN message'
What specific bytes are at msg end? Maybe others have seen or know of whether that sequence has been problematic in the past?
Hey, I just noticed in post #13 that some bytes are not printing both numerical values to stay in the 0-255 0x00-0xff range. Could one of those 0x0 be detected as a null and blowing up memory at that point?
You are reading a single comment by @Robin and its replies.
Click here to read the full conversation.
Robin
@Robin,
Thanks a million for your thoughtful responses...
process.memory().free is 1942 at the end of the receipt of data.
No external memory module in use, literally nothing physically attached to the Wifi module.
Re: possible memory allocation issue: in the "real" version of my app, where I'm actually processing the incoming data, I perform memory allocation, enough to hold whatever has arrived in TCPSocket on 'data', but has yet to be processed. You can see that code here. Because I suspected my own code at fault, I commented all that out, and am doing only the console.log() bit described above. So no memory allocation at all. Well, that's a lie - I'm doing 'E.toUint8Array(chunk)' in the console.log(). But still, that's allocating up to a few hundred bytes, not allocating an ever-growing buffer.
RE: looping through the data, yes, that's exactly what my real code does: the TCPSocket on 'data' callback calls handlers that process the data, and spit out a single NRPN (or SysEx) message at a time. And it works flawlessly, until it encounters a truncated NRPN message (as shown previously), then it stops being able to process. The point here is that the issue occurs in every context in which I am receiving the data. I'm not seeing buffer overflows, out-of-memory errors, etc. The behavior, excuse me, behaviour, of the incoming data stream that my processing code sees is identical to the "just dump the buffer" code.
I could modify dumpArray() to manage the large buffer, but I already know what I'm going to get: ~8kb of missing data. The 18,796 byte count (in a 32kb pre-allocated buffer) already proves that.