• Thanks for all the thought provoking input. I'll try to summarise what I've learnt in the past 24 hours.

    • @Gordon pointed me in the right direction to the ESP8266WIFI_0v25 module. I'd failed to remember that modules get minified, which explains why I couldn't identify the module myself. So thanks.
    • @DrAzzy, yes this is a Pico with the Esp8266 connected via the shim.
    • So I instrumented the ESP8266WIFI_0v25 module and found that the problem lies there or in the AT module: once the serial data starts to arrive from the Esp8266 the ser.on('data',... gets all the priority and all the incoming data is buffered by the ipdHandler in sockData[sckt].
    • only once all the data is buffered, then the socketServer calls netcallback.recv to start getting chunks to send on to req.on('data',...
    • this is the point where I got the out of memory error: because JavaScript does expensive string copies for functions like String.slice() and String.substr(); as the netcallback.recv returns a chunk from sockData[sckt] it tries to prune the chunk from the buffer causing it to copy the larger remaining part of the buffer

    So there are plenty of places where you could run out of memory with this behaviour, basically because the code doesn't let you process and discard each incoming chunk of data as it arrives, but buffers it and then chunks it up after it is all buffered.

    Ideally the ESP8266WIFI_0v25 module and/or the AT module need to be redesigned to behave as a stream, and not just to emulate a stream by actually buffering everything and then chucking out from the buffer.

    I hope all this makes sense.

About

Avatar for Snerkle @Snerkle started