How to go about tracking down this error?

OK, I've been doing a lot of testing in the meantime and found:

• it all works nicely if you don't do anything that blocks in req.on('data',...
  
  • so counting bytes; buffering into a string; etc... all works well
    
• things you'd like to do like file i/o don't work so well, causing incoming serial packets from the ESP8266 to get dropped
  
  • creating a file with E.openFile() takes a long time and blocks
    
  • file write with File.write() can cause packet loss, but if you rebuffer into 2048 blocks its quite good, File.pipe(...,{chunkSize:2048}) can do this nicely but...
    
  • File.pipe() never ends because of the issue you've now addressed, so I will test now with this
    
    

If I drop the baudrate of the ESP8266 usart down to 19200, I can make things work fairly reliably. But this speed isn't ideal for more than a proof-of-concept.

The other thing I see happen (sometimes) is long (2 seconds plus) pauses that I assume are garbage collection. Sometimes the streaming of data survives this, sometimes it doesn't and again incoming serial packets from the ESP8266 to get dropped

Is there a way to request the GC to pause and unpause upon request?

My write speed to my SD card is effectively ~650000 bps so the file writing shouldn't be an issue for incoming data at 115200 bps, but something is blocking the serial interrupt and a few packets always get dropped at speeds over 19200 i.e. 38400 or higher.