How to go about tracking down this error?

Well, ideally the Network API would allow data to be 'pushed' into it. However I still think the problem is in the way you're using the HTTP code - we need to see your code.

So the way Espruino works is:

• Data comes in via Serial, it's handled in an IRQ and buffered
  
• Each time around the idle loop, Espruino empties that buffered data, calling AT and ESP8266WiFi modules, which put the data into sockData.
  
• Immediately after that in the idle loop, the socket server code in Espruino calls ESP8266WiFi's recv, which then passes chunks of data back to it, where it is passed on via the socket.on('data' handler. However it often only does this 64 bytes at a time.
  
  

So if Espruino isn't busy, it keeps going round the idle loop, emptying the buffers and everything is fine.

However, if your code is taking too long to execute, Espruino doesn't go around the idle loop enough times, and data builds up in the buffers (probably sockData).

To be honest I don't see how redesigning the ESP8266 network will help in this case. It'll mean that the data that comes in just gets pushed straight to your code, where it won't be handled fast enough and Espruino will still fail with out of memory, just in a different place.