How to go about tracking down this error?

OK, I've tried both hardware and software SPI and end up with the same problem: I've opened a new post to follow that issue separately, although I believe it is now the root cause of my problems with dropping incoming packets of serial data from the ESP8266.

Meanwhile I'll let you know that the last binary/build that I can use is e54a6b982545da111b2f24d725fb30b0279c11da­ , anything after that has a regression error when trying to require a module

Uncaught SyntaxError: Got ?[255] expected EOF
 at line 1 col 257
...t(),h.cmd(a[0],a[1],a[2]))}ÿsco
                              ^
in function "cmd" called from line 2 col 79
...er AT+RST");else return cb})
                              ^
in function "reset" called from line 1 col 152
...ncs.reset(connectedCallback);return wifiFuncs
                              ^
in function "connect" called from line 24 col 2
});
 ^

you'll notice that \xFF character on line 3 that isn't there, and I don't get this error with the build linked just above.

I like where its heading with detecting the end of the body by comparing the bytes received with the Content-Length header. This is what I've been doing in my javaScript in the req.on('data',... for a while now. It looks like the implementation is still unfinished, and it currently also logs the message Closing now to the console from socketserver.c. And req.pipe(...) still doesn't end as yet.

I'd like to propose a few things at this point:

1. can the (new) chunksize parameter be user-definable if using NewtworkJS type 'drivers'. The place this could go is in the 'driver' module, in my case ESP8266WIFI_0v25 by including a chunksize attribute in the object passed to require("NetworkJS").create(obj), that if present, overrides the default value of 536 specified in network_js.c. As I have shown on line 2 here:

var netCallbacks = {
  chunkSize: 1460,
  create : function(host, port) {
    /* Create a socket and return its index, host is a string, port is an integer.
    If host isn't defined, create a server socket */  
    if (host===undefined) {
.
.
.
  require("NetworkJS").create(netCallbacks­);

I would try and add it myself but I still haven't got my head around handling jsVars in the Espruino source code. This would remove the trouble of having to make a new build to try out different socket buffer sizes.

2. I propose it is a good opportunity to introduce the req.on('end',... event into Espruino. This is how node works with its ReadableStream implementation. Checkout https://nodejs.org/en/docs/guides/anatom­y-of-an-http-transaction/#request-body for some background.

• an http transaction works on a 'half-closed' basis between client and server
  
• the client (half-)closes the transaction after transmitting the request body with a tcp FIN packet
  
• of course devices like the ESP8266 don't make this visible to us, so
  
• it is typical to count the incoming bytes and compare with the Content-Length header and req.emit('end') when everything has arrived
  
• a timeout is probably also required (reset whenever data does arrive) in case the client stops sending, or some of the data never arrives - if there is a timeout, it should req.emit('error')
  
• the server only closes the transaction when it issues a res.end() or the socket is taken down for another reason (its been 'half-closed' until now)
  
• when the client gets the result of the res.end() the http transaction is complete
  
• if the client never gets a complete response (because the socket goes down for another reason) its up to the client to decide how to deal with this
  
  

The upshot of this approach is that the http server does not and should not have to issue a res.end() just to get a close (or end) event and then determine if the the whole of the request body has been received: an end event should happen as soon as the byte count is right.

I've noticed a lot of the sample/tutorial code for an http server does this 'early' res.end() to close the transaction/socket, which is fine if you are only expecting an http request header anyway, but not if you are expecting to receive/stream in an http request body. In fact when the res.end() is done you'd probably see a req.on('close',... event fire but I'll have to check this out in node to be sure.

Gosh this post got long but I really hope to make this work on the Espruino so I can complete my larger project, which by the way is an attempt to implement a light-weight, modular express (V4) like http router for the Espruino to simplify building http server apps along with my first http server app which would be a web based UI to manage content on an attached SD storage card, including drag-and-drop file uploads - so you can see why reliably receiving long http request bodies is important to me. I've got everything else working quite well, its just receiving long request bodies that is still causing problems.