Slow ESP8266 file transfer - buffer size issue?

I've set up an http file server using code similar to the Internet (HTTP) examples.
I wanted to try serving large files direct from my Pico with a SD card module and an Olimex MOD-WIFI-ESP8266-DEV board attached. The idea was to try using the HTML+SVG code examples to make a nice 'shiny' user interface using only the Pico.
Note that I'm only using the 'normal' AT commands on the ESP8266, no IoT firmware functionality (at least not yet!).
It's all working, using the ESP8266WiFi_0v25 module. It's reading files from the SD card and then using pipe to serve the chunks of the files as an http server, but it's very, very slow. It's painful to watch each line of text appear in the browser (for example, a copy of the Espruino Reference HTML page takes something like 25 minutes or so to serve from the Pico!), but it eventually sends the whole page via the ESP8266 without reporting any errors.
I've had a look at the code and played around with the chunkSize until I noticed that the buffer in the socket code seems to be fixed at a maximum of 64 bytes. Anything more than 64 doesn't have any effect. Of course less than 64 means the sending of the file gets even slower, but at least I can see that changing the chunkSize on the pipe has an effect!
According to all the ESP8266 AT command documentation it should be able to transfer 2048 bytes of data in a single call which would be a much better chunkSize.
I don't think it's any kind of firmware problem, at least everything looks clean with the at.debug switched on. But it's worth a mention that I'm using the latest firmware for the Olimex MOD-WIFI-ESP8266-DEV (which has 2MB of flash memory on the board), I'm NOT using the 512KB firmware that is officially supported. AT+GMR returns "AT version:0.30.0.0(Jul 3 2015 19:35:49)"
Is there any quick way around the Espruino 64 byte 'limit' or, if I want to try getting better performance, do I have to bite the bullet and think about handling all the AT commands myself with my own larger data buffer?
Or, more likely, have I simply misunderstood how to correctly use pipe and the chunkSize option?!

What speed is the ESP8266 serial using? It defaults to 9600 baud with most ESP8266 firmware, which is <960 bytes per second - that's pretty slow. There's an AT command to change the serial speed, so what you do I think is send that command, then another call to serial.setup() to do the same on your end.

(it'd 10 bits per byte over serial - start bit, 8 data bit, stop bit for each character, and baud is in bits per second)

The default baud rate in this firmware is 115200 and my Pico serial is already set to that value.
I can really see each chunk 'ticking' into the page in the browser, unfortunately it really is that slow.
As I said the at.debug looks like everything is being handled as expected, and it clearly shows only 64 bytes are ever sent in each send by the ESP8266.

Using Espruino standard or Pico to run a Web server over ESP8266 for 'shiny UIs' is not why Espruino supports http(s) and ESP8266 Wifi...

Just let the sensor data run up this stack and let the browser that will display the few data in a shiny way pull from a server the masses of html5/css3/js to for that shine!

This post shows how (on the example of dojo gauges...). Replace the function in lines 16..21 with sensor reading via an ajax call.

The shown solution has though still challenges: with many 'clients' pulling this data, Espruino/ESP8266 will not be able to keep up. Therefore, choose a strong midtier - for example node.js on a linux box so you can stick with Javascript - on which something UI less runs to pull the data and hold it in mid tier's memory, and all consumers go just for that last information. Such a solution allows also to log the data...

I'm hosting the server on AWS - regardless of the speed, there is no way in hell I want to open up my network to incoming connections, pointed at something that I can't lock down to the max +10.

So I have my Pico periodically checking this URL, and updating it with additional information from sensors, by firing off requests to a php page. Meanwhile, a separate php page can also spit out that information, and the rest of the website is hosted on that system so it can get the data with an xhr. Similarly, commands can be left on the server that get picked up next time it polls.

I only use server mode when it's only taking requests over the LAN (in fact, part of what the project I mentioned will do is relay commands to another espruino device and a non-espruino device on the LAN which are acting as servers (where appropriate get requests turn lights on and off )

But, regardless of the security considerations, it sounds like it's much slower than it should be - I think Gordon is the one best placed to answer this; it may be a bug that each chunk is taking so long - or the chunk size needs to be able to go higher.

Don't Panic! It's just an experiment following Gordon's ideas about what can be achieved with Espruino software together with his boards.
One client (me!) is requesting the html page and sensor data over my home wireless network. It was just to see what is possible, not a real solution.
In the real world just sending the sensor values (in JSON format for simplicity) from the Espruino Pico to a data 'hub' would be much, much more practical.
I was just worried about the overall speed, as something seems wrong.
As DrAzzy said, I was also wondering if it is a Espruino/ESP8266 bug or a feature regarding the very slow transfer rate. Which seems to me to be caused by a small Espruino internal buffer.

Success! At least I know I was on the right trail...
I rebuilt Espruino for my Pico with a socketserver send buffer and chunkSize of 2048 bytes instead of 64. Now my copy of the Espruino Reference HTML file takes about 3 minutes to load from the SD card and appear via the ESP8266 over wifi in the browser. Previously it took around 25 minutes for exactly the same file! ;))
I know I've sacrificed some precious Pico RAM 'just' for a transmit buffer, but the experiment was worth it to learn how to rebuild and very slightly modify Espruino for myself.
I'm just wondering if it's worth adding some optional parameters to the socket functions to point to user-defined buffers (and define their size) so that instead of using the small pre-defined buffer you could, depending on RAM constraints, the hardware modules and the application, define your own buffers.

@veryalien great! I was away last week so didn't get to answer this, but glad you got it sorted.

The 64 byte limit is left over from the CC3000, which required at least 2 large statically allocated buffers, so totally munched up RAM even when it wasn't in use. While the actual overhead of sending a 'send data' command shouldn't be huge, my guess is that the ESP8266 and CC3000 (I'm not sure about the WIZnet - it might be smarter) send a single 64 byte packet over the network each time (rather than waiting and filling up a 1500 byte packet), and that's probably really slow.

I've just made a bug for this so I'll see if I can get the buffer size increased in builds somehow.