Filesystem load to RAM vs piping

Looks like the Espruino_WiFi module doesn't like the 1.3.0 AT command set, so I've stuck with 0.50.0 for now.

esptool command to get that going:

esptool.py -p /dev/tty.SLAB_USBtoUART -b 115200 write_flash 0x00000 noboot/eagle.flash.bin 0x40000 noboot/eagle.irom0text.bin 0x7c000 esp_init_data_default.bin 0x3e000 blank.bin 0x7e000 blank.bin

AT+UART_CUR now works, and properly resets to default params when the module reboots. Maximum baud I can get is 1843200, so the command is:

AT+UART_CUR=1843200,8,1,0,0

Nice - thanks! Glad you got it sorted!

And you're using 0.50 from this link? http://bbs.espressif.com/viewtopic.php?f=46&t=1123

Yes, however stability has gone completely downhill, getting frozen http connections and all sorts. Going to try my fresh Espruino WiFi to see if its a firmware issue.

Hmm, interesting.

Sending an index.html file that contains a link to a stylesheet with pipe() without a chunk size, works fine. If the stylesheet is sent using pipe() with a chunk size, all ok.

However, if you send the index.html with a chunk size, the http request for the stylesheet never arrives at the Espruino even though the browser sends it. I think what is happening is that the connection is held open during the pipe, and when another request arrives, the ESP is too busy sending to deal with the arrival.

Also +Gordon, while you're looking, can you give me any more info on https://www.espruino.com/Reference#l_SPI_setup with regards to the "baud" rate? What is the unit there? I see a default of baud:integer=100000. Can you give some wild guesses what I can increase that to in order to speed up the SD card reads?

And that's all happening on a fresh Espruino WiFi with 0.40 firmware, at the default baud rates? But it's serving off the SD card?

Can you check E.getErrorFlags() and see if it reports something? Could be Espruino's internal buffer has got full.

I guess another option is that the browser is attempting to send the request for the CSS on the same connection, and that Espruino is closing the connection right after finishing the HTML? It shouldn't do that though as Espruino reports that it only supports HTTP 1.0.

Can you come up with a really basic no-frills example that exhibits the problem? Even better if you can do it without using an SD card.

SPI "baud" rate? What is the unit there?

It's bits per second. You could give 10000000 a try, but it may depend a little on your wiring. Try reading and listing files first - things that don't write to the SD card :)

I'll try and work up a minimum breaking example tomorrow.

Can you think of any more tips for getting the maximum rates out of the wifi? At the moment I've pushed everything about as far as I think it'll go and I'm getting 16KB in 2.5 seconds, so about 6.4kBps. I must admit I was hoping for something a bit more in the mBps range (although perhaps thats a little over optimistic)... what is likely to be the bottleneck here, the ESP8266 or the Espruino itself?

I've found a thread or two to do with ESP8266 speed, https://github.com/esp8266/Arduino/issues/1853, http://www.esp8266.com/viewtopic.php?f=5&t=245 Espressif quote a slightly higher rate http://bbs.espressif.com/viewtopic.php?t=2187

If I watch the AT debug data, I see 64byte packets being sent more often than not. Is this something I can control?

["AT+CIPSEND=1,131\r\n"
] "\r\nOK\r\n> " <--- "\r\nOK\r\n> "
] "\r\nRecv 131 bytes\r\n" <--- "\r\nRecv 131 bytes\r\n"
] "\r\nSEND OK\r\n" <--- "\r\nSEND OK\r\n"
["AT+CIPSEND=1,64\r\n"
] "\r\nOK\r\n> " <--- "\r\nOK\r\n> "
] "\r\nRecv 64 bytes\r\n\r\nSEND OK\r\n" <--- "\r\nRecv 64 bytes\r\n\r\nSEND OK\r\n"
["AT+CIPSEND=1,64\r\n"
] "\r\nOK\r\n> " <--- "\r\nOK\r\n> "
] "\r\nRecv 64 bytes\r\n" <--- "\r\nRecv 64 bytes\r\n"
] "\r\nSEND OK\r\n" <--- "\r\nSEND OK\r\n"
["AT+CIPSEND=1,64\r\n"
] "\r\nOK\r\n> " <--- "\r\nOK\r\n> "

According to the docs for AT+CIPSEND I think it will take up to 2048, so 64 is a bit tiny! http://www.ctr-electronics.com/downloads/pdf/4A-ESP8266__AT_Instruction_Set__EN_v0.40.pdf

I must admit I was hoping for something a bit more in the mBps range

Yes, you're not going to get close to that I'm afraid! If you want that kind of thing IMO you really need to be looking at much more heavyweight, linux-based systems.

I believe that page could be quoting bits per second, so at 300kbits you're still only transmitting 30k bytes per second. Given Espruino's having to read from the SD card as well, 8kbytes per second isn't that unreasonable.

what is likely to be the bottleneck here?

The serial comms to the ESP8266 won't be helping at all, but the throughput from the SD card really isn't amazing either. Unfortunately you can't wind the serial baud rate right up because then it's possible that Espruino can miss characters that come in, and it gets unstable.

If I watch the AT debug data, I see 64byte packets being sent

That is strange - did you remove the chunkSize from Pipe again? It should send around 530 byte chunks each time if left to its own accord, so with the chunkSize set to 512 the FAT sector size and the amount of data sent at once should match up pretty well.

It's worth noting that with at.debug on you might well notice some instability/slowdowns - since Espruino is having to relay everything that happens on the serial link over USB, buffers could get full and it might miss something.

Yes at.debug definitely causes some issues, its only switched on when I'm trying to debug something :D

chunkSize was definitely still set. I'm trying to figure out a way to sniff the traffic without at.debug. I guess wireshark is my best bet but I'm running short on time, need to get this thing working.

Going to try serving gzipped files and see if the size reduction will make this viable. I definitely don't want to switch away from the Espruino... maybe one day you will release an Espruino-Mega ;)

I guess you could modify the EspruinoWiFi library send and recv handlers to dump simple information like sent X bytes, recv X bytes - that might help without spamming too much information?