1. Home
  2. Other Boards
  3. ESP8266

How to send a large amount of data faster ?

Posted on
Page
of 2
/ 2
Next
  • • #1

    Hi all,
    I am working on a webserver using espruino and the esp8266, to store sensor data in the flash, and serve it to a web browser. But the network seems very slow, e.g. about 1kb per second, depending on the size of each write, much slower than when using Arduino. In this example, it takes 15second to serve 20kb :

    function onPageRequest(req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  var n = 4, chk=5*1024, t=getTime();
  res.on('drain',function() {
    if (n==0) res.end();
    else {  
          res.write(E.toString(new Uint8Array(chk)));  
         n--;  }
 
    print("t:",(getTime()-t).toFixed(3),"sec­");
  });
 
}
require('http').createServer(onPageReque­st).listen(8080);

     _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v95.23 Copyright 2017 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:512/512, manuf 0xef chip 0x4016
>
=undefined
t: 0.021 sec
t: 5.057 sec
t: 9.288 sec
t: 13.494 sec
t: 17.717 sec
>

    I tried with Arduino, it was 100x faster, and served 1.5Mb in the same 15 seconds. So it cannot be a hardware limitation. Is there a workaround or another way to send data faster with espruino ?

  • • #2

    ...no surprise to me... Arduino leaves the ESP8266 alone to do just its (Wifi) work... running now the serving and everything on ESP8266 - and in JS - ... it could be a buffer size issue (in general and in particular in comm w/ Espruino and ESP8266 Wifi firmware... RAM is after all limited... now it does not only serve the Wifi send and receive, but all Espruino. --- I amy be totally mistaken, though...

  • • #3

    You mean there is no work around or better way to send faster on espruino ?

    To download its usable 3 mb of data, it would take ... 50 minutes. :(

  • • #4

    The drain event might not send until a buffer is full...

    What happens if you send all of the sequence in one go?

  • • #5

    Is it possible to flush this buffer to be sent immediately somehow ? or get called until the buffer is really full ?
    As if it is making an unnecessary break before each send. If I reduce the chunk size from 5kb to 256bytes, the average speed is dramatically decreased.

    I can't send 20kb in one go, I get "New interpreter error: LOW_MEMORY,MEMORY". If I send only 1 chunk of 5bk, it still takes 5 seconds even straight away without ondrain:

    function onPageRequest(req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  var n = 1, chk=5*1024, t=getTime();
  res.write(E.toString(new Uint8Array(chk))); 
  res.end();
}
require('http').createServer(onPageReque­st).listen(8080);

    It actually seems hanging few seconds before sending anything.

  • • #6

    I'm trying to visualize your hardware setup.

    1. Esp8266 with Espruino flashed into the Esp8266
    2. Esp8266 connected via serial port to a Pico, Espruino board or Arudino.

    The first case would be slower because the uP has to handle more tasks (marshall data to send, and send data)
    The second cases have more than one uP to divide up the tasks.

  • • #7

    I am in the first case: Esp8266 with Espruino flashed into the Esp8266. By marshalling data, I assume you refer to the E.toString. It's not much different without marshalling. The loop finish in only 13ms with a straight string (instead of 26ms for uint8array), but the 5kb of data still make 5 seconds to finish loading in the browser, in at least ten pieces :

    var data="";
var chk=1024;
for(var i=0;i<chk;i++) data+="0";

function onPageRequest(req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  var n= 5, t=getTime();
  for(var i=0;i<n;i++) {
    res.write(data,'binary'); 
    print("t:",(getTime()-t).toFixed(3),"sec­");
  }
  res.end();
}
require('http').createServer(onPageReque­st).listen(8080);

     _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v95.23 Copyright 2017 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:512/512, manuf 0xef chip 0x4016
>
=undefined
t: 0.002 sec
t: 0.005 sec
t: 0.008 sec
t: 0.010 sec
t: 0.013 sec
t: 0.002 sec
t: 0.005 sec
t: 0.008 sec
t: 0.010 sec
t: 0.013 sec
  • • #8

    We you test it using Arduino, is the Arduino flashed onto the Esp8266?

  • • #9

    @ClearMemory041063, not sure I understand correctly, yes I tested with an arduino program flashed on the esp8266. With arduino, I believe there is no other way than to flash my program in there.

  • • #10

    Please post the Arduino code for comparison.

    Here is another approach:

    Try moving the new UintArray(chk) to a Global scope

    //Global scope
var n = 4, chk=5*1024;
var tdata=new Uint8Array(chk);

function onPageRequest(req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  var t=getTime();
  res.on('drain',function() {
  print("n=",n,"t:",(getTime()-t).toFixed(­3),"sec");
  if (n===0){ res.end();
   print("n=",n,"t:",(getTime()-t).toFixed(­3),"sec");
  }else {  
   res.write(E.toString(tdata));  
    n--;
   }//end else
  });//end opr
 }//end function

    IP=  192.168.1.9
null
Wi-Fi Connected
n= 4 t: 0.008 sec
n= 3 t: 1.097 sec
n= 2 t: 2.214 sec
n= 1 t: 3.447 sec
  • • #11

    Try moving some data to see how much is being moved

    var n = 4, chk=5*1024;
var tdata=new Uint8Array(chk);
var i;
for(i=0;i<1024;i++){
  tdata[i]=0x30;
  tdata[i+1024]=0x31;
  tdata[i+1024*2]=0x32;
  tdata[i+1024*3]=0x33;
  tdata[i+1024*4]=0x34;
}//next i

function onPageRequest(req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  var t=getTime();
  res.on('drain',function() {
  print("n=",n,"t:",(getTime()-t).toFixed(­3),"sec");
  if (n===0){ res.end();
   print("n=",n,"t:",(getTime()-t).toFixed(­3),"sec");
  }else {  
   res.write(E.toString(tdata));  
    n--;
   }//end else
  });//end opr
 }//end function

    IP=  192.168.1.9
null
Wi-Fi Connected
n= 4 t: 0.008 sec
n= 3 t: 1.092 sec
n= 2 t: 2.859 sec
n= 1 t: 4.056 sec
n= 0 t: 5.809 sec
n= 0 t: 5.811 sec
n= 0 t: 0.008 sec
n= 0 t: 0.009 sec

    Saved the page as test.txt and it contains 20 kB of data.

    1 Attachment

  • • #12

    @ClearMemory041063, I am surprised it goes a bit faster on your setup. I still got 17 sec for the 20kb, when I tried the two programs you posted. I tried with/out AP/station, no change. What version of espruino are you running ?
    The only change I did was 5*4kb chunk, instead of 4*5kb chunks, as the latter stopped after n=3 (I guess a memory shortage).

    var n = 5, chk=4*1024;
var tdata=new Uint8Array(chk);
var i;
for(i=0;i<1024;i++){
  tdata[i]=0x30;
  tdata[i+1024]=0x31;
  tdata[i+1024*2]=0x32;
  tdata[i+1024*3]=0x33;
//  tdata[i+1024*4]=0x34;
}//next i
function onPageRequest(req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  var t=getTime();
  n=5;
  res.on('drain',function() {
      print("n=",n,"t:",(getTime()-t).toFixed(­3),"sec");
      if (n===0){ 
        res.end();
        print("n=",n,"t:",(getTime()-t).toFixed(­3),"sec");
      }else {  
        res.write(E.toString(tdata));  
        n--;
   }//end else
  });//end opr
 }//end function
require('http').createServer(onPageReque­st).listen(80);

     _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v95.23 Copyright 2017 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:512/512, manuf 0xef chip 0x4016
>
=undefined
n= 5 t: 0.004 sec
n= 4 t: 4.339 sec
n= 3 t: 7.640 sec
n= 2 t: 10.941 sec
n= 1 t: 14.263 sec
n= 0 t: 17.558 sec
n= 0 t: 17.559 sec

    var n = 5, chk=4*1024;
var tdata=new Uint8Array(chk);
function onPageRequest(req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  var t=getTime();
  res.on('drain',function() {
  print("n=",n,"t:",(getTime()-t).toFixed(­3),"sec");
  if (n===0){ res.end();
   print("n=",n,"t:",(getTime()-t).toFixed(­3),"sec");
  }else {  
   res.write(E.toString(tdata));  
    n--;
   }//end else
  });//end opr
 }//end function

require('http').createServer(onPageReque­st).listen(80);

     _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v95.23 Copyright 2017 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:512/512, manuf 0xef chip 0x4016
>
=undefined
n= 5 t: 0.003 sec
n= 4 t: 3.330 sec
n= 3 t: 6.626 sec
n= 2 t: 9.945 sec
n= 1 t: 13.246 sec
n= 0 t: 16.599 sec
n= 0 t: 16.601 sec
  • • #13

    The arduino code I used is from this tutorial : https://www.hackster.io/rayburne/nodemcu­-esp8266-remote-temperature-using-websoc­kets-5956c4
    The code make a webserver serving files on the flash with spiffs, I just added a 1mb file, and I get it in less than 13 seconds, I just tried again.

  • • #14

    Problem identified

    the res.on('drain') is slow to trigger.
    Hypothesis Somewhere in the firmware is a setInterval that is used to test if the drain buffer is empty. Without delving into the source code, there seems to be an issue with the frequency that the drain empty condition is tested and an emit to the ondrain is issued.
    The Aruduino code runs faster because the drain empty condition is recognised immediately

    The following code was executed on a Pico connected via serial port to an Esp8266 at 115 Kbaud. It uses a setTimeout function to issue emit commands at a higher rate. It doesn't test for the amount of data in the drain buffer and thus is an incomplete fix.

    var n = 4, chk=5*1024;
var tdata=new Uint8Array(chk);
var i;
for(i=0;i<1024;i++){
  tdata[i]=0x30;
  tdata[i+1024]=0x31;
  tdata[i+1024*2]=0x32;
  tdata[i+1024*3]=0x33;
  tdata[i+1024*4]=0x34;
}//next i

function onPageRequest(req, res) {
 res.writeHead(200, {'Content-Type': 'text/plain'});
 var t=getTime();
 res.on('drain',function() {
  print("n=",n,"t:",(getTime()-t).toFixed(­3),"sec");
  if (n===0){ res.end();
   print("n=",n,"t:",(getTime()-t).toFixed(­3),"sec");
  }else {  
   res.write(E.toString(tdata));  
    n--;
   setTimeout(function () {
//    console.log("Hello World");
    res.emit('drain');
   }, 400);
    }//end else
  });//end opr
 }//end function

    1v95 Copyright 2017 G.Williams
>2
Start connection process
end require
=undefined
Reset the ESP8266
end reset
Test for error
Try Connecting to WiFi  faux
end connect
Test for error
Test for error
Test for error
null
end getIP
IP=  192.168.1.9
null
Wi-Fi Connected
n= 4 t: 0.008 sec
n= 3 t: 0.443 sec
n= 2 t: 0.883 sec
n= 1 t: 1.305 sec
n= 0 t: 1.742 sec
n= 0 t: 1.743 sec
n= 0 t: 0.008 sec
n= 0 t: 0.009 sec

    @Gordon any comments on the hypothesis?

    @Polypod please give it a try on your Esp8266 with flashed Espruino and see if you can get similar results.

    I just saw your latest post about the Arduino using Websockets and not HTML. My limited experience with websockets left me with the impression that they are much faster than using HTML.

  • • #15

    I'd be very wary about comparing Espruino running on the ESP8266 and running on something else connected to an ESP8266 by AT commands.

    drain is fired during HTTP handling when the send buffer is empty, not on a timeout: https://github.com/espruino/Espruino/blo­b/09412d6a02830abeee16f9582c8c4c7cf0ea4d­f9/libs/network/socketserver.c#L277

    So it'll be very marginally faster if you keep the buffer full all the time.

    I'd be pretty sure your issue is https://github.com/espruino/Espruino/blo­b/09412d6a02830abeee16f9582c8c4c7cf0ea4d­f9/libs/network/esp8266/network_esp8266.­c#L763

    the chunk size is 268 bytes, so it's only sending that amount each time - which probably results in a whole bunch of small TCP/IP packets being sent, rather than a few large ones.

    Even so, 5 secs is particularly slow. You could take a look at req and res. In one of those there should be a field called dSnd which is the sent data buffer. You could check in an interval and see how quickly that's being emptied.

  • • #16

    I hope we can raise this upload speed limit I am reaching.
    @Gordon, it looks like the chunk size you mention 256 bytes every 200ms, that's where my ~1kb per second comes from.
    @ClearMemory041063, I tried your code, but it runs after 2 steps in a "Execution Interrupted New interpreter error: LOW_MEMORY,MEMORY".
    There is another strange thing, drain get called one more times after res.end() has been called.
    Here's the code that looks into the dSnd, 12.5sec for 15ko :

    var n = 5, chk=3*1024;
var tdata=new Uint8Array(chk);
var i;
for(i=0;i<1024;i++){
  tdata[i]=0x30;
  tdata[i+1024]=0x31;
  tdata[i+1024*2]=0x32;
//  tdata[i+1024*3]=0x33;
//  tdata[i+1024*4]=0x34;
}//next i
function onPageRequest(req, res) {
 res.writeHead(200, {'Content-Type': 'text/plain'});
 var t=getTime();
 res.on('drain',function() {
  print("n=", n,"t:",(getTime()-t).toFixed(3),"sec");
   if (n===0){ 
    res.end();
    clearInterval(inter);
   print("end n=",n,"t:",(getTime()-t).toFixed(3),"sec­");
  } else {
   res.write(E.toString(tdata));  
    n--;
  }
   //end else
  });//end opr
  inter=setInterval(function () {
     console.log("--t:",(getTime()-t).toFixed­(3),"res",res.dSnd.length);  
   }, 100);

 }//end function

require('http').createServer(onPageReque­st).listen(80);

     _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v95.23 Copyright 2017 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:512/512, manuf 0xef chip 0x4016
>
=undefined
n= 5 t: 0.004 sec
--t: 0.103 res 3072
--t: 0.203 res 3072
--t: 0.303 res 2804
--t: 0.403 res 2804
--t: 0.503 res 2536
--t: 0.603 res 2536
--t: 0.703 res 2268
--t: 0.803 res 2268
--t: 0.903 res 2000
--t: 1.003 res 2000
--t: 1.103 res 1732
--t: 1.203 res 1732
--t: 1.303 res 1732
--t: 1.403 res 1464
--t: 1.503 res 1464
--t: 1.603 res 1196
--t: 1.703 res 1196
--t: 1.803 res 928
--t: 1.903 res 928
--t: 2.003 res 660
--t: 2.103 res 660
--t: 2.203 res 392
--t: 2.303 res 392
--t: 2.403 res 124
--t: 2.503 res 124
n= 4 t: 2.589 sec
--t: 2.603 res 3072
--t: 2.703 res 3072
--t: 2.803 res 3072
--t: 2.903 res 2804
--t: 3.003 res 2804
--t: 3.103 res 2536
--t: 3.203 res 2536
--t: 3.303 res 2268
--t: 3.403 res 2268
--t: 3.503 res 2000
--t: 3.603 res 2000
--t: 3.703 res 1732
--t: 3.803 res 1732
--t: 3.903 res 1464
--t: 4.003 res 1464
--t: 4.103 res 1196
--t: 4.203 res 1196
--t: 4.303 res 928
--t: 4.403 res 928
--t: 4.503 res 660
--t: 4.603 res 660
--t: 4.703 res 660
--t: 4.803 res 392
--t: 4.903 res 392
--t: 5.003 res 124
--t: 5.103 res 124
n= 3 t: 5.118 sec
--t: 5.203 res 3072
--t: 5.303 res 3072
--t: 5.403 res 2804
--t: 5.503 res 2804
--t: 5.603 res 2536
--t: 5.703 res 2536
--t: 5.803 res 2268
--t: 5.903 res 2268
--t: 6.003 res 2000
--t: 6.103 res 2000
--t: 6.203 res 1732
--t: 6.303 res 1732
--t: 6.403 res 1464
--t: 6.503 res 1464
--t: 6.603 res 1196
--t: 6.703 res 1196
--t: 6.803 res 928
--t: 6.903 res 928
--t: 7.003 res 660
--t: 7.103 res 660
--t: 7.203 res 660
--t: 7.303 res 392
--t: 7.403 res 392
--t: 7.503 res 124
--t: 7.602 res 124
n= 2 t: 7.619 sec
--t: 7.703 res 3072
--t: 7.803 res 3072
--t: 7.903 res 2804
--t: 8.003 res 2804
--t: 8.103 res 2536
--t: 8.203 res 2536
--t: 8.303 res 2268
--t: 8.403 res 2268
--t: 8.503 res 2000
--t: 8.603 res 2000
--t: 8.703 res 2000
--t: 8.803 res 1732
--t: 8.903 res 1732
--t: 9.003 res 1464
--t: 9.103 res 1464
--t: 9.203 res 1196
--t: 9.303 res 1196
--t: 9.403 res 928
--t: 9.503 res 928
--t: 9.603 res 660
--t: 9.703 res 660
--t: 9.803 res 392
--t: 9.903 res 392
--t: 10.003 res 124
--t: 10.103 res 124
n= 1 t: 10.176 sec
--t: 10.203 res 3072
--t: 10.303 res 3072
--t: 10.403 res 2804
--t: 10.503 res 2804
--t: 10.603 res 2804
--t: 10.703 res 2536
--t: 10.803 res 2536
--t: 10.903 res 2268
--t: 11.003 res 2268
--t: 11.103 res 2000
--t: 11.203 res 2000
--t: 11.303 res 1732
--t: 11.403 res 1732
--t: 11.503 res 1464
--t: 11.603 res 1464
--t: 11.703 res 1464
--t: 11.803 res 1196
--t: 11.903 res 1196
--t: 12.003 res 928
--t: 12.103 res 928
--t: 12.203 res 660
--t: 12.303 res 660
--t: 12.403 res 392
--t: 12.503 res 392
--t: 12.603 res 124
--t: 12.703 res 124
n= 0 t: 12.767 sec
end n= 0 t: 12.769 sec
n= 0 t: 0.004 sec
end n= 0 t: 0.006 sec
  • • #17

    Try using websockets

    Webserver method

    Websockets

    var n = 4, chk=5*1024;
var tdata=new Uint8Array(chk);
var i;
for(i=0;i<1024;i++){
  tdata[i]=0x30;
  tdata[i+1024]=0x31;
  tdata[i+1024*2]=0x32;
  tdata[i+1024*3]=0x33;
  tdata[i+1024*4]=0x34;
}//next i

var page = '<html><body><script>var ws;setTimeout(function(){';
page += 'ws = new WebSocket("ws://" + location.host + "/my_websocket", "protocolOne");';
page += 'ws.onmessage = function (event) { document.write("MSG:"+event.data); };';
page += 'setTimeout(function() { ws.send("Hello to Espruino!"); }, 1000);';
page += '},1000);</script></body></html>';

function onPageRequest(req, res) {
  res.writeHead(200, {'Content-Type': 'text/html'});
  res.end(page);
}
//////
function test(){
if(Hardware===1)Serial=Serial2;
if(Hardware===0)Serial=Serial4;

if(Hardware===1){
 digitalWrite(B9,1); // enable on Pico Shim V2
 Serial.setup(115200, { rx: A3, tx : A2 }); //Pico
}
if(Hardware===0)Serial.setup(115200, { rx: C11, tx : C10 }); 
//espruino board

console.log("Start connection process");
var wifi = require("ESP8266WiFi_0v25").connect(Seri­al, function(err) {
 if (err){Startagain=1;return;}
 console.log("Reset the ESP8266");
 wifi.reset(function(err) {
  if (err){Startagain=1;return;}
  console.log("Try Connecting to WiFi ",hotspots[hotspot].SSID);

  wifi.connect(hotspots[hotspot].SSID,hots­pots[hotspot].key, function(err) {console.log(err);
   if (err){Startagain=1;return;
}

    wifi.getIP(function(l,ip){
     if (err){Startagain=1;return;}
     console.log("IP= ",ip,"\n\r"+l);
     console.log("Wi-Fi Connected");
     clearInterval( myinterval);

var server = require('ws').createServer(onPageRequest­);
server.listen(8080);
server.on("websocket", function(ws) {
    ws.on('message',function(msg) { print("[WS] "+JSON.stringify(msg)); });
 var t=getTime();
  ws.send(E.toString(tdata));
  print("n=",n,"t:",(getTime()-t).toFixed(­3),"sec");
});

  });//end getIP
  console.log("end getIP");
 });//end connect
 console.log("end connect");
});//end reset
 console.log("end reset");
});//end require
  console.log("end require");
}//end test

     1v95 Copyright 2017 G.Williams
>2
>Start connection process
end require
=undefined
Reset the ESP8266
end reset
Test for error
Try Connecting to WiFi  faux
end connect
Test for error
Test for error
Test for error
null
end getIP
IP=  192.168.1.9
null
Wi-Fi Connected
n= 4 t: 0.029 sec
[WS] "Hello to Espruino!"
>
  • • #18

    Thanks @ClearMemory041063 for giving attention to my problem, and for the codes to try. You are not testing them on an esp8266, are you ? The last code makes 11,5kb minified, it does not fit, it interrupt the transfer from the IDE. So I had to remove the connecting code and reduce the data to send to 3kb. I cannot send more at once without getting memory errors.

    Unfortunately, at the end, the result with ws is as slow as with http: 2.5sec for 3kb (see the code below). Maybe because ws server is built on top of an http server.

    var n = 5, chk=3*1024;
var tdata=new Uint8Array(chk);
var i;
for(i=0;i<1024;i++){
  tdata[i]=0x30;
  tdata[i+1024]=0x31;
//  tdata[i+1024*2]=0x32;
//  tdata[i+1024*3]=0x33;
//  tdata[i+1024*4]=0x34;
}//next i
 
var page = '<html><body><script>var ws;setTimeout(function(){';
page += 'ws = new WebSocket("ws://" + location.host + "/my_websocket", "protocolOne");';
page += 'var t=(new Date()).getTime();ws.onmessage = function (event) { console.log("t:",(new Date()).getTime()-t,"MSG:",event.data); };';
page += 'setTimeout(function() { ws.send("Hello to Espruino!"); }, 1000);';
page += '},1000);</script></body></html>';

function onPageRequest(req, res) {
  res.writeHead(200, {'Content-Type': 'text/html'});
  res.end(page);
}

var server = require('ws').createServer(onPageRequest­);
server.listen(80);
server.on("websocket", function(ws) {
    ws.on('message',function(msg) { print("[WS] "+JSON.stringify(msg)); });
  var t=getTime();
  ws.send(E.toString(tdata));
  print("n=",n,"t:",(getTime()-t).toFixed(­3),"sec");
  var inter=setInterval(function(){
    var l=ws.socket.dSnd.length;
    print("t",getTime()-t,"l",l);  
    if(l===0) clearInterval(inter);
  },100);
 //   ws.send("Hello from Espruino!");
});

     _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v95.155 Copyright 2017 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:512/512, manuf 0xef chip 0x4016
>
=undefined
n= 5 t: 0.018 sec
t 0.12109500000 l 2984
t 0.22103200000 l 2984
t 0.32097500000 l 2716
t 0.420522 l 2716
t 0.52113300000 l 2448
t 0.62102500000 l 2448
t 0.72063500000 l 2180
t 0.82108700000 l 2180
t 0.921061 l 1912
[WS] "Hello to Espruino!"
t 1.020693 l 1644
t 1.12096200000 l 1644
t 1.22103700000 l 1376
t 1.32081700000 l 1376
t 1.42095800000 l 1108
t 1.52093400000 l 1108
t 1.62055600000 l 840
t 1.72074900000 l 840
t 1.82093100000 l 840
t 1.92100099999 l 572
t 2.020918 l 572
t 2.12098700000 l 304
t 2.220733 l 304
t 2.320561 l 36
t 2.42057400000 l 36
t 2.52083200000 l 0
  • • #19

    Hi - i tried the code above on the linux version of espruino.

    In the browser - I get an empty page, and only one line of output in the console:

     _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v94.213 Copyright 2017 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
>
=undefined
n= 5 t: 0.001 sec
t 0.10073614120 l 0
[WS] "Hello to Espruino!"
>

    How are you getting mutiple lines output?

  • • #20

    The other lines comes from the interval every 100ms looking and printing how long is the dSnd buffer, and I see it decreasing in my output. Strange that the interval does not seem to start at all in your setup.

  • • #21

    Running on Sparkfun Thing -dev ESP8266 board

    The HTML code splits data into lines in this one.

    //WSDesp8266 24Jan2018

//http//192.168.1.6:8080
//ESP8266 with Espruino flashed


// list of Wifi and passwords
var SSID="xxxxxx";
var key="keykeykey";
var Startagain=0;
var myinterval;

var n =4, chk=2*1024;
var tdata=new Uint8Array(chk);
var i;
for(i=0;i<1024;i++){
  tdata[i]=0x30;
  tdata[i+1024]=0x31;
}//next i

var page = 
"<html>\r\n<body>\r\n<textarea id=\"demo\" rows=\"32\" cols=\"64\"></textarea>\r\n<script>\r\nv­ar ws;\r\nvar data=\"\";\r\n//setTimeout(function(){\r­\nws = new WebSocket(\"ws://\" + location.host + \"/my_websocket\", \"protocolOne\");\r\nws.onmessage = function (event) { \r\ndata+=event.data;\r\ndocument.getEle­mentById(\"demo\").innerHTML = data;\r\nws.send(\"Hello to Espruino!\"); \r\n };\r\n//setTimeout(function() { ws.send(\"Hello to Espruino!\"); }, //1000);}\r\n//,1000);\r\n\r\n</script><­/body>\r\n</html>\r\n"
;
function onPageRequest(req, res) {
  res.writeHead(200, {'Content-Type': 'text/html'});
  res.end(page);
}

function test(){

console.log("Start connection process");
var wifi = require("Wifi");
 console.log("Try Connecting to WiFi ",SSID);
 wifi.connect(SSID,{password:key}, function(err) {
  console.log(err);
   if (err){Startagain=1;return;
}
 console.log("connected? err=", err, "info=", wifi.getIP());
     console.log("Wi-Fi Connected");
     clearInterval( myinterval);
     var t=getTime();
var server = require('ws').createServer(onPageRequest­);
server.listen(8080);
server.on("websocket", function(ws) {
    ws.on('message',function(msg) {
    n--;
  print("n=",n,"t:",(getTime()-t).toFixed(­3),"sec");
      print("[WS] "+JSON.stringify(msg));
      if(n) ws.send(E.toString(tdata));
    });
  t=getTime();
  ws.send(E.toString(tdata));
  print("n=",n,"t:",(getTime()-t).toFixed(­3),"sec");
});
 });//end connect
}//end test

myinterval=setInterval(function () {
  console.log("Test for error");
  if(Startagain){
   Startagain=0;
   test();
  }//end of Startagain
}, 2000);

test();

    Sending 2 kbytes of data 4 times

     1v95 Copyright 2017 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 512KB:256/256, manuf 0x1f chip 0x8401
>Start connection process
Try Connecting to WiFi  faux
=undefined
null
connected? err= null info= {
  "ip": "192.168.1.12",
  "netmask": "255.255.255.0",
  "gw": "192.168.1.1",
  "mac": "5c:cf:7f:8b:1b:99"
 }
Wi-Fi Connected
n= 4 t: 0.015 sec
n= 3 t: 0.118 sec
[WS] "Hello to Espruino!"
n= 2 t: 0.209 sec
[WS] "Hello to Espruino!"
n= 1 t: 0.327 sec
[WS] "Hello to Espruino!"
n= 0 t: 0.426 sec
[WS] "Hello to Espruino!"
>

    8192 bytes sent
    0.426 - 0.015 = 0.411 seconds
    8192 / 0.411 = 19,931.9 bytes per second
    19931.9 * 8 = 159,454.9 bits per second

  • • #22

    Hmm, that's interesting, it seems hardware related. Your Sparkfun Thing dev-board rocks with espruino !
    I don't get your result, with both my poor nodemcus-e12 v3 with 1.95.0 not 4mb , and my v4 lolin with 1.95.20 4mb.
    Where can I find this flash map 256/256 you used ? I'm still getting data as slow as before:

     _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v95 Copyright 2017 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:512/512, manuf 0xef chip 0x4016
>Start connection process
Try Connecting to WiFi  Gardening, cheaper than therapy
=undefined
Test for error
Test for error
Test for error
Test for error
Test for error
null
connected? err= null info= {
  "ip": "192.168.2.8",
  "netmask": "255.255.255.0",
  "gw": "192.168.2.1",
  "mac": "a0:20:a6:14:52:af"
 }
Wi-Fi Connected
n= 4 t: 0.015 sec
n= 3 t: 1.701 sec
[WS] "Hello to Espruino!"
n= 2 t: 3.207 sec
[WS] "Hello to Espruino!"
n= 1 t: 4.681 sec
[WS] "Hello to Espruino!"
n= 0 t: 6.207 sec
[WS] "Hello to Espruino!"
  • • #23

    Sparkfun Thing

    Sparkfun Thing

    Flashing Software

    I'm using a Windows 10 platform.
    To get the Thing into programming mode, ground the 0 pin and plug it in.
    I use Putty at 74,880 Baud to see th eBootloader message.

    Info

    ExpressIF flasher software

    I'm not sure if it matters but I first installed the latest AT software on the thing.
    The latest ExpressIF AT firmware

    Latest Espruino Download

    For the Thing I used the code in the "espruino_1v95_esp8266" folder
    "C:\Users\jj\Documents\espruinoEsp8266Fl­ash\espruino_1v95_esp8266"
    "C:\Users\jj\Documents\espruinoEsp8266Fl­ash\espruino_1v95_esp8266_4mb"

    The ExpressIF flashing software uses a file named
    "C:\Users\jj\Documents\esp8266flasher\FL­ASH_DOWNLOAD_TOOLS_v2.4_150924\FLASH_DOW­NLOAD_TOOLS_v2.4_150924\tool_config.txt"­

    To configure the download.
    For the Thing AT download
    "C:\Users\jj\Documents\esp8266flasher\FL­ASH_DOWNLOAD_TOOLS_v2.4_150924\FLASH_DOW­NLOAD_TOOLS_v2.4_150924\tool_configThing­dev AT.txt"

    path_data:
choosed:C:\Users\jj\Documents\ESP8266Upd­ate\ESP8266_NONOS_SDK-2.1.0\bin\boot_v1.­2.bin;0x00000;
choosed:C:\Users\jj\Documents\ESP8266Upd­ate\ESP8266_NONOS_SDK-2.1.0\bin\at\512+5­12\user1.1024.new.2.bin;0x1000;
choosed:C:\Users\jj\Documents\ESP8266Upd­ate\ESP8266_NONOS_SDK-2.1.0\bin\esp_init­_data_default.bin;0xfc000;
choosed:C:\Users\jj\Documents\ESP8266Upd­ate\ESP8266_NONOS_SDK-2.1.0\bin\blank.bi­n;0x7e000;
choosed:C:\Users\jj\Documents\ESP8266Upd­ate\ESP8266_NONOS_SDK-2.1.0\bin\blank.bi­n;0xfe000;
empty:;;
empty:;;
default_path:C:\Users\jj\Documents\ESP82­66Update\ESP8266_NONOS_SDK-2.1.0\bin;

    For the Thing Espruino download I used
    "C:\Users\jj\Documents\esp8266flasher\FL­ASH_DOWNLOAD_TOOLS_v2.4_150924\FLASH_DOW­NLOAD_TOOLS_v2.4_150924\tool_configThing­Espruino.txt"

    path_data:
choosed:C:\Users\jj\Documents\ESP8266Upd­ate\espruino_1v95_esp8266\boot_v1.6.bin;­0x00000;
choosed:C:\Users\jj\Documents\ESP8266Upd­ate\espruino_1v95_esp8266\espruino_esp82­66_user1.bin;0x1000;
choosed:C:\Users\jj\Documents\ESP8266Upd­ate\espruino_1v95_esp8266\esp_init_data_­default.bin;0x3FC000;
choosed:C:\Users\jj\Documents\ESP8266Upd­ate\espruino_1v95_esp8266\blank.bin;0x3F­E000;
empty:;;
empty:;;
empty:;;
default_path:C:\Users\jj\Documents\ESP82­66Update\ESP8266_NONOS_SDK-2.1.0\bin;

    2 Attachments

  • • #24

    Trying it with a different ESP8266 chip like the ones for a Pico shim

    Flashed Espruino on to the ESP8266

    Running the previous test progrem gives:

     1v95 Copyright 2017 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:1024/1024, manuf 0xe0 chip 0x4014
>Start connection process
Try Connecting to WiFi  faux
=undefined
null
connected? err= null info= {
  "ip": "192.168.1.13",
  "netmask": "255.255.255.0",
  "gw": "192.168.1.1",
  "mac": "18:fe:34:cb:2c:23"
 }
Wi-Fi Connected
n= 4 t: 0.014 sec
n= 3 t: 0.082 sec
[WS] "Hello to Espruino!"
n= 2 t: 0.149 sec
[WS] "Hello to Espruino!"
n= 1 t: 0.231 sec
[WS] "Hello to Espruino!"
n= 0 t: 0.399 sec
[WS] "Hello to Espruino!"
>

    8192 / 0.231 = 35,463 bytes per second
    35,463 * 8 = 283,705 bits per second

    The tool_config.txt

    path_data:
choosed:C:\Users\jj\Documents\ESP8266Upd­ate\espruino_1v95_esp8266\boot_v1.6.bin;­0x00000;
choosed:C:\Users\jj\Documents\ESP8266Upd­ate\espruino_1v95_esp8266\espruino_esp82­66_user1.bin;0x1000;
choosed:C:\Users\jj\Documents\ESP8266Upd­ate\espruino_1v95_esp8266\esp_init_data_­default.bin;0x3FC000;
choosed:C:\Users\jj\Documents\ESP8266Upd­ate\espruino_1v95_esp8266\blank.bin;0x3F­E000;
empty:;;
empty:;;
empty:;;
default_path:C:\Users\jj\Documents\ESP82­66Update\ESP8266_NONOS_SDK-2.1.0\bin;

    Crystal freq 26.2
    SPI Speed 80M
    SPI Mode QIO
    Flash Size 32 Mbit-C1
    Baudrate 115,200

  • • #25

    So you're a lucky guy, it's working fast for you for 2 different esp8266 based boards..
    I really don't understand what's wrong with my nodemcu-e12. I don't have any other esp8266 to try. I installed mine with the same files, but using esptool. If I install last espressif firmware before, it wont make a difference, it will be erased by esptool, right ?
    I still get a slow download. Honestly, I don't know where to go from here.

    python esptool.py --port COM20 --baud 115200 write_flash --flash_freq 80m --flash_mode qio --flash_size 32m 0x0000 boot_v1.6.bin 0x1000 espruino_esp8266_user1.bin 0x3FC000 esp_init_data_default.bin 0x3FE000 blank.bin

     _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v95 Copyright 2017 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:512/512, manuf 0xef chip 0x4016
>Start connection process
Try Connecting to WiFi  Gardening
=undefined
Test for error
Test for error
Test for error
Test for error
Test for error
null
connected? err= null info= {
  "ip": "192.168.2.5",
  "netmask": "255.255.255.0",
  "gw": "192.168.2.1",
  "mac": "2c:3a:e8:0e:9f:1c"
 }
Wi-Fi Connected
n= 4 t: 0.006 sec
n= 3 t: 0.869 sec
[WS] "Hello to Espruino!"
n= 2 t: 1.525 sec
[WS] "Hello to Espruino!"
n= 1 t: 2.197 sec
[WS] "Hello to Espruino!"
n= 0 t: 2.859 sec
[WS] "Hello to Espruino!"
Page
of 2
/ 2
Next
  • Post a reply
    • Bold
    • Italics
    • Link
    • Image
    • List
    • Quote
    • code
    • Preview
Post a reply
About

How to send a large amount of data faster ?

Posted by Avatar for Polypod @Polypod

Actions

© Espruino, powered by microcosm.

About Terms of Use Privacy Policy Cookie Policy Report a problem