CPU load during HTTP response can get sockets stuck

Posted on
Page
of 2
/ 2
Next
  • Given the following code sample, I can fairly reliably have all the "stylesheets" fail to load when there is CPU load during try to respond to their HTTP requests.

    Dial var junk = generateJunk(45); up and down to experiment, I got it as far as 1024 when using indexOK, and up to about 60 using indexBAD.

    If the indexBAD page loads first time, try refreshing a few times, eventually it should stall out.

    I'm not sure if anything can be done here. Ideally no matter how intensive the CPU task the HTTP request would eventually complete.

    var wifi = require("EspruinoWiFi");
    var http = require('http');
    var indexOK = "<html><head><link rel='stylesheet' href='1'><link rel='stylesheet' href='2'></head><body>OK!</body></html>"­;
    var indexBAD = "<html><head><link rel='stylesheet' href='1'><link rel='stylesheet' href='2'><link rel='stylesheet' href='3'></head><body>NOT OK!</body></html>";
    var indexVERYBAD = "<html><head><link rel='stylesheet' href='1'><link rel='stylesheet' href='2'><link rel='stylesheet' href='3'><link rel='stylesheet' href='4'></head><body>NOT OK!</body></html>";
    
    function generateJunk(length) {
      var text = "";
      var possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklm­nopqrstuvwxyz0123456789";
    
      for(var i = 0; i < length; i++) {
        text += possible.charAt(Math.floor(Math.random()­ * possible.length));
      }
    
      return text;
    }
    
    wifi.startAP('test', {
      password: 'mytestap123',
      authMode: 'wpa2',
    }, function(err) {
      if (err) { throw err; }
    
      http.createServer(function(req, res) {
        switch (req.url) {
          case '/':
            res.writeHead(200, {
              "Content-Type": "text/html",
            });
    //        res.end(indexOK);
            res.end(indexBAD);
            break;
          default:
            // Generate some CPU load
            var junk = generateJunk(45);
            //console.log(process.memory());
            res.writeHead(200, {
              "Content-Type": "text/css",
            });
            res.end(junk);
        }
      }).listen(80);
    });
    
  • Thanks for the example code! I guess if you check E.getErrorFlags() here you'll see an error?

    When I finally complete the native AT command firmware this should work a lot better since much more of the AT command handling will be done in interrupts, but as I guess you figured, currently the input data processing is handled in the main thread. There's a roughly 512 byte buffer - but if you stall the main thread for long enough that the buffer gets full, data gets lost.

    I imagine that if you pre-calculated the junk variable and sent it, it'd work fine?

    There is hardware flow control for the ESP8266 on Espruino WiFi, which I haven't turned on because at the moment there isn't a way to use it in Espruino.

    However, you could still use it manually by doing something like:

    digitalWrite(A15,0);
    var junk = generateJunk(45);
    digitalWrite(A15,1);
    

    (I think - I could have got the polarity of A15 wrong).

    In your initial code, where you were using the filesystem, did E.getErrorFlags() report any errors when the request got lost? I guess it's possible that just revising that with the raised SD card baud rate would solve your problems, as it could have been the SD card reads that were taking too much time?

  • Aha, I keep forgetting the getErrorFlags.

    E.getErrorFlags()
    =[
    "FIFO_FULL"
    ]

    Is there any concept of multithreading in Espruino? I'm not even sure if the CPU supports it.

    Pre-calculated junk works fine, I was just using it as an example of a "busy" Espruino, whether that is CPU or talking to the SD card.

    I tried setting the SD card baud ridiculously high and it still read data, just again I get the stuck HTTP threads.

    I'll try this hardware flow control and see if I can get anywhere.

  • Huh... this is interesting.

    console.log('about to start ap');
    console.log(E.getErrorFlags());
    wifi.startAP('test', {
      password: 'mytestap123',
      authMode: 'wpa2',
    }, function(err) {
      if (err) { throw err; }
    
      console.log('started ap');
      console.log(E.getErrorFlags());
      console.log('about to start http');
      http.createServer(handleHTTPRequest).lis­ten(80);
      console.log('started http');
      console.log(E.getErrorFlags());
    });
    

    about to start ap
    [ ]
    =undefined
    started ap
    [
    "FIFO_FULL"
    ]
    about to start http
    started http
    [ ]

  • If you look in chrome in the developer tools, in the network tab you can see the requests sent.

    You'll see the requests that never get fulfilled...

    As the http server can only handle one request at a time, when they overlap it does not work and buffers get ful and it all goes to custard!

    One work around is to use github and the github.io with your user name. You can out the CSS and js file and images there, and have the espruino just send the main code.

  • It definitely can handle more than one request, it just depends on how busy the Espruino is.

    Unfortunately, reliable HTTP serving is a key component of what I'm working on, I can't just shift the job out to an external HTTP host :)

  • And you can also use quoted literal for your HTML - so you can then span multiple lines:

    var html=`
    Line 1
    Line 2
    `;
    
  • It definitely can handle more than one request, it just depends on how busy the Espruino is.

    Can it if the requests overlap?

  • Yup, heres 1 html file that kicks off 4 concurrent/overlapping requests


    1 Attachment

    • Screen Shot 2017-05-11 at 12.15.07.png
  • Thanks. I'll try the code tomorrow.

  • Oh Gordon you genius!

    Flow control works, you just have to reconfigure the UART to enable it.

    AT+UART=<baudrate>,<databits>,<stopbits>­,<parity>,<flow control>

    : flow control
    ‣ 0: flow control is not enabled
    ‣ 1: enable RTS
    ‣ 2: enable CTS
    ‣ 3: enable both RTS and CTS

    not sure what RTS is, but CTS works fine.

    Also, looks like we now have the baud range that ESP8266 supports:

    The range of baud rates supported: 110~115200*40.

    From my reading I think the *40 has something to do with the clock speed of the serial line?

    Because 115200*40 is 4608000, and that is the baud rate I set that got me into my very nasty mess of needing to reflash the ESP8266. I'm guessing the Espruino just can't go that high.

    var wifi = require("EspruinoWiFi");
    var http = require('http');
    var indexOK = "<html><head><link rel='stylesheet' href='1'><link rel='stylesheet' href='2'></head><body>OK!</body></html>"­;
    var indexBAD = "<html><head><link rel='stylesheet' href='1'><link rel='stylesheet' href='2'><link rel='stylesheet' href='3'></head><body>NOT OK!</body></html>";
    var indexVERYBAD = "<html><head><link rel='stylesheet' href='1'><link rel='stylesheet' href='2'><link rel='stylesheet' href='3'><link rel='stylesheet' href='4'></head><body>NOT OK!</body></html>";
    
    function generateJunk(length) {
      var text = "";
      var possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklm­nopqrstuvwxyz0123456789";
    
      for(var i = 0; i < length; i++) {
        text += possible.charAt(Math.floor(Math.random()­ * possible.length));
      }
    
      return text;
    }
    
    function handleHTTPRequest(req, res) {
      //Flow control close
      digitalWrite(A15,1);
      switch (req.url) {
          case '/':
            res.writeHead(200, {
              "Content-Type": "text/html",
            });
    //        res.end(indexOK);
            res.end(indexVERYBAD);
            break;
          default:
          // Generate some CPU load
          
            var junk = generateJunk(1024);
            //console.log(process.memory());
            res.writeHead(200, {
              "Content-Type": "text/css",
            });
            res.end(junk);
        }
      //flow control open
      digitalWrite(A15,0);
    }
    
    
    wifi.startAP('test', {
      password: 'mytestap123',
      authMode: 'wpa2',
    }, function(err) {
      if (err) { throw err; }
    
      var at = wifi.at;
    
      at.cmd('AT+UART_CUR=115200,8,1,0,2\r\n',­ 10000, function (data) {
        if (data === 'OK') {
          // Flow control open
          digitalWrite(A15,0);
          // Reconfigure serial 
          Serial2.setup(115200, { rx: A3, tx : A2 });
    
          //Flush anything in at's buffer
          at.cmd('AT\r\n', 1000, function(data) {
            console.log(data);
            console.log('http server started');
            http.createServer(handleHTTPRequest).lis­ten(80);
          });
        }
      });
    });
    
  • Great! I'll add an issue for that on GitHub - adding proper IRQ based flow control is probably not that painful since there's already support for software flow control (which I don't think ESP8266 has). It might even mean you should shift to a significantly higher baud rate without problems.

    And yes, Espruino can definitely handle concurrent HTTP connections - it should manage about 4 with the ESP8266 in AT command mode.

    There's no multitasking as such - very urgent tasks get done in IRQs (like getting bytes from serial and dumping them into a FIFO) but the JS just runs in sequence - why it really helps to have a series of small functions that execute quickly, since you're effectively then doing multithreading yourself.

  • +Gordon I guess there is no way to have flow control closed while I'm trying to pipe a long response back to the client, because ofc the data needs to flow back... would be nice if there was just a way to pause incoming connections!

    Stuck again! Will your native implementation of the networking help me out of this situation?

  • No, it's a bit tricky as you can't inspect the current fifo usage from JS - ideally you'd just pause it for a bit when the fifo started getting full.

    And yes, the native implementation should - but realistically the addition of the hardware flow control is what you really need.

  • Also, any idea why I'm getting FIFO full immediately after starting an AP as per my post: http://forum.espruino.com/comments/13629­011/

  • any idea why I'm getting FIFO full immediately after starting an AP as per my post:

    I noticed that too just now - I think it's the big splurge of data the ESP8266 sends out right after it's reset. Most likely it's because of the way data is stored in the UART buffer. UART data is normally packed densely, but if there's a framing/parity error it is also stored in the FIFO, but it takes the space of 4 (sometimes 8) characters - so it doesn't take much of the ESP8266's 76800 baud boot transmission to fill the FIFO up :(

    There should now be a WiFi build in http://www.espruino.com/binaries/travis/­5fd503c0b6ae7a24f044e23fa3f7a8690ce7fed7­

    If you then try that and then this slight modification of your code (note I'm not using an AP in my case):

    function onInit() {
      wifi.connect(WIFI_NAME, { password : WIFI_KEY }, function(err) {
        if (err) {
          console.log("Connection error: "+err);
          return;
        }
        console.log("Connected!");
    
        wifi.at.cmd('AT+UART_CUR=115200,8,1,0,2\­r\n', 10000, function (data) {
          console.log(data);
          if (data === 'OK') {
            Serial2.setup(115200, { rx: A3, tx : A2, cts: A15 });
    
            wifi.getIP(function(err,ip) {
              console.log("IP address is http://"+ip.ip);
              createServer();
            });
          }
        });
      });
    }
    

    It'll enable proper hardware flow control for the Espruino WiFi and it should start working. It now reliably loads all pages for me even with your indexVERYBAD.

  • Glorious. I had just finished a very hacky workaround, but this works better.

    Judging by the heat of the CPU I'm going to need a heatsink soon, LOL.

    Any thoughts on the baud rates?

    The range of baud rates supported: 110~115200*40.

    What is the clock frequency of the UART on Espruino WiFi? I'm guessing it can't be the full CPU clock speed else 4608000 baud would be working which it isn't.

    Nevermind, some trial and error gives us... 3000000 as the absolute maximum baud. 3000001 breaks things.

  • So that, combined with what some might call an 'irresponsible' baud rate of 10000000 on the SD card, gets my 6kb worth of assets loaded in 1.32 seconds.

    Not bad!

  • Interesting! I didn't know what the max baud rate is - I imagine there will be diminishing returns pretty quickly as the rate rises past 1M baud though.

    The heat'll all be coming from the ESP8266 - by comparison the STM32 doesn't draw much at all.

    6k in 1.32 seconds on the low side, but I guess that's actually with multiple HTTP connections? I guess the real win will come for bigger files, so you might get a better experience by inlining your CSS.

    If you're a total sucker for punishment there's also http://www.espruino.com/Reference#l_E_se­tClock

    So you can overclock the chip itself. You might just be able to run the code for 160Mhz (default is 100), but it is a different chip so I'm not sure. Apart from pointing you at the STM32 chip's datasheet on the WiFi board page there's not much I can do to help you there though - that's getting very extreme!

  • An interesting effect I've just noticed. If I console.log anything, and am not connected to the serial console to pick it up, it messes up communication with the ESP8266.

    Yes thats over multiple connections. To be honest it spends most of its time waiting while in flow control. Maybe your native network module will help requests run more concurrently.


    1 Attachment

    • Screen Shot 2017-05-11 at 16.30.29.png
  • When you unplug, the console will swap over to Serial1 (you can stop it with USB.setConsole(1)) and by default Serial1 is at 9600 baud - so you're probably filling your output buffer with the debug info, which is then slowing everything else down a huge amount.

    I'm surprised it breaks things, but I guess at the speed you're running Serial2 at, it might be that also servicing the Serial1 interrupts causes it to lose the occasional character from Serial2.

    Such is the fun of embedded software - usually I try and run everything at sensible enough speeds that nobody has to worry about it :)

  • Ruh-roh. Do not anger the SDcard gods.

    Uncaught Error: Unable to read file : DISK_ERR
    Uncaught Error: Unable to read file : DISK_ERR
    Uncaught Error: Unable to read file : DISK_ERR
    Uncaught Error: Unable to read file : DISK_ERR
    Uncaught Error: Unable to read file : DISK_ERR
    Uncaught Error: Unable to read file : DISK_ERR
    Uncaught Error: Unable to read file : DISK_ERR
    Uncaught Error: Unable to read file : DISK_ERR
    Uncaught Error: Unable to read file : DISK_ERR
    Uncaught Error: Unable to read file : DISK_ERR
    Uncaught Error: Unable to read file : DISK_ERR
    Uncaught Error: Unable to read file : DISK_ERR
    Uncaught Error: Unable to read file : DISK_ERR
    Uncaught Error: Unable to read file : DISK_ERR
    Uncaught Error: Unable to read file : DISK_ERR

    time for me to go home!

  • And after leaving it for a night the SD card works fine again. -.-

    However, I'm now intermittently getting

    Uncaught No 'ready' after AT+RST

    this is before I set a custom baud rate, so I'm not entirely sure what is happening here.

  • +Gordon I think this might be related to the full FIFO buffer. Especially with the v0.50 AT firmware, I think the init string is longer than with v0.40.

    If I wait for my initial wifi.startAP to fail, and then send an AT command directly, the buffer seems to be full of junk. If I send the AT command again, everything clears up:

    Uncaught No 'ready' after AT+RST
    wifi.at.cmd('AT\r\n', 1000, console.log);
    =undefined
    rllrcân lì b|ìpìl bónolnob bpblslpónà l c oââo b|~ònîllon AT
    wifi.at.cmd('AT\r\n', 1000, console.log);
    =undefined
    AT

  • I guess you'd have to try with at.debug() and see what happens. 0.40 sends the text ready right after boot, which is detected and helps to ignore all the random characters. It'd be annoying if 0.50 doesn't do that - but I guess it could be due to the overflow. I'd make sure you're not executing anything that takes a lot of time right after boot though, as that won't help at all.

  • Post a reply
    • Bold
    • Italics
    • Link
    • Image
    • List
    • Quote
    • code
    • Preview
About

CPU load during HTTP response can get sockets stuck

Posted by Avatar for dave_irvine @dave_irvine

Actions