ESP32 memory leaks WIFI and MQTT

Posted on
  • I'm pretty new to Javascript so I'm hoping this is something that is easy to fix in my code. I've tried setting up my ESP32 to connect to WIFI and MQTT and no matter how I write it, I am getting memory leaks. Luckily, I do not need Bluetooth and I saw on here how to disable that (which practically doubled my available memory).

    Below is my current version that will connect to my WIFI and publish MQTT. It will reconnect on a lost WIFI signal or lost MQTT connection. However, every 30 mins or so my free memory drops by 50 or so. I've tried writing this various different ways such as using setInterval vs setTimeout functions. I don't think I am mixing up the scope of any of my global variables. I even once tried putting everything all in one function and periodically resetting it to null which just seemed to make the problem worse. Is there a way to do this that will not run through all the memory?

    //WIFI config
    var wifi_options = 
       {
         ssid: 'xxxxxx',
         password: 'xxxxxx',
       };
    
    //MQTT config
    var mqtt_freq = 1000; //how often to send MQTT(ms)
    var mqtt_options = 
      {
    //    keep_alive: 0,
        server: "192.168.1.100",
        port: 1883,
      };
    
    var led_speed = 500;
    var wifi_stat;
    var wifi_recon_count=0;
    var wifi_pre_mem;
    var wifi_post_mem;
    
    var Wifi = require('Wifi');
    
    //get wifi status every second
    function iw1() {
      wifi_stat = Wifi.getDetails().status;
      //set led blink rate
      if(wifi_stat !== "connected") {led_speed = 100;}
      else {led_speed=500;}
      setTimeout(iw1,1000);
    }
    iw1();
    
    //reconnect to WIFI (only try once every 30 seconds)  
    function iw2(){
      if(wifi_stat !== "connected") {
        wifi_recon_count++;
        wifi_pre_mem = process.memory().free;
        print("WIFI disconnected: Reconecting...");
        Wifi.connect(wifi_options.ssid, {password: wifi_options.password});
        wifi_post_mem = process.memory().free;
      }
      setTimeout(iw2,1000*30);
    }
    iw2();
    
    
    //BLINK LED
    var  D2_setting = false;
    function iLED () {
      D2_setting = !D2_setting;
      digitalWrite(D2, D2_setting);
      
      setTimeout(iLED, led_speed);
    }
    iLED();
    
    
    
    //mqtt globals
    var mqtt_r=require("MQTT");
    mqtt = mqtt_r.create(mqtt_options.server, {
      port: mqtt_options.port,//8883
      });
    var mqtt_conn_count = 0;
    var mqtt_stat = false; 
    
    function mqt0(){
      function mqt1 () {
        mqtt_stat = mqtt.connected;
      }
      var imq1 = setInterval(mqt1,1000);
      
      //Reconnect MQTT if disconnected:
      function mqt2() {
        if (mqtt_stat === false){
          mqtt_conn_count++;
          print("MQTT disconnected, reconnecting...");
          mqtt.connect();
        }
      }
      var imq2 = setInterval(mqt2,30*1000); //only try to reconnect every xx seconds
      
      //publish free memory every minute
      function mqt3(){
        if (mqtt_stat === true){
          free_mem = process.memory().free; 
          print("memory publishing: " + free_mem);
          mqtt.publish("process.memory()",free_mem­);}
      }  
       var imq3 = setInterval(mqt3,1000*60);
    
      //loop back to mqt0 and clear intervals
      setTimeout(function(){
        clearInterval(imq1);
        clearInterval(imq2);
        clearInterval(imq3);
        imq1 = null;
        imq2 = null;
        imq3 = null;
        mqt0();
      },60*1000*15); //setloop time to 15 mins
    }
    mqt0();
    
  • Wed 2019.10.09

    Hello @dwinfiel

    Would you mind posting the results of process.env and separatelyprocess.memory() before uploading, then again after the memory leak is observed, so that we may get a better understanding of the scope of this issue. Thanks

  • Hi!

    Just connecting to the AP, and then saving Wifi credentials with Wifi.save() was the best for me.
    Wifi tries to reconnect if disconnected from the AP without any explicit code. Connect and reconnect sometimes completes quickly, sometimes it takes longer. And this can mess if you have timed reconnects.

    Also, in the function

    function iw2(){
      if(wifi_stat !== "connected") {
        wifi_recon_count++;
        wifi_pre_mem = process.memory().free;
        print("WIFI disconnected: Reconecting...");
        Wifi.connect(wifi_options.ssid, {password: wifi_options.password});
        wifi_post_mem = process.memory().free;
      }
      setTimeout(iw2,1000*30);
    }
    

    The status can be for example connecting, if the function runs just after a random disconnect, while the ESP is still trying to connect. Calling Wifi.connect() while connecting caused all kinds of strange behaviour for me.

  • I guess there is a better way: Do not create new timer every 1000ms, but only once. Just to quote one function:

    //get wifi status every second
    function iw1() {
      wifi_stat = Wifi.getDetails().status;
      //set led blink rate
      if(wifi_stat !== "connected") {led_speed = 100;}
      else {led_speed=500;}
      setTimeout(iw1,1000);
    }
    iw1();
    

    And how to do it better:

    //get wifi status every second
    function iw1() {
      wifi_stat = Wifi.getDetails().status;
      //set led blink rate
      if(wifi_stat !== "connected") {led_speed = 100;}
      else {led_speed=500;}
    }
    iw1();
    setInterval(iw1,1000);
    

    While there is a garbage collection, it might be called too late, or not working properly - I didn't look into it. But if you call setTimeout every 1000ms instead of setInterval once there's a really high chance it'll allocate a new block of memory for a new timer every 1000ms.
    The other functions shall be modified accordingly.

  • Thank everyone for their suggestions. I have just gotten back to trying this again (had some OT at work and other things going on).

    I am starting with a fresh install of everything (even my OS because when I tried to update Debian from stretch to buster, I broke it).

    The ESP32 board that I'm using are MELIFE 2 ESP-32S (ESP-WROOM-32 module with 4MB flash, no pSRAM).

    I've done a fresh flash of the firmware and I get the following right after flashing:

    >ESP32.getState();
    ={
      sdkVersion: "v3.1.3-dirty",
      freeHeap: 35712, BLE: true, Wifi: true, minHeap: 33112 }
    
    >process.memory();
    ={ free: 2264, usage: 36, total: 2300, history: 13,
      gc: 0, gctime: 1.876 }
    
    >process.env
    ={
      VERSION: "2v04",
      GIT_COMMIT: "3956264e",
      BOARD: "ESP32",
      FLASH: 0, RAM: 524288,
      SERIAL: "3c71bff1-179c",
      CONSOLE: "Serial1",
      MODULES: "Flash,Storage,hea" ... "r,crypto,neopixel",
      EXPTR: 1073484860 }
    

    I then disabled bluetooth and get much more memory:

    >ESP32.enableBLE(false); 
    >process.memory();
    ={ free: 4071, usage: 29, total: 4100, history: 0,
      gc: 0, gctime: 3.209 }
    

    Then I turned on WIFI using

    var ssid = 'YOUR_SSID';
    var password = 'YOUR_SSID_PASSWORD';
    
    var wifi = require('Wifi');
    wifi.connect(ssid, {password: password}, function() {
        console.log('Connected to Wifi.  IP address is:', wifi.getIP().ip);
        wifi.save(); // Next reboot will auto-connect
    });
    

    My WIFI does not re-connect on it's own (if I say reboot my WIFI router). However, it does connect when I reboot the ESP32.

    I'll try some of the other suggestions and see what I get.

  • I've modified the code to not try to connect if wifi status is "connecting". I've also modified it so that it won't try to reconnect MQTT if WIFI is disconnected. While connected with USB, I've tried starting and stopping my WIFI and/or my MQTT server so that I can see the printout from the ESP32. I'm noticing a few errors I'm hoping someone can clarify what the following errors are:

    E (265203) event: post event to user fail!
    E (268048) event: post event to user fail!
    E (270893) event: post event to user fail!
    E (455363) event: post event to user fail!
    

    I get the following after reseting:

    WARNING: gap set scan error code = 103
    WARNING: set rssi scan not implemeted yet
    

    Also, I've noticed the device go into a state where the Wifi status is "NO_AP_FOUND" and it wouldn't reconnect. Once I disconnected power and re-connected then the WIFI was able to reconnect.

  • Mon 2019.10.14

    Thank you for the above postings @dwinfiel

    Are we still dealing with a memory leak? If so,

    would you please load your code and re-run process.memory(); posting the results of each interval after several iterations. We would be seeking the value of ':free' to observe how much space is available for Espruino to do it's thing while running, and also if there is in fact a perceived leak after several events have been confirmed.    or, . . . maybe run the above snippet inside a setInterval() and using console.log() to directly output to the Left-Hand console side of the WebIDE.



    Errors indicated above #6 are unknown to me.

    'does not re-connect on it's own (if I say reboot my WIFI router)'

    What logic is being used to reboot or dis-connect/re-connect when just the router is rebooted? Has a sufficient time interval been used to allow internal WiFI workings to settle?

  • Post a reply
    • Bold
    • Italics
    • Link
    • Image
    • List
    • Quote
    • code
    • Preview
About

ESP32 memory leaks WIFI and MQTT

Posted by Avatar for dwinfiel @dwinfiel

Actions