Avatar for Julian1

Julian1

Member since Mar 2021 • Last active Apr 2021
  • 2 conversations
  • 12 comments

Hello

Most recent activity

  • in ESP8266
    Avatar for Julian1

    Thank you, @hungryforcodes I just tried it out and now my ram is on a page request at 1156 free blocks that's an improvement of 300 blocks.

  • in ESP8266
    Avatar for Julian1

    Thank you for your help @allObjects and @Robin.
    I got it now working, and it's a good solution (for me).
    The idea is from Best way to stream base64 encoded string to a web client and I used the drain event.
    I saved the head and the body in the Storage module and send them chunk for chunk.

    function sendMainPage(res) {
        res.writeHead(200, {'Content-Type': 'text/html'});
        res.write('<html lang="en">');
        //chunk size
        let bytesperChunk = 2048;
        //cssN stores how many chunks i need to send
        let cssN = Math.ceil(storage.read('main.css').lengt­h / bytesperChunk );
        //cssI stores how many chunks are already send
        let cssI = 0;
        let htmlN = Math.ceil(storage.read('main.html').leng­th / bytesperChunk );
        let htmlI = 0;
        res.on('drain', function(){
            if (cssN == cssI) {
                if (htmlN == htmlI) {
                    //Here is my client js code with all the values and the end of the html file 
                    res.end(``);
                    print('end');
                }else {
                    res.write(storage.read('main.html', htmlI * bytesperChunk , bytesperChunk ));
                    print(process.memory().free);
                    htmlI++;
                }
            }else {
                res.write(storage.read('main.css', cssI * bytesperChunk , bytesperChunk ));
                cssI++;
                print(cssI);
            }
        });
    }
    

    I don't know if it's the best solution, but it works for me and when I request the page the loading time is only 2-3 seconds.
    Not only that, but I still have 850 free blocks of ram, because now I don't need to load the long HTML string before I send it.

  • in ESP8266
    Avatar for Julian1

    I changed a bit of my code, and now I am starting the function with 900 free blocks.
    I looked how much memory the function is using with E.getSizeOf() and got this:

    >E.getSizeOf(sendMainPage, 1)
    =[
      {
        name: "\xFFa",
        size: 1 },
      {
        name: "\xFFcod",
        size: 323 },
      {
        name: "\xFFlin",
        size: 1 }
     ]
    >E.getSizeOf(sendMainPage)
    =326
    //326 Blocks * 16 =  5216 bytes
    > 
    

    So the function is using 20% from the total blocks (1600) or 36.2% from all available blocks (900) of the ram without the CSS.
    But that's only the size when I request the page its using 430 blocks:

    //start of the function sendMainPage
    { "free": 895, "usage": 705, "total": 1600, "history": 85,
      "gc": 0, "gctime": 1.827, "blocksize": 16 }
    //after res.end();
    467
    >900-470
    =430 //blocks
    

    That's 47.7% of the free ram without CSS.
    I think I got something that could help (How to send a large amount of data faster ?). But I don't know how that should work with the long HTML string. It would be nice if you could help me with this.

  • in ESP8266
    Avatar for Julian1

    Hey @Robin

    Under the assumption the entire code segment is the 515 'usage' value,

    What do you mean with this?

    Here is my code from my function that sends the main page.
    When I remove the CSS from the function its working fine, but then the page is only ~ 4 KB big, and I want the page to be with the CSS.
    I split the string in several pieces, so I can see how much memory is left in between.

    function sendMainPage(res) {
        print(process.memory());
        let hostname = wifi.getHostname();
        let ip = wifi.getIP();
        res.writeHead(200, {'Content-Type': 'text/html'});
        res.write('<html lang="en"><head><meta charset="UTF-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width, initial-scale=1.0"><title>ESP8266 V0.1</title></head>');
        print(process.memory());
        res.write(`<body><div class="side-bar"><div><span id="hostname" class="font-20">${hostname}</span><br><s­pan class="font-18"> ESP8266 V0.1 </span></div><div class="verzeichniss"><h3>Einstellungen</­h3><ul><li onclick="activateAllgemein()">Allgemein<­/li><li onclick="activateWlan()">WLAN</li><li onclick="activateSensoren()">Sensoren</l­i><li onclick="activateNtp()">NTP-Server</li><­/ul></div><div class="main-buttons"><button type="submit" form="mainContent">Save</button></div><d­iv class="main-buttons"><a href="./reboot"><button name="Reboot">Neustarten</button></a></d­iv></div>`);
        print(process.memory());
        res.write(`<form action="/" method="post" class="main-content" id="mainContent"><div id="allgemein"><div class="header">Allgemein</div><div class="content"><table><tbody><tr><td>Ho­stname:</td><td><input type="text" name="hostname" id="hostname" value="${hostname}"></td><td>Currenttime­:</td><td>25.03.2021 17:07:54</td></tr><tr><td>Wlan:</td><td>­${wifi.getDetails().ssid}</td><td>Uptime­:</td><td>25.03.2021 17:07:44</td></tr><tr><td>Mac:</td><td>$­{ip.mac}</td><td>Ip:</td><td>${ip.ip}</t­d></tr></tbody></table></div></div>`);
        print(process.memory());
        res.write(`<div id="wlan" style="display: none;"><div class="header">WLAN</div><div class="content normal"><div><div><div><span>Wlan SSID:</span><input type="text" name="wlanSSID" id="wlanSSID" placeholder="Name des Wlans." value="${config.wifi[0].ssid}"></div><di­v><span>Wlan Passwort:</span><input type="text" name="wlanPW" id="wlanPW" placeholder="Passwort des Wlans." value="${config.wifi[0].pw}"></div></div­><div><div><span>Statische IP:</span><input type="text" name="staticIP" id="staticIP" placeholder="Leer lassen für DHCP."></div><div><span>Gateway-IP:</spa­n><input type="text" name="gatewayIP" id="gatewayIP" placeholder="Nur wenn man eine Statische Ip nimmt."></div></div><div><div><span>Netz­werkmaske:</span><input type="text" name="networkMask" id="networkMask" placeholder="Normalerweise 255.255.255.0."></div><div><span>DNS IP:</span><input type="text" name="dnsIP" id="dnsIP" placeholder="Nur wenn man eine Statische Ip nimmt."></div></div></div></div></div>`)­;
        print(process.memory());
        res.write(`<div id="sensoren" style="display: none;"><div class="header">Sensoren</div><div class="content normal"><div><span>Temparatur Sensor DS18B20:</span><div><div><span>Pin:</spa­n><input type="text" name="pin" id="pin" placeholder="Welchen Pin (1, 2, 3, 5...)."></div><div><span>Sensor:</span><­select name="sensorTyp" id="sensorTyp"><option value="0">DS18B20</option></select></div­></div><div><div><span>Temparatur unterschied:</span><input type="text" name="eichung" id="eichung" placeholder="Um die abweichung anzugleichen."></div><div><span>Sendeint­ervall:</span><input type="text" name="sendIntervall" id="sendIntervall" placeholder="Der Sende Intervall in Sekunden."></div></div><div><div><span>N­ode.js Server IP:</span><input type="text" name="serverIP" id="serverIP" placeholder="Ip des Node.js Servers."></div><div><span>Node.js Server Pfad:</span><input type="text" name="serverPfad" id="serverPfad" placeholder="Der Pfad an dem es Senden soll."></div></div><div><div><span>Gerät­e ID:</span><input type="text" name="clientID" id="clientID" placeholder="Die ID die im Server vermerkt sein muss."></div><div><span>Gerät Passwort:</span><input type="text" name="clientPW" id="clientPW" placeholder="Das Passwort zur ID."></div></div></div></div></div>`);
        print(process.memory());
        res.write(`<div id="ntp" style="display: none;"><div class="header">NTP-Server</div><div class="content normal"><div><div><div><span>NTP-Server:­</span><input type="text" name="ntpServer" id="ntpServer" placeholder="IP des NTP-Servers." value="${config.ntp.server || ""}"></div><div><span>Zeitzone:</span><i­nput type="text" name="timeZone" id="timeZone" placeholder="Zeitzone des NTP-Server." value="${config.ntp.timeZone || ""}"></div><div><span>Winterzeit/Sommerz­eit:</span><input class="apple-switch" name="summerTime" id="summerTime" value="true" type="checkbox" ${config.ntp.summerTime ? 'checked=""' : ''}></div></div></div></div></div></form­>`);
        print(process.memory());
        res.write(`<script src="https://ajax.googleapis.com/ajax/li­bs/jquery/3.2.1/jquery.min.js"></script>­<script>function dilsplayNoneForMainContainer(){let allDivs=$('#mainContent > div').toArray(); for (let i=0; i < allDivs.length; i++){allDivs[i].setAttribute("Style", "display: none;");};}; function activateAllgemein(){dilsplayNoneForMainC­ontainer(); allgemein.setAttribute("Style", "display: inline-block;");}; function activateWlan(){dilsplayNoneForMainContai­ner(); wlan.setAttribute("Style", "display: inline-block;");}; function activateSensoren(){dilsplayNoneForMainCo­ntainer(); sensoren.setAttribute("Style", "display: inline-block;");}; function activateNtp(){dilsplayNoneForMainContain­er(); ntp.setAttribute("Style", "display: inline-block;");}; </script></body></html>`);
        res.end();
        print(process.memory());
    }
    

    Here is the Console:

    >print(process.memory().free);
    928
    =undefined
    //request of page.
    { "free": 730, "usage": 870, "total": 1600, "history": 92,
      "gc": 0, "gctime": 1.795, "blocksize": 16 }
    686
    639
    595
    516
    412
    365
    299
    //done
    > 
    

    Code with CSS:

    function sendMainPage(res) {
        print(process.memory());
        let hostname = wifi.getHostname();
        let ip = wifi.getIP();
        res.writeHead(200, {'Content-Type': 'text/html'});
        res.write('<html lang="en"><head><meta charset="UTF-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width, initial-scale=1.0"><title>ESP8266 V0.1</title><style>@import url("https://fonts.googleapis.com/css2?f­amily=Poppins:wght@200;300;400;500;600;7­00;800&display=swap");*{padding: 0; margin: 0; box-sizing: border-box;}:root{--border-1px: solid 1px black; --blue: [#4481EB](http://forum.espruino.com/sear­ch/?q=%234481EB); --breite-left-panel: 240px; --breite-main-content: 75%;}body{font-family: "Poppins", sans-serif; width: 100%; height: max-content;}.font-18{font-size: 18px; font-weight: 500;}.font-20{font-size: 24px; font-weight: 500;}.side-bar{position: fixed; top: 0; left: 0; border-left: none; border-top: none; border-bottom: none; border-right: var(--border-1px); height: 100%; width: var(--breite-left-panel);}.side-bar > div{border-left: none; border-right: none; border-top: none; border-bottom: var(--border-1px); height: max-content; width: 100%;}.side-bar > div:hover{background-color: var(--blue);}.side-bar > div:first-child{padding: 10px;}.verzeichniss > h3{font-size: 20px; font-weight: 500; padding: 10px 0 10px 10px;}.verzeichniss > ul{margin-left: 40px; list-style-type: none;}.verzeichniss > ul > li{border-top: var(--border-1px); border-left: var(--border-1px); background-color: white; padding: 5px 5px 5px 10px;}.verzeichniss > ul > li:hover{background-color: var(--blue); cursor: pointer;}.main-buttons > button,.main-buttons > a > button{font-family: "Poppins", sans-serif; background: none; outline: none; height: 60px; width: 100%; font-size: 20px; text-align: left; padding: 5px; border: none; text-align: center; cursor: pointer;}.side-bar > div:last-child{padding: 10px; font-size: 16px;}.main-content{height: max-content; width: var(--breite-main-content); margin: 84px 0 0 calc((100% - var(--breite-main-content) + var(--breite-left-panel)) / 2); border: var(--border-1px);}.main-content > div{height: max-content; width: 100%;}.header{height: max-content; width: 100%; font-size: 35px; font-weight: 600; text-align: center; padding: 20px;}.content{padding: 30px;}.content > table{width: auto; margin: auto; border-collapse: collapse; letter-spacing: 0px; word-spacing: 0px; color: #000000; font-weight: 400; font-style: normal; font-variant: normal; text-transform: none; font-size: 20px;}.content > table td{padding: 5px 15px; border: 1px solid #000; text-align: left; height: 100%; width: 25%;}.content > table > tbody > tr > td > input{width: 100%; font-family: "Poppins", sans-serif; background: none; outline: none; padding: 2.5px 2.5px; font-weight: 400; font-size: 16px; color: #333; border: solid 2px black;}.content > table td:first-child,.content > table td:nth-child(3){border-right: none;}.content > table td:nth-child(2),.content > table td:last-child{border-left: none;}.normal > div{height: max-content; width: 100%; display: grid; grid-template-rows: 1fr 1fr;}.normal:last-child > div{height: max-content; width: 100%; display: inline-block;}.normal > div > span{font-size: 26px; font-weight: 500; margin-left: 10px;}.normal > div > div{width: 100%; display: grid; grid-template-columns: 1fr 1fr;}.normal > div > div > div{display: grid; grid-template-columns: 0.7fr 1.3fr; padding: 10px;}.normal > div > div > div > *{line-height: 25px;}.normal > div > div > div > span{font-size: 18px; font-weight: 400; padding: 8px 0;}.normal > div > div > div > input,.normal > div > div > div > select{width: 80%; font-family: "Poppins", sans-serif; background: none; outline: none; padding: 5px 5px; margin-right: 7.5px; font-weight: 400; font-size: 16px; color: #333; border: solid 2px black;}.apple-switch{position: relative; -webkit-appearance: none; outline: none; width: 60px !important; height: 35px; background-color: [#fff](http://forum.espruino.com/search/­?q=%23fff); border: 1px solid [#D9DADC](http://forum.espruino.com/sear­ch/?q=%23D9DADC); border-radius: 50px; box-shadow: inset -20px 0 0 0 [#fff](http://forum.espruino.com/search/­?q=%23fff);}.apple-switch:after{content:­ ""; position: absolute; top: 0; left: 0; background: transparent; width: 31px; height: 31px; border-radius: 50%; box-shadow: 2px 4px 6px rgba(0,0,0,0.2);}.apple-switch:checked{b­ox-shadow: inset 20px 0 0 0 [#4ed164](http://forum.espruino.com/sear­ch/?q=%234ed164); border-color: [#4ed164](http://forum.espruino.com/sear­ch/?q=%234ed164);}.apple-switch:checked:­after{left: 20px; box-shadow: -2px 4px 3px rgba(0,0,0,0.05);}</style></head>');
        print(process.memory().free);
        res.write(`<body><div class="side-bar"><div><span id="hostname" class="font-20">${hostname}</span><br><s­pan class="font-18"> ESP8266 V0.1 </span></div><div class="verzeichniss"><h3>Einstellungen</­h3><ul><li onclick="activateAllgemein()">Allgemein<­/li><li onclick="activateWlan()">WLAN</li><li onclick="activateSensoren()">Sensoren</l­i><li onclick="activateNtp()">NTP-Server</li><­/ul></div><div class="main-buttons"><button type="submit" form="mainContent">Save</button></div><d­iv class="main-buttons"><a href="./reboot"><button name="Reboot">Neustarten</button></a></d­iv></div>`);
        print(process.memory().free);
        res.write(`<form action="/" method="post" class="main-content" id="mainContent"><div id="allgemein"><div class="header">Allgemein</div><div class="content"><table><tbody><tr><td>Ho­stname:</td><td><input type="text" name="hostname" id="hostname" value="${hostname}"></td><td>Currenttime­:</td><td>25.03.2021 17:07:54</td></tr><tr><td>Wlan:</td><td>­${wifi.getDetails().ssid}</td><td>Uptime­:</td><td>25.03.2021 17:07:44</td></tr><tr><td>Mac:</td><td>$­{ip.mac}</td><td>Ip:</td><td>${ip.ip}</t­d></tr></tbody></table></div></div>`);
        print(process.memory().free);
        res.write(`<div id="wlan" style="display: none;"><div class="header">WLAN</div><div class="content normal"><div><div><div><span>Wlan SSID:</span><input type="text" name="wlanSSID" id="wlanSSID" placeholder="Name des Wlans." value="${config.wifi[0].ssid}"></div><di­v><span>Wlan Passwort:</span><input type="text" name="wlanPW" id="wlanPW" placeholder="Passwort des Wlans." value="${config.wifi[0].pw}"></div></div­><div><div><span>Statische IP:</span><input type="text" name="staticIP" id="staticIP" placeholder="Leer lassen für DHCP."></div><div><span>Gateway-IP:</spa­n><input type="text" name="gatewayIP" id="gatewayIP" placeholder="Nur wenn man eine Statische Ip nimmt."></div></div><div><div><span>Netz­werkmaske:</span><input type="text" name="networkMask" id="networkMask" placeholder="Normalerweise 255.255.255.0."></div><div><span>DNS IP:</span><input type="text" name="dnsIP" id="dnsIP" placeholder="Nur wenn man eine Statische Ip nimmt."></div></div></div></div></div>`)­;
        print(process.memory().free);
        res.write(`<div id="sensoren" style="display: none;"><div class="header">Sensoren</div><div class="content normal"><div><span>Temparatur Sensor DS18B20:</span><div><div><span>Pin:</spa­n><input type="text" name="pin" id="pin" placeholder="Welchen Pin (1, 2, 3, 5...)."></div><div><span>Sensor:</span><­select name="sensorTyp" id="sensorTyp"><option value="0">DS18B20</option></select></div­></div><div><div><span>Temparatur unterschied:</span><input type="text" name="eichung" id="eichung" placeholder="Um die abweichung anzugleichen."></div><div><span>Sendeint­ervall:</span><input type="text" name="sendIntervall" id="sendIntervall" placeholder="Der Sende Intervall in Sekunden."></div></div><div><div><span>N­ode.js Server IP:</span><input type="text" name="serverIP" id="serverIP" placeholder="Ip des Node.js Servers."></div><div><span>Node.js Server Pfad:</span><input type="text" name="serverPfad" id="serverPfad" placeholder="Der Pfad an dem es Senden soll."></div></div><div><div><span>Gerät­e ID:</span><input type="text" name="clientID" id="clientID" placeholder="Die ID die im Server vermerkt sein muss."></div><div><span>Gerät Passwort:</span><input type="text" name="clientPW" id="clientPW" placeholder="Das Passwort zur ID."></div></div></div></div></div>`);
        print(process.memory().free);
        res.write(`<div id="ntp" style="display: none;"><div class="header">NTP-Server</div><div class="content normal"><div><div><div><span>NTP-Server:­</span><input type="text" name="ntpServer" id="ntpServer" placeholder="IP des NTP-Servers." value="${config.ntp.server || ""}"></div><div><span>Zeitzone:</span><i­nput type="text" name="timeZone" id="timeZone" placeholder="Zeitzone des NTP-Server." value="${config.ntp.timeZone || ""}"></div><div><span>Winterzeit/Sommerz­eit:</span><input class="apple-switch" name="summerTime" id="summerTime" value="true" type="checkbox" ${config.ntp.summerTime ? 'checked=""' : ''}></div></div></div></div></div></form­>`);
        print(process.memory().free);
        res.write(`<script src="https://ajax.googleapis.com/ajax/li­bs/jquery/3.2.1/jquery.min.js"></script>­<script>function dilsplayNoneForMainContainer(){let allDivs=$('#mainContent > div').toArray(); for (let i=0; i < allDivs.length; i++){allDivs[i].setAttribute("Style", "display: none;");};}; function activateAllgemein(){dilsplayNoneForMainC­ontainer(); allgemein.setAttribute("Style", "display: inline-block;");}; function activateWlan(){dilsplayNoneForMainContai­ner(); wlan.setAttribute("Style", "display: inline-block;");}; function activateSensoren(){dilsplayNoneForMainCo­ntainer(); sensoren.setAttribute("Style", "display: inline-block;");}; function activateNtp(){dilsplayNoneForMainContain­er(); ntp.setAttribute("Style", "display: inline-block;");}; </script></body></html>`);
        res.end();
        print(process.memory().free);
    }
    

    This happens:

    >Uncaught SyntaxError: Got EOF expected '}'
     at line 51 col 114
    ...in.js"></script><script>fu
                                 ^
    Execution Interrupted
    New interpreter error: LOW_MEMORY,MEMORY
    

    I need a way to send the HTML, CSS and JS without using up all the RAM, but I still need to be able to insert data into the HTML.

  • in General
    Avatar for Julian1

    For people who are interested here is an example:

    First you look if the client is already logged in.
    If not you send a Header with Status 401 and WWW-Authenticate.

    // The header you check is "Basic Username:Password".
    if(req.headers.Authorization != "Basic YWRtaW46YWRtaW4=") {
    //Between the "" you can write a message that should than be displayed.
            res.writeHead(401, { 'WWW-Authenticate': 'Basic realm=""' });
            res.end();
    } else {
            //If logged in
    }
    

    Edit:

    Here is a little function.

    function basicAuth(req, res, username, password, message, callback) {
        var credentials = btoa(username + ':' + password);
        if(req.headers.Authorization != "Basic " + credentials) {
            res.writeHead(401, {
               'WWW-Authenticate': 'Basic realm="' + message + '"'
            });
            res.write("401 Unauthorized");
            res.end();
        } else {
            callback();
        }
    }
    
  • in ESP8266
    Avatar for Julian1

    Thank you for your quick response. I just realized I am in the wrong forum I should actually be in the "other boards". I hope that's not a big problem.

  • in ESP8266
    Avatar for Julian1

    I am new to this and I have a simple http server that responds with an 8KByte website but still says that memory is low and stops after halfway through. What's the best way to do it?

Actions