Transferring 8KB of Data

Posted on Mon 29th, March 2021

• #1

Julian1

I am new to this and I have a simple http server that responds with an 8KByte website but still says that memory is low and stops after halfway through. What's the best way to do it?
• #2

Jean-Philippe_Rey

If you need fast RAM to store live pages it could be worth checking at FRAM modules such as this one: https://www.adafruit.com/product/1895

FRAM stands for ferroelectric RAM, which is non-volatile, yet fast for read and write cycles.
• #3

Robin in reply to @Julian1

Mon 2021.03.29

'I am new to this'

Hi @Julian1 please elaborate, 'new' to Espruino, Javascript, micro-controllers, programming in general?

'responds with an 8KByte website but still says that memory is low and stops after halfway through'

Based on your above clarification, will assist us in focusing on where to expend energies. My gut feel is this may take a bit of back-n-forth effort before we narrow down where to focus that effort.

While @Jean-Philippe_Rey ( BTW thanks for that link on that new technology. FRAM now on my list of play things to tinker with!! ) pointed out a method to increase physical memory, there may be plenty using the Espruino device you currently have.

I found my notes on stuffing chars into memory, and I was able to get a 30K code file, that was approx 1500 lines of 80 chars, which used 1500-2000 JsVars. So, your 8K is definitely doable.

Run process.env; and process.memory(); in the left hand side of the WebIDE and post the results please, and we'll go from there.

http://www.espruino.com/Reference#l_process_memory

Stuff to read up on while we ponder:

https://www.espruino.com/Internals
16 byte JsVars
• #4

Julian1
Thank you for your quick response. I just realized I am in the wrong forum I should actually be in the "other boards". I hope that's not a big problem.

1 Attachment
• #5

Robin in reply to @Julian1
Tue 2021.03.30

' "other boards". I hope that's not a big problem '

Hi @Julian1 and thank you for posting that detail. While it is encouraged to code on an official board to contribute to the overall Espruino project for full support, there are a few that peruse the forums to assist, but that contribution is limited. The moderator may feel the need to move this to a more appropriate heading.

IMO I felt your inquiry was generic enough that many could learn from the steps required to solve, and could easily be used by those with an Espruino WiFi. Although I'm not entirely sure I have a complete solution just yet, I do recall a thread recently, within the last two to six months, that was along this same topic. Will keep searching over the next few days.

EDIT: 2021.03.31 @allObjects found and posted the link and may be viewed in post #8
```
From process.memory()

free: 1085  usage: 515  total: 1600  history: 27


1600 x 16 = 25,600

1085 x 16 = 17,360

Let's round down and reserve 200 JsVars for Espruino overhead, 
leaving ~800 JsVars free

800 x 16 = 12,800
```
Under the assumption the entire code segment is the 515 'usage' value, then the one 8K fetch, 'should' fit.

Has an attempt to view using process.memory().free; before, during, and immediately after been done to see if actual usage is somewhat close to our estimates?

'still says that memory is low'

What was the specific error seen? Please post. Easy to insert, just click the < / >Code button, just above where typing the thread response is done.

'stops after halfway through'

What amount of memory remains? Could this be a Http transport issue, and does a browser fetch exhibit the same? Is the 8K payload just a data file or an actual Html page?

• #6

Julian1

Under the assumption the entire code segment is the 515 'usage' value,

What do you mean with this?

Here is my code from my function that sends the main page.
When I remove the CSS from the function its working fine, but then the page is only ~ 4 KB big, and I want the page to be with the CSS.
I split the string in several pieces, so I can see how much memory is left in between.

function sendMainPage(res) {
    print(process.memory());
    let hostname = wifi.getHostname();
    let ip = wifi.getIP();
    res.writeHead(200, {'Content-Type': 'text/html'});
    res.write('<html lang="en"><head><meta charset="UTF-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width, initial-scale=1.0"><title>ESP8266 V0.1</title></head>');
    print(process.memory());
    res.write(`<body><div class="side-bar"><div><span id="hostname" class="font-20">${hostname}</span><br><span class="font-18"> ESP8266 V0.1 </span></div><div class="verzeichniss"><h3>Einstellungen</h3><ul><li onclick="activateAllgemein()">Allgemein</li><li onclick="activateWlan()">WLAN</li><li onclick="activateSensoren()">Sensoren</li><li onclick="activateNtp()">NTP-Server</li></ul></div><div class="main-buttons"><button type="submit" form="mainContent">Save</button></div><div class="main-buttons"><a href="./reboot"><button name="Reboot">Neustarten</button></a></div></div>`);
    print(process.memory());
    res.write(`<form action="/" method="post" class="main-content" id="mainContent"><div id="allgemein"><div class="header">Allgemein</div><div class="content"><table><tbody><tr><td>Hostname:</td><td><input type="text" name="hostname" id="hostname" value="${hostname}"></td><td>Currenttime:</td><td>25.03.2021 17:07:54</td></tr><tr><td>Wlan:</td><td>${wifi.getDetails().ssid}</td><td>Uptime:</td><td>25.03.2021 17:07:44</td></tr><tr><td>Mac:</td><td>${ip.mac}</td><td>Ip:</td><td>${ip.ip}</td></tr></tbody></table></div></div>`);
    print(process.memory());
    res.write(`<div id="wlan" style="display: none;"><div class="header">WLAN</div><div class="content normal"><div><div><div><span>Wlan SSID:</span><input type="text" name="wlanSSID" id="wlanSSID" placeholder="Name des Wlans." value="${config.wifi[0].ssid}"></div><div><span>Wlan Passwort:</span><input type="text" name="wlanPW" id="wlanPW" placeholder="Passwort des Wlans." value="${config.wifi[0].pw}"></div></div><div><div><span>Statische IP:</span><input type="text" name="staticIP" id="staticIP" placeholder="Leer lassen für DHCP."></div><div><span>Gateway-IP:</span><input type="text" name="gatewayIP" id="gatewayIP" placeholder="Nur wenn man eine Statische Ip nimmt."></div></div><div><div><span>Netzwerkmaske:</span><input type="text" name="networkMask" id="networkMask" placeholder="Normalerweise 255.255.255.0."></div><div><span>DNS IP:</span><input type="text" name="dnsIP" id="dnsIP" placeholder="Nur wenn man eine Statische Ip nimmt."></div></div></div></div></div>`);
    print(process.memory());
    res.write(`<div id="sensoren" style="display: none;"><div class="header">Sensoren</div><div class="content normal"><div><span>Temparatur Sensor DS18B20:</span><div><div><span>Pin:</span><input type="text" name="pin" id="pin" placeholder="Welchen Pin (1, 2, 3, 5...)."></div><div><span>Sensor:</span><select name="sensorTyp" id="sensorTyp"><option value="0">DS18B20</option></select></div></div><div><div><span>Temparatur unterschied:</span><input type="text" name="eichung" id="eichung" placeholder="Um die abweichung anzugleichen."></div><div><span>Sendeintervall:</span><input type="text" name="sendIntervall" id="sendIntervall" placeholder="Der Sende Intervall in Sekunden."></div></div><div><div><span>Node.js Server IP:</span><input type="text" name="serverIP" id="serverIP" placeholder="Ip des Node.js Servers."></div><div><span>Node.js Server Pfad:</span><input type="text" name="serverPfad" id="serverPfad" placeholder="Der Pfad an dem es Senden soll."></div></div><div><div><span>Geräte ID:</span><input type="text" name="clientID" id="clientID" placeholder="Die ID die im Server vermerkt sein muss."></div><div><span>Gerät Passwort:</span><input type="text" name="clientPW" id="clientPW" placeholder="Das Passwort zur ID."></div></div></div></div></div>`);
    print(process.memory());
    res.write(`<div id="ntp" style="display: none;"><div class="header">NTP-Server</div><div class="content normal"><div><div><div><span>NTP-Server:</span><input type="text" name="ntpServer" id="ntpServer" placeholder="IP des NTP-Servers." value="${config.ntp.server || ""}"></div><div><span>Zeitzone:</span><input type="text" name="timeZone" id="timeZone" placeholder="Zeitzone des NTP-Server." value="${config.ntp.timeZone || ""}"></div><div><span>Winterzeit/Sommerzeit:</span><input class="apple-switch" name="summerTime" id="summerTime" value="true" type="checkbox" ${config.ntp.summerTime ? 'checked=""' : ''}></div></div></div></div></div></form>`);
    print(process.memory());
    res.write(`<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script><script>function dilsplayNoneForMainContainer(){let allDivs=$('#mainContent > div').toArray(); for (let i=0; i < allDivs.length; i++){allDivs[i].setAttribute("Style", "display: none;");};}; function activateAllgemein(){dilsplayNoneForMainContainer(); allgemein.setAttribute("Style", "display: inline-block;");}; function activateWlan(){dilsplayNoneForMainContainer(); wlan.setAttribute("Style", "display: inline-block;");}; function activateSensoren(){dilsplayNoneForMainContainer(); sensoren.setAttribute("Style", "display: inline-block;");}; function activateNtp(){dilsplayNoneForMainContainer(); ntp.setAttribute("Style", "display: inline-block;");}; </script></body></html>`);
    res.end();
    print(process.memory());
}

Here is the Console:

>print(process.memory().free);
928
=undefined
//request of page.
{ "free": 730, "usage": 870, "total": 1600, "history": 92,
  "gc": 0, "gctime": 1.795, "blocksize": 16 }
686
639
595
516
412
365
299
//done
>

Code with CSS:

function sendMainPage(res) {
    print(process.memory());
    let hostname = wifi.getHostname();
    let ip = wifi.getIP();
    res.writeHead(200, {'Content-Type': 'text/html'});
    res.write('<html lang="en"><head><meta charset="UTF-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width, initial-scale=1.0"><title>ESP8266 V0.1</title><style>@import url("https://fonts.googleapis.com/css2?family=Poppins:wght@200;300;400;500;600;700;800&display=swap");*{padding: 0; margin: 0; box-sizing: border-box;}:root{--border-1px: solid 1px black; --blue: [#4481EB](https://forum.espruino.com/search/?q=%234481EB); --breite-left-panel: 240px; --breite-main-content: 75%;}body{font-family: "Poppins", sans-serif; width: 100%; height: max-content;}.font-18{font-size: 18px; font-weight: 500;}.font-20{font-size: 24px; font-weight: 500;}.side-bar{position: fixed; top: 0; left: 0; border-left: none; border-top: none; border-bottom: none; border-right: var(--border-1px); height: 100%; width: var(--breite-left-panel);}.side-bar > div{border-left: none; border-right: none; border-top: none; border-bottom: var(--border-1px); height: max-content; width: 100%;}.side-bar > div:hover{background-color: var(--blue);}.side-bar > div:first-child{padding: 10px;}.verzeichniss > h3{font-size: 20px; font-weight: 500; padding: 10px 0 10px 10px;}.verzeichniss > ul{margin-left: 40px; list-style-type: none;}.verzeichniss > ul > li{border-top: var(--border-1px); border-left: var(--border-1px); background-color: white; padding: 5px 5px 5px 10px;}.verzeichniss > ul > li:hover{background-color: var(--blue); cursor: pointer;}.main-buttons > button,.main-buttons > a > button{font-family: "Poppins", sans-serif; background: none; outline: none; height: 60px; width: 100%; font-size: 20px; text-align: left; padding: 5px; border: none; text-align: center; cursor: pointer;}.side-bar > div:last-child{padding: 10px; font-size: 16px;}.main-content{height: max-content; width: var(--breite-main-content); margin: 84px 0 0 calc((100% - var(--breite-main-content) + var(--breite-left-panel)) / 2); border: var(--border-1px);}.main-content > div{height: max-content; width: 100%;}.header{height: max-content; width: 100%; font-size: 35px; font-weight: 600; text-align: center; padding: 20px;}.content{padding: 30px;}.content > table{width: auto; margin: auto; border-collapse: collapse; letter-spacing: 0px; word-spacing: 0px; color: #000000; font-weight: 400; font-style: normal; font-variant: normal; text-transform: none; font-size: 20px;}.content > table td{padding: 5px 15px; border: 1px solid #000; text-align: left; height: 100%; width: 25%;}.content > table > tbody > tr > td > input{width: 100%; font-family: "Poppins", sans-serif; background: none; outline: none; padding: 2.5px 2.5px; font-weight: 400; font-size: 16px; color: #333; border: solid 2px black;}.content > table td:first-child,.content > table td:nth-child(3){border-right: none;}.content > table td:nth-child(2),.content > table td:last-child{border-left: none;}.normal > div{height: max-content; width: 100%; display: grid; grid-template-rows: 1fr 1fr;}.normal:last-child > div{height: max-content; width: 100%; display: inline-block;}.normal > div > span{font-size: 26px; font-weight: 500; margin-left: 10px;}.normal > div > div{width: 100%; display: grid; grid-template-columns: 1fr 1fr;}.normal > div > div > div{display: grid; grid-template-columns: 0.7fr 1.3fr; padding: 10px;}.normal > div > div > div > *{line-height: 25px;}.normal > div > div > div > span{font-size: 18px; font-weight: 400; padding: 8px 0;}.normal > div > div > div > input,.normal > div > div > div > select{width: 80%; font-family: "Poppins", sans-serif; background: none; outline: none; padding: 5px 5px; margin-right: 7.5px; font-weight: 400; font-size: 16px; color: #333; border: solid 2px black;}.apple-switch{position: relative; -webkit-appearance: none; outline: none; width: 60px !important; height: 35px; background-color: [#fff](https://forum.espruino.com/search/?q=%23fff); border: 1px solid [#D9DADC](https://forum.espruino.com/search/?q=%23D9DADC); border-radius: 50px; box-shadow: inset -20px 0 0 0 [#fff](https://forum.espruino.com/search/?q=%23fff);}.apple-switch:after{content: ""; position: absolute; top: 0; left: 0; background: transparent; width: 31px; height: 31px; border-radius: 50%; box-shadow: 2px 4px 6px rgba(0,0,0,0.2);}.apple-switch:checked{box-shadow: inset 20px 0 0 0 [#4ed164](https://forum.espruino.com/search/?q=%234ed164); border-color: [#4ed164](https://forum.espruino.com/search/?q=%234ed164);}.apple-switch:checked:after{left: 20px; box-shadow: -2px 4px 3px rgba(0,0,0,0.05);}</style></head>');
    print(process.memory().free);
    res.write(`<body><div class="side-bar"><div><span id="hostname" class="font-20">${hostname}</span><br><span class="font-18"> ESP8266 V0.1 </span></div><div class="verzeichniss"><h3>Einstellungen</h3><ul><li onclick="activateAllgemein()">Allgemein</li><li onclick="activateWlan()">WLAN</li><li onclick="activateSensoren()">Sensoren</li><li onclick="activateNtp()">NTP-Server</li></ul></div><div class="main-buttons"><button type="submit" form="mainContent">Save</button></div><div class="main-buttons"><a href="./reboot"><button name="Reboot">Neustarten</button></a></div></div>`);
    print(process.memory().free);
    res.write(`<form action="/" method="post" class="main-content" id="mainContent"><div id="allgemein"><div class="header">Allgemein</div><div class="content"><table><tbody><tr><td>Hostname:</td><td><input type="text" name="hostname" id="hostname" value="${hostname}"></td><td>Currenttime:</td><td>25.03.2021 17:07:54</td></tr><tr><td>Wlan:</td><td>${wifi.getDetails().ssid}</td><td>Uptime:</td><td>25.03.2021 17:07:44</td></tr><tr><td>Mac:</td><td>${ip.mac}</td><td>Ip:</td><td>${ip.ip}</td></tr></tbody></table></div></div>`);
    print(process.memory().free);
    res.write(`<div id="wlan" style="display: none;"><div class="header">WLAN</div><div class="content normal"><div><div><div><span>Wlan SSID:</span><input type="text" name="wlanSSID" id="wlanSSID" placeholder="Name des Wlans." value="${config.wifi[0].ssid}"></div><div><span>Wlan Passwort:</span><input type="text" name="wlanPW" id="wlanPW" placeholder="Passwort des Wlans." value="${config.wifi[0].pw}"></div></div><div><div><span>Statische IP:</span><input type="text" name="staticIP" id="staticIP" placeholder="Leer lassen für DHCP."></div><div><span>Gateway-IP:</span><input type="text" name="gatewayIP" id="gatewayIP" placeholder="Nur wenn man eine Statische Ip nimmt."></div></div><div><div><span>Netzwerkmaske:</span><input type="text" name="networkMask" id="networkMask" placeholder="Normalerweise 255.255.255.0."></div><div><span>DNS IP:</span><input type="text" name="dnsIP" id="dnsIP" placeholder="Nur wenn man eine Statische Ip nimmt."></div></div></div></div></div>`);
    print(process.memory().free);
    res.write(`<div id="sensoren" style="display: none;"><div class="header">Sensoren</div><div class="content normal"><div><span>Temparatur Sensor DS18B20:</span><div><div><span>Pin:</span><input type="text" name="pin" id="pin" placeholder="Welchen Pin (1, 2, 3, 5...)."></div><div><span>Sensor:</span><select name="sensorTyp" id="sensorTyp"><option value="0">DS18B20</option></select></div></div><div><div><span>Temparatur unterschied:</span><input type="text" name="eichung" id="eichung" placeholder="Um die abweichung anzugleichen."></div><div><span>Sendeintervall:</span><input type="text" name="sendIntervall" id="sendIntervall" placeholder="Der Sende Intervall in Sekunden."></div></div><div><div><span>Node.js Server IP:</span><input type="text" name="serverIP" id="serverIP" placeholder="Ip des Node.js Servers."></div><div><span>Node.js Server Pfad:</span><input type="text" name="serverPfad" id="serverPfad" placeholder="Der Pfad an dem es Senden soll."></div></div><div><div><span>Geräte ID:</span><input type="text" name="clientID" id="clientID" placeholder="Die ID die im Server vermerkt sein muss."></div><div><span>Gerät Passwort:</span><input type="text" name="clientPW" id="clientPW" placeholder="Das Passwort zur ID."></div></div></div></div></div>`);
    print(process.memory().free);
    res.write(`<div id="ntp" style="display: none;"><div class="header">NTP-Server</div><div class="content normal"><div><div><div><span>NTP-Server:</span><input type="text" name="ntpServer" id="ntpServer" placeholder="IP des NTP-Servers." value="${config.ntp.server || ""}"></div><div><span>Zeitzone:</span><input type="text" name="timeZone" id="timeZone" placeholder="Zeitzone des NTP-Server." value="${config.ntp.timeZone || ""}"></div><div><span>Winterzeit/Sommerzeit:</span><input class="apple-switch" name="summerTime" id="summerTime" value="true" type="checkbox" ${config.ntp.summerTime ? 'checked=""' : ''}></div></div></div></div></div></form>`);
    print(process.memory().free);
    res.write(`<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script><script>function dilsplayNoneForMainContainer(){let allDivs=$('#mainContent > div').toArray(); for (let i=0; i < allDivs.length; i++){allDivs[i].setAttribute("Style", "display: none;");};}; function activateAllgemein(){dilsplayNoneForMainContainer(); allgemein.setAttribute("Style", "display: inline-block;");}; function activateWlan(){dilsplayNoneForMainContainer(); wlan.setAttribute("Style", "display: inline-block;");}; function activateSensoren(){dilsplayNoneForMainContainer(); sensoren.setAttribute("Style", "display: inline-block;");}; function activateNtp(){dilsplayNoneForMainContainer(); ntp.setAttribute("Style", "display: inline-block;");}; </script></body></html>`);
    res.end();
    print(process.memory().free);
}

This happens:

>Uncaught SyntaxError: Got EOF expected '}'
 at line 51 col 114
...in.js"></script><script>fu
                             ^
Execution Interrupted
New interpreter error: LOW_MEMORY,MEMORY

I need a way to send the HTML, CSS and JS without using up all the RAM, but I still need to be able to insert data into the HTML.

• #7

Julian1
I changed a bit of my code, and now I am starting the function with 900 free blocks.
I looked how much memory the function is using with E.getSizeOf() and got this:
```
>E.getSizeOf(sendMainPage, 1)
=[
  {
    name: "\xFFa",
    size: 1 },
  {
    name: "\xFFcod",
    size: 323 },
  {
    name: "\xFFlin",
    size: 1 }
 ]
>E.getSizeOf(sendMainPage)
=326
//326 Blocks * 16 =  5216 bytes
> 
```
So the function is using 20% from the total blocks (1600) or 36.2% from all available blocks (900) of the ram without the CSS.
But that's only the size when I request the page its using 430 blocks:
```
//start of the function sendMainPage
{ "free": 895, "usage": 705, "total": 1600, "history": 85,
  "gc": 0, "gctime": 1.827, "blocksize": 16 }
//after res.end();
467
>900-470
=430 //blocks
```
That's 47.7% of the free ram without CSS.
I think I got something that could help (How to send a large amount of data faster ?). But I don't know how that should work with the long HTML string. It would be nice if you could help me with this.
• #8

allObjects

I don't know about your hardware configuration, but there is a way to stream data from flash... take. look at this conversation about Best way to stream base64 encoded string to a web client. This is not the only conversation about how to handle large amounts of data. You find more, also in the way how to upload and save it right away in flash which also gives you more options of managing your resources. But as said, it depends on your hardware because not all of Espruino runs or is supported in some hardware. Best is to use an Espruino genuine board, because the Espruino firmware works there the best.
• #9

Robin in reply to @Julian1

Wed 2021.03.31

post # 'What do you mean with this?' ref: 'the 515 'usage' value'

See 'usage' below heading 'Description'

http://www.espruino.com/Reference#l_process_memory

My intent was to make aware the amount of memory used by the actual code instructions initially uploaded. As no source code had been provided before that #5 post, I had no way of knowing whether additional code was being uploaded from the Left-Hand console side of the WebIDE.

LAW of Unintended Consequences

post #1 'I have a simple http server that responds with an 8KByte website'

I mis-understood what task was being attempted while comparing your thread 'Is there a way to make a Basic Auth with the Webserver?' with this thread.

See sample snippet below example function getPage()

http://www.espruino.com/WiFi#using-wifi

As in that example, I took that phrase to mean a Http Request whose fetched content responds with 8K and memory became full during reception. And as there was no objection to the addition of FRAM in post #2, continued down that line of thinking.

A wild guess for post #6 last code block after heading 'This happens:' L3 might be that there is a missing trailing quote, mis-matched internal quote(s), improper nesting of quotes, missing semi-colon, etc.

Has the use of a Html validator been tried?

The use of base64 as @allObjects points out would assist in reducing those types of errors, but at the penalty of ~30% increase in string size.

Try reorganizing memory:

Definition of a flat string and example
• #10

allObjects

I did not intend to propose base64, I just wanted to point out the approach.

More is to say: Reading strings form require("Storage").... has the advantage of being mapped until pulled and is perfect for streaming; see transferring files using special setup for Transferring large amounts of data on https://www.espruino.com/Internet. With that the application can control how big the memory chunk is to be. At best, it could be that no RAM - or just for control - is used because some DMA was in discussion, and that could move data directly from flash to http transmit buffer. I do not know the internals, but streaming is for sure the way to go.

Next thing is, @Julian1, do you really need to serve everything from Espruino? UI infrastructure stuff can be pulled from anywhere and when you are on the www, only essentials have to be delivered from Espruino device and all the other stuff comes from a regular server, even from Google or so... For an example take a look at poast about Shiny Web UIs - you can check out the whole conversation and conversation related to this.

A different approach is to just ship the data to a mid tier Web and App server stack using mqtt. A flavor of that can be boot the app from that Web and App server and the connect only for the data directly (triangle) using pull / push-pull / long-polling...
• #11

Robin in reply to @allObjects

'I just wanted to point out the approach.'

Thank you @allObjects for that clarification.

@Julian1 had pointed out that 'I am new to this' so was attempting to keep things as simple as possible on each post.

As I pointed out in post #3 there might be a bit of back-n-forth while in the process of resolving this task, as I've now three sub-topic path directions we could possibly go.

With your clarification, a nice place to segway to the external memory method using storage.

The SDCard route:

http://www.espruino.com/File+IO

https://www.espruino.com/Reference#l_E_connectSDCard

• #12

Julian1

Thank you for your help @allObjects and @Robin.
I got it now working, and it's a good solution (for me).
The idea is from Best way to stream base64 encoded string to a web client and I used the drain event.
I saved the head and the body in the Storage module and send them chunk for chunk.

function sendMainPage(res) {
    res.writeHead(200, {'Content-Type': 'text/html'});
    res.write('<html lang="en">');
    //chunk size
    let bytesperChunk = 2048;
    //cssN stores how many chunks i need to send
    let cssN = Math.ceil(storage.read('main.css').length / bytesperChunk );
    //cssI stores how many chunks are already send
    let cssI = 0;
    let htmlN = Math.ceil(storage.read('main.html').length / bytesperChunk );
    let htmlI = 0;
    res.on('drain', function(){
        if (cssN == cssI) {
            if (htmlN == htmlI) {
                //Here is my client js code with all the values and the end of the html file 
                res.end(``);
                print('end');
            }else {
                res.write(storage.read('main.html', htmlI * bytesperChunk , bytesperChunk ));
                print(process.memory().free);
                htmlI++;
            }
        }else {
            res.write(storage.read('main.css', cssI * bytesperChunk , bytesperChunk ));
            cssI++;
            print(cssI);
        }
    });
}

I don't know if it's the best solution, but it works for me and when I request the page the loading time is only 2-3 seconds.
Not only that, but I still have 850 free blocks of ram, because now I don't need to load the long HTML string before I send it.

• #13

hungryforcodes

It's just an ancillary note, but there is a function to write out your code to flash directly as opposed to storing it in RAM. It's in the Web IDE -> Settings -> Communications -> Save On Send -> "Direct To Flash".

You can experiment with that as it might free up some additional JSVars if you were not already using it.

-hfc
• #14

Julian1

Thank you, @hungryforcodes I just tried it out and now my ram is on a page request at 1156 free blocks that's an improvement of 300 blocks.
• #15

allObjects

@Julian1, glad you got it working great for you. Initially, I wanted to point out the path @hungryforcodes pointed out. It is a great option when you ran out of memory for the the code. For development though, I don't like it much. When I ran out memory, I developed the components with plain upload and used 'save-on-send' for the whole with all the components together. Using the Storage module, or - as @Robin mentioned - use serially connected, external flash. Doing so, you can easily chane the data on the flash. You can even just put a loader into Espruino and have all the code (and data) on the external flash. On startup, the loader reads the code as modules dynamically / on demand and operates on the data. (Of course, you do not get the support from the upload regarding dependencies, but you get great flexibility (cross devleopincluding practically unlimited storage.
• #16

fanoush in reply to @Jean-Philippe_Rey

I guess, not much can be said as the issue seems to be solved so will just comment this

If you need fast RAM to store live pages it could be worth checking at FRAM modules such as this one: https://www.adafruit.com/product/1895

There is also quite cheap SPI RAM, e.g. 8 megabytes for <$1 https://www.aliexpress.com/item/4000164056594.html

The commands and pinout are exactly the same as with SPI flash, it is just that the storage is volatile. So may be useful for some cases. One can even put RAM chip on top of Flash chip and have just different CS pin (picture here). Disadvantage of such RAM is power draw - 400uA to keep the data, miliamps to read/write.
• #17

hungryforcodes

So this is an ESP8266, let's be clear. Has anyone ACTUALLY been able to increase practical RAM in this device using external i2c or SPI modules? The ESP32 certainly has PSRAM, but that Espressif provides this solution suggests RAM expansion itself is not so straight forward as just wiring up modules. For other Espruino Official boards is it possible? Wouldn't there be required some modifications (or at least an i2c driver for example) to Espruino itself to support it?

Also keep in mind that the original poster said he was "new to this". While uploading individual code modules using Storage is a new and innovative approach to coding in Espruino for sure (and then after say using "eval" to load these modules), it's certainly more complicated.

That said, I'm certainly looking into it, but it's a new way of thinking I find.

Post a reply
- Bold
- Italics
- Link
- Image
- List
- Quote
- code
- Preview
Formatting Help

Don't worry about formatting, just type in the text and we'll take care of making sense of it. We will auto-convert links, and if you put asterisks around words we will make them bold.

Tips:

Create headers by underlining text with ==== or ----

To *italicise* text put one asterisk each side of the word

To **bold** text put two asterisks each side of the word

Embed images by entering:
![](https://www.google.co.uk/images/srpr/logo4w.png)
That's the hard one: exclamation, square brackets and then the URL to the image in brackets.

* Create lists by starting lines with asterisks

1. Create numbered lists by starting lines with a number and a dot

> Quote text by starting lines with >

Mention another user by @username

For syntax highlighting, surround the code block with three backticks:

```
Your code goes here
```
Just like Github, a blank line must precede a code block.

If you upload more than 5 files we will display all attachments as thumbnails.

For a full reference visit the Markdown syntax.

Transferring 8KB of Data

About