Extending storage module

Would it be possible to extend the storage module to

• userequire("storage", options) to use storage for external flash storage or specific flash area?

• prep storage content offline like possible for spiff?

• upload offline content to a specific flash are?

Any further ideas how storage could be extended?

Is this mainly for ESP32 usage? I can't imagine many other devices where this makes too much sense? Using external flash would be neat but you'd have to be able to supply external read/write functions too.

But I guess possibly it could be done, although right now none of the functions take an instance parameter so doing require("storage", options) is nontrivial. There is already code in there to handle two flash areas, and usage of C:filename to force where data is stored - I guess making the address of the second area configurable would be possible - but I wonder whether this would be any more useful than if it were just configured for and baked into the firmware?

Is this mainly for ESP32 usage?

YES, other devices do not have the posibility of flash segmentation. Only ESPs have extra flash areas.

but I wonder whether this would be any more useful than if it were just configured for and baked into the firmware?

Yes, that is the other option we have ;-)

Only ESPs have extra flash areas.

So how many there are?

The second area currently supported is configured like this
https://github.com/espruino/Espruino/blo­b/master/boards/BANGLEJS2.py#L116

Then when creating file one of them is "filename" and the other is using C: driver letter like "c:filename" (forgot which one is default). When reading file both areas are searched for the file name so it is some kind of overlay.

The idea was to not really have two separate 'disks' but one merged from both areas.

The code is in common layer
https://github.com/espruino/Espruino/blo­b/master/src/jsflash.c#L31
https://github.com/espruino/Espruino/blo­b/master/src/jsflash.c#L651
so could be usable also for ESP32.

Many thanks fanoush, very interesting feature!

I missed the espruino command line tool that can upload files to storage via myfile.js containing a save to storage command with the wanted content. So all fine.

So can we use spiffs or any other way to load files to storage?
I really want to serve react files from storage (react app) with webserver and websockets on my esp32

Can this be done, or what needs to happen for it?

use spiffs or any other way to load files to storage

The Storage https://www.espruino.com/Reference#Stora­ge already has its own simplified filesystem so no spiffs is needed/possible there. The main feature of Storage is that each file takes continuous area of flash and files are directly mappable to some memory address. The disadvantage is that you cannot append to such file or overwrite existing data, you need to create it with some size and either fill it with data completely at creation time or write only to areas not written yet = bytes with all bits set (0xff = 255). So it is something between low level flash access and simplified filesystem.

Then there is a hybrid on top of it https://www.espruino.com/Reference#Stora­geFile which can append but cannot write/contain 0xff=255 bytes.

But while it is limited as described it can work just fine for serving text files with webserver.

No spiff, because it is not necessary, the method you can use is described in #5

Can you please give an example on how to do this?
Can I upload a list of file, or just one minimized file? (I want to upload a react app and serve it)
How to do that (I could not find/understand where to look),
How to serve it from espruino.

That would be great.
(or some more hints would also be good, as I do not know exactly what to look for)

see this conversation Howto write modules to storage?

and you can just load the file as a string with require("Storage").read("filename") - the string will be kept in flash and will be loaded on demand. If it's too big to send in one go you can use E.pipe to pipe it to the HTTP response