Lost storage data on esp8266

Hey guys, I found an issue that sometimes I will lose all data on the storage. I use websocket to get new code from the server and write it into the storage. At first, I simply erase and write, but then I realized free space becomes less and less so I add compact between erase and write, it works normally most of the time, but it did lost all data sometimes.

I would like to know how to prevent this, I'm a noob, thanks.

Can you share some infos.

• Board
  
• Espruino version
  
• code snippet
  

Yeah, the board is esp8266-01s and the version of espruino is 2.04. Usually, the program will get new code from WebSocket server and run

require("Storage").erase("usercode");
require("Storage").compact();
require("Storage").write("usercode", newCode);

Maybe it is related to the duplicate files issue, see http://forum.espruino.com/conversations/345948/#comment15216864

Can you do a

require("Storage").list("usercode")

to see if there are duplicate files? But, they may they only appear if you are acutally experiencing the "data loss".

You could possibly also update to 2.05 and then see if the issue is gone.

Yes, a upgrade would be helpful as @Raik suggested.

Yes, due to some code changes, the issue does not appear frequently. I will try the same actions on 2.05 and post the result. Thanks for your help.

Yeah, I will try that, thank you.

@MaBe @Raik I tried to update to 2.05 and run some tests. The result is this issue does not vanish.
I have code in .boot0 which is

if(fs.getFree()<4096) fs.compact();

and I can see espruino will compact and restart when the free space down below 4096, then I run fs.compact() again in console after it restarted, and after reboot, all data were lost. fs.getFree() shows me 16384 on esp8266-01s, and fs.list() has an empty array as result.

and I haven't tried this on nodemcu, but I guess I will get the same result.
So someone please tell me how to deal with it, is my action wrong? Or just an issue in espruino, I don't want to take it out to code or upgrade after I set it into a dishwasher. Thanks!

esp8266-01s

With 1 Mb ?

@JumJum is the firmware builder online ?

Yes

I guess require("Storage").compact(); can't work because storage space is to small for what you are trying.

What about eraseAll(), write .boot and usercode, or by a 4MB board or

build a 1MB firmware Espruino version without FOTA but lots of space like a 4MB board.

        BOARD=ESP8266_4MB make clean
        BOARD=ESP8266_4MB RELEASE=1 NO_FOTA=1 FLASH_1MB=1 make 

Fixed typo

I have NodeMCU(32MB version), but the size(physical) is too big. And my project only needs 2 IOs.

Besides, this happened when I tried to require("Storage").compact();, the free space on the flash at least has 60%, and my purpose is to compact it myself instead of automatically(when the free space run out), cause that will lead to data loss.

And I don't know how to compile it myself, I cloned the source and tried to compile, but I have no idea where I could find the hex file, I am noob. :D.

But anyway, thank you for the reply.

I made a version using your arguments and flashed the 01s, but the issue still happened when I do the same actions. But I do get 196608 space like 4MB board.
And, could I ask why the NO_FOTA=1 don't work for 4MB board, like NodeMCU, I can't still use FOTA on it. My purpose is to get more space beyond 196608 on 4MB board, thank you.

And, could I ask why the NO_FOTA=1 don't work for 4MB board, like NodeMCU

If running run a 4MB firmware image on a 1MB chip, flash over the air is not possible.

Flash over 1MB is not uses with the Storage module, but can be used with the Flash library.

but the issue still happened when I do the same actions

Please share snippet to reproduce it.

I mean, I don't FOTA, so I would like to disable it on 01s(1mb board) and NodeMCU(12F). Now, I did it on 01s but failed on NodeMCU by using different firmware, but two firmware both added argument NO_FOTA=1.
Are there any methods to do this on NodeMCU? Thanks.

To reproduce it, you need to:
(1).Get a string, length about 1000(I download string from WebSocket).
(2).

fs = require("Storage");
fs.erase("aName");
fs.write("aName", thatString);

(3). Code in .boot0:

if(fs.getFree()<4096) fs.compact();

if the free space did less than 4096 and compact() has run and reboot itself after compact, then run fs.compact(); again, if those actions right, you will lost all data in flash.

So according to this issue, I guess flash must have an amount of the abandoned data before running compact, if not, the compact function will cause data loss.

To reproduce it, you need to:
fs = require("Storage");
fs.erase("aName");
fs.write("aName", thatString);
(3). Code in .boot0:
if(fs.getFree()<4096) fs.compact();

Well this is no code to reproduce the error.