MDBT42Q's flash has been emptied! Everything disappeared from the storage.

I use the MDBT42Q with the 2v09 firmware.
I connected the ESP8266 to it.
After 2 months of operation, the flash became empty!

>require("Storage").list()
=[  ]
>require("Storage").getFree();
=40960

The code and the logs are disappeared.
The device has been self-initialized, working, and can be connected.
I tried another MDBT42Q yesterday and the same thing happened!
Is it possible that the storage write could cause a problem?
I do a simple log, I write very little data (basically i am writing a counter):

function log(f){
     if (require("Storage").read(f) == "undefined") {
      require("Storage").write(f,'1');
    }else{
      var n=parseInt(require("Storage").read(f))+1­;
      n=n.toString();
      require("Storage").write(f,n);
    }
}

This log runs very rarely.

I couldn't manually reproduce this thing maybe someone has a similar experience?
This is very scary for me.

Wow, that's not great. Is the MDBT42Q getting turned on and off a lot? Is it possible that it got turned off while it was writing to flash? If that happened at just the right point while it was compacting then it could cause the issue.

What happens is repeated writes slowly fill up flash, and at some point the flash has to be 'compacted' to remove the old files. If that fails then the data could be lost, but that code should be extremely well tested now.

Historically there have been issues but assuming you're on 2v09, that should be good. There was an issue in 2v09 (fixed in cutting edge) which meant that functions stored in Flash got relocated wrong in some cases, but that wouldn't erase flash - it'd just mess things up until a power cycle.

Is that function log(f){ exactly the same function you're using? If there were a compaction problem It's possible that just running it thousands of times would reproduce it.

If you can get it reproduced I'm very happy to look into this and see if I can get a fix in

Ok, I'm just running this code (writing to multiple files, 100 times a second) on an MDBT42Q with firmware 2v09 and we'll see if it breaks:

function log(){"ram"
  var f = "log"+Math.floor(Math.random()*8)+".txt"­;
   if (require("Storage").read(f) == "undefined") {
    require("Storage").write(f,'1');
  }else{
    var n=parseInt(require("Storage").read(f))+1­;
    n=n.toString();
    require("Storage").write(f,n);
  }
}

require("Storage").write("A","Hello World");
setInterval(log, 10);
require("Storage").write("B","Yet another bit of data for us to store");
setInterval('print(require("Storage").ge­tFree(), require("Storage").list())', 2000);

However it did get me thinking - are you using a watchdog timer? While we should kick the watchdog often enough, I guess if you used a small timeout then it's possible that the device would reboot itself during the compaction phase.

My test has now written about 300,000 files and compacted hundreds of times now, and it's still perfectly fine... I'm going to stop it before I start wearing out flash memory.

Are you sure you're not writing to Storage in some other part of your code?

can it be inadequate power-some power glitch/brownout? if you use esp8266 with it an connect it to same power source the total system power draw may change a lot e.g due to wifi traffic and may cause random power issues if it draws too much.

I have just discovered an issue while working on something else...

If you manage to completely fill up Storage, the next time you reboot, Storage is read as invalid and is automatically erased!

It's actually surprisingly hard to do that...

s=require("Storage");s.write("test","Hel­lo",0,s.getFree()-40)

It only happens with values between s.getFree()-32 and s.getFree()-64. Everything else is fine!

However it's entirely likely that with repeated small writes and frequent resets, you could actually hit this issue.

Cutting edge builds now have this fixed, as will 2v10 when released!

@Gordon thank you for your suggestions and efforts! I tested a lot too. And I have come to the conclusion that sometimes reading and writing to storage is very slow. This is not a problem anyway, but if there is a delayed reboot after that, the memory will be cleared. But it also happens very rarely and I think it has to do with running out of memory.

This strange code sometimes produced this error:

function log(f){  
     if (require("Storage").read(f) == "undefined") {
      require("Storage").write(f,'1');
    }else{
      var n=parseInt(require("Storage").read(f))+1­;
      n=n.toString();
      require("Storage").write(f,n);
    }
}

for (var g=0; g<10; g++) {
	clearInterval();
	clearWatch();
	console.log(require("Storage").getFree()­+'-'+parseInt(require("Storage").read('g­s4.pc'))+1);
	log('gs4.pc');
	setTimeout(E.reboot,2000);
}

Thank you very much!

can it be inadequate power-some power glitch/brownout? if you use esp8266 with it an connect it to same power source the total system power draw may change a lot e.g due to wifi traffic and may cause random power issues if it draws too much.

I thought about it too but i use very good voltage regulators.

I think the issue with the slow storage read/write is that you have literally hundreds of very small files (most of which are old files which were modified). Doing a require("Storage").compact() would fix that - but it increases flash wear.

Hopefully if you use one of the cutting edge builds your problem should be sorted now!

Great, thank you very much!