New Flash API, and nice surprise for Pico owners!

Hi,

I just did a new build, with a more universal API for accessing flash memory, which will work on both the Original board and the Pico. It's here and will be in 1v80 when it's released.

Rough outline:

>var f = require("Flash");
>var freeFlash = process.memory().flash_code_start;
>f.getPage(freeFlash)
={ "addr": 134234112, "length": 16384 }
>f.read(12, freeFlash)
=new Uint8Array([255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255])
>f.write("Hello World!", 134234112)
=undefined
>f.read(12, freeFlash)
=new Uint8Array([72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100, 33])
>f.erasePage(freeFlash)
=undefined
>f.read(12, freeFlash)
=new Uint8Array([255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255])

It'll all go in the reference when the full release is made.

Be careful, because you can easily blow away your bootloader with it if you're not careful!

And the surprise is...

>var freeFlash = process.memory().flash_start + 511*1024;
=134740992
>f.read(12, freeFlash)
=new Uint8Array([255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255])
>f.write("Hello World!", freeFlash)
=undefined
>f.read(12, freeFlash)
=new Uint8Array([72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100, 33])

So there's a whole extra page of flash memory, 128kB long at address 0x8060000 that we shouldn't have. Much like on the original boards, the chip is a STM32F401CE even though it's marked as a STM32F401CD.

While there's no guarantee that newer boards will be like this, I reckon all the current ones are!

... also this firmware has some new changes that use simple RLE compression when saving program code to flash. It means I've been able to raise the amount of variables from 3000 up to 5100. While you won't be able to save a program that uses all of them, realistically you need a few free variables to run your program, and those will compress down.

WOOT WOOT WOOT! MOAR RAM!
Just in time too - I might have scraped by under 3kJSV, but it might have been tight. But 5.2kJSV will be plenty!

It's a shame we can't rely on the bonus flash, that'd be the perfect place to save the code to!

So I should discontinue the picoram builds now?

So I should discontinue the picoram builds now?

Totally up to you. You could I guess tweak PICO_1V3.py such that it sticks the saved data in that final page of flash, then you won't hit problems if the RLE compression doesn't manage to compress the data enough.

Well, this build seems to work for me? I tried it on the room controller (the one I've posted pics of here), and save()'ed code seems to work correctly, though I wasn't pushing the size limits.

Great, thanks!

We should be able to do a lot better on the compression front, I'll have to look at it a bit more - but for now RLE seems to do a relatively good job.

!!!

I just realized something. This new flash capability could be used to overwrite the bootloader, you say... Doesn't that also mean it could be a way to update the bootloader without having to enter DFU mode? Entering DFU mode is a bit difficult if you've soldered the pico onto something, so the pads you have to short can't be accessed without unsoldering 26 castellated pads.

Yes, you can do that with it too...

I was just avoiding doing it because it's a great way to brick your Pico (you'd then have to enter DFU to fix it) if something goes wrong :)

But if you created a Uint8Array with the first 16kB of the Espruino binary, then erased the page at 0x08000000 and then wrote the array into that address, you should be sorted.

but if you want to enter DFU with a Pico soldered down, there's another method too. If you look on the top of the board there's a small gold teardrop. That's BOOT0, so if you just short that to 3.3v while power is applied the board will enter DFU mode :)

Uhh...

Bad news. I've been having hard failures that require reflashing firmware to fix.

No idea what I do, but sometimes after save(), next time I unplug and replug, the serial isn't recognized, and appears as Unknown USB Device (Device descriptor request failed).

:-(

Very strange. The issue may be linked to the webserver it was trying to talk to not being online.

I've gone back to release version for now.

1 Attachment

• VoiceController.js

:( I actually did a full release on Fri 24th, then had to back it all out because I realised there was a problem.

I found something bad happened repeatably when I did:

• save();
  
• Maybe power off/on
  
• do not reset()
  
• Change something
  
• save();
  
  

Does that sound like your problem?

Yup

Please could you try this build when it's compiled - it's just the latest on GitHub.

I think I may have found the problem now - at least I haven't managed to get it to die yet.

Cool, I'll flash it as soon as I get home :-D

Anyone had any luck with this yet?

Hi Gordon

I'm able to access the extra page on the pico. Great.
Have a question regarding Flash.erasePage. Will this allways erase the complete page, in this case the 128KB flashmemory ? Is there a way to only erase a part and rewrite that part ?

Thanks

Sacha

From my experience with other NAND flash memories, I would strongly assume that you can only erase a whole page, because that's the way they work. There are non-volative RAMs out there as well, without that limitation, but they are much more expensive.

I'm not sure about this particular chip though.

Yes, I'm afraid it's only the complete page. The STM32F4 does have different size pages (there are 16k and 64k ones too, earlier in memory) but those are used for Espruino.

Hi Gordon

Thanks for your answer. Now i do have some problems with the extra flashmemory. I've wrote 5 junks of 256 bytes. The first ones passed successfull. But then the pico freezed. I have to isolate the code to reproduce it.

Another question. Is your example above using memory at process.memory().flash_code_start save ? Can i start using the 16kb there ?

"flash_code_start" is not documented here:

http://www.espruino.com/Reference#l_process_memory

Many thanks
Sacha

Yes, it'd be good if you could come up with a bit of code that showed the problem. When the Pico freezed, did it work when you reset it, or did you have to re-flash it?

process.memory().flash_code_start save - can I start using the 16kb there ?

Yes and no... That is the area of memory used by save()... So you can use it, but if you do it'll destroy any program code you saved (which is probably not what you want).

Unfortunately due to the page arrangement on the Pico, pretty much everything apart from that final 128kb is used.

One solution would be to treat the whole 128kB as a 16kB page. If you needed to change something you could then read the whole 16kB into a Uint8Array in RAM, erase the flash, and then write the new data back.

edit: I'll update the docs for process.memory too

Thanks. Upps. Yes i do not want to use flashmem where save() stores data itself. ;-).
After a reset() the pico worked as expected. Will try to give some code examples this evening.

Sorry, not reset() of course. After unplug and replug the pico.

Hi Gordon,

Running this code:

// Flashtest

function gen_256() {
    var i;
    var str='';
    for(i=0;i<256;i++) {
        str+='X';
    }
    return(str);
}

var bytes256=gen_256();
var f = require("Flash");
var freeFlash = process.memory().flash_start + 511*1024;
console.log(f.getPage(freeFlash));
console.log('LEN:'+bytes256.length);
f.erasePage(freeFlash);
console.log('Nr.1, at '+freeFlash);
f.write(bytes256, freeFlash);
freeFlash+=256;
console.log('Nr.2, at '+freeFlash);
f.write(bytes256, freeFlash);
freeFlash+=256;
console.log('Nr.3, at '+freeFlash);
f.write(bytes256, freeFlash);
freeFlash+=256;
console.log('Nr.4, at '+freeFlash);
f.write(bytes256, freeFlash);
freeFlash+=256;
console.log('Nr.5, at '+freeFlash);
f.write(bytes256, freeFlash);
freeFlash+=256;
console.log('Nr.6, at '+freeFlash);
f.write(bytes256, freeFlash);
freeFlash+=256;
console.log('Nr.7, at '+freeFlash);
f.write(bytes256, freeFlash);

Sorry have troubles formatting it as code.

The output running above code is:

 _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v81.149 Copyright 2015 G.Williams
>echo(0);
{ "addr": 134610944, "length": 131072 }
LEN:256
Nr.1, at 134740992
Nr.2, at 134741248
Nr.3, at 134741504
Nr.4, at 134741760

Freeze. Have to unplug/plug it back to recover.

Sacha

Doublepost

Hi Sacha,

To format code, just stick 3 backticks before and after it - check out this page. It seems like the forum guys have broken the code button now!

The issue is just that you're going past the end of available memory...

• You have have 512*1024 bytes total in the chip
  
• You start at 511*1024
  
• The first 4*256 = 1024 bytes write great
  
• But now you're at the end of memory and any further writes cause a hardware error
  
  

If you change the start address to 510*1024 then it should work fine - in fact you've got the whole page from 384*1024 (which you can set with freeFlash = f.getPage(freeFlash).addr)

Wow, that easy. I did not see where the staring point of the free memory is. Will try it this evening.
Thanks

Sacha