nrf51288-DK storage

Hello, I'm new to Espruino, however I really like it!

I've built nrf51288-DK Espruino image to play with an nrf51288 chip. It mainly works, e.g. Bluetooth stuff, reading digital analog pins etc. However I can't get it to store something in EEPROM. I'm following some example where you need to load a module, e.g. Storage, Flash etc but non of them actually loads...

I've got a simple code that does:

require("Storage");
console.log(Storage.debug());

This print the following:

Uncaught Error: Module Storage not found
 at line 29 col 18
require("Storage");
                 ^

Does it mean that nrf581 chip does not support Storage?

PS. I've tried to load "Flash" module with the same result.

nrf51288 has only 256KB of flash (and no EEPROM) so even the Espruino build had to be cut down on features so Flash and Storage modules are not present. See e.g. https://www.espruino.com/Reference#Storage "Note: This is not available in devices with low flash memory"

I'd suggest to get nRF52832 for Espruino instead, it is better and not that much expensive, there are e.g very cheap <$5 nrf52 modules on aliexpress (like this one posibly with this adapter) . Or the DS-D6 smartwatch is currently $6.99 on gearbest and you also get battery+charger and display with the nrf52 chip :-) So IMO there in no reason to get nrf51 today if you target Espruino unless your needs are very, very basic. Arduino might be better fit for nrf51.

Hi,

You mean the nRF51822?

That's not quite the way you'd use storage. You could try:

var storage = require("Storage");
print(storage.list());

But I just checked and the 51822 build doesn't have it in, since it's got SAVE_ON_FLASH defined. nRF51 is really tight on flash memory so you can't easily build without.

You could modify NRF51822.py to remove the debugger and tab complete, then edit jswrap_storage.c to remove all the "ifndef" : "SAVE_ON_FLASH", - but honestly you're going to keep hitting trouble with nRF51 - it's just not really big enough to handle Bluetooth and Espruino

Well, if you fit Flash/Storage code in like this, it is still a question if there is enough spare flash pages to actually use it.

Adding cheap SPI flash might be a solution however Flash module may not support it? There was some conversation here http://forum.espruino.com/conversations/269195/ but I don't see it in list of current modules. Or i2c/spi eeproms might do, these are supported http://www.espruino.com/AT24 http://www.espruino.com/AT25 .

Thank you guys for your answers. I've got a bunch of those nrf51822 boards that I want to make use of to build some very basic sensors, e.g. water meter, pir, temperature, humidity etc. Effectively, scripts are going to be very basic and small in size, however I need to store a number, e.g. rotation count etc, this is just a small number to back up readings in case of any power outages.

This is an experimental project as a part of Eclipse SmartHome/OpenHab Bluetooth binding and Java Bluetooth Manager development.

I will definitely try to enable Flash module first, if this fails, I'll get some nrf52 boards.

Or maybe someone has a better idea on how to store a small number/variable?

Hi @Gordon, thanks for your suggestion to comment out "ifndef" : "SAVE_ON_FLASH", this actually works! I've removed all ifndef statements in the jswrap_storage.c file and also removed 'DEFINES += -DUSE_DEBUGGER -DUSE_TAB_COMPLETE', line in the NRF51822DK.py file. Now I can read/write stuff via Storage lib.

var storage = require("Storage");
print("Initial: " + storage.getFree());
storage.write("MyFile", "Some data");
print(storage.read("MyFile"));
print("After storing some data: " + storage.getFree());

This prints:

>Initial: 3072
>Some data
After storing some data: 3044

It seems to me 3072 bytes is more than enough for storing some basic backup data.

BTW, could you please let me know what are these USE_DEBUGGER and USE_TAB_COMPLETE ? Anything important/useful? Cheers

You also need some Storage space for your javascript code if you need it permanetly stored to run after poweron, this is saved into same space. Tab complete is used in interactive console to complete object methods/properties when you press tab key after dot. For debugger type 'debugger' in console to see it.

I see now, getting 896 bytes free space after saving a simple script.

Is there anything else I could disable to free up some space?

Update: JS minification helped a bit: 1312 bytes vs 896 (unminified)

When checking https://github.com/espruino/Espruino/blob/master/boards/NRF51822DK.py try removing GRAPHICS library, then removing gpio as pin reset define could save some code (reset should still work as long as this code run at least once on this board). Also in same board file try to increase page constants '3' in saved_code section as the size and start of Storage is hardcoded there. If you set it too large the linker should fail.
EDIT:
It looks that CONFIG_GPIO_AS_PINRESET is perhaps only relevant for nrf52 and got copied there by mistake and does nothing for nrf51

I just tried with bare 51822 module from ebay. I only removed debugger and tab complete added storage (and reduced RAM to 16KB and variables to 300 since my module only has 16kb) and while building I see

CODE: 110592 -> 255820

so I increased storage pages to 6 and while building I see

Testing espruino_2v01.102_nrf51822.elf for NRF51822DK
STORAGE: 256000 -> 262144
CODE: 110592 -> 255820
Code area Fits before Storage Area
GEN espruino_2v01.102_nrf51822.hex

and in console I see

>require("Storage").getFree()
=6144
>process.memory()
={ free: 246, usage: 54, total: 300, history: 33, 
  gc: 0, gctime: 3.60107421875, "stackEndAddress": 536884592, flash_start: 0, "flash_binary_end": 255828, 
  "flash_code_start": 256000, flash_length: 262144 }

Thanks @fanoush, I've pushed 3 to 4, looks like working.

When I build a new FW, it prints this:

Testing espruino_2v01.108_nrf51822.elf for NRF51822DK
STORAGE: 258048 -> 262144
CODE: 110592 -> 247628
Code area Fits before Storage Area
GEN espruino_2v01.108_nrf51822.hex
Merging SoftDevice

Does it mean that I still can allocate even more space for storage? E.g. tune that 3 number so storage starts with 247628 (is it address right?)? If so, looks like I can get roughly another 10k (258048 - 247628 = 10420)? Cheers

Yes. If you give it too much the build will fail. Removing GRAPHICS gives me similar numbers. Also when adding back tab complete (i.e just removing debugger and keep everything else in) I can still have 5 pages. This is not so bad after all :-)

EDIT: Also if you would enable Flash module too you could keep some space for Storage and the rest is automatically available for Flash - which allows saving pages directly. For some use cases saving/erasing pages directly to store data might make better sense? See src/jswrap_flash.c the getFree method there should figure out free space automatically.

Perfect! Thank you.

Just wondering why nrf51822-dk image is not available on the official image download page? There are only Micro:bit and nrf52 firmwares.

Alright... After doing this:

1. Disabling GRAPHICS
   
2. Removing 'DEFINES += -DUSE_DEBUGGER -DUSE_TAB_COMPLETE',
   

3. Configuring saved_code as per this:

   'saved_code' : {
   'address' : ((256 - 14) * 1024),
   'page_size' : 1024,
   'pages' : 14,
   'flash_available' : (256 - 108 - 14)
   }
   

I'm getting:

1. In the build output:

   STORAGE: 247808 -> 262144
   CODE: 110592 -> 247628
   

2. In the app: storage.getFree(): 14336 bytes.

Great success, I think it is more than enough.

Thanks for your suggestion to look at the Flash module, I will soon.

Great! Glad that's all working for you now. Since you are now using all available flash for Storage, you may find the Flash module isn't too useful for you.

wondering why nrf51822-dk image is not available on the official image download page?

While Espruino is Open Source, it still takes effort to maintain the builds - and I can't afford to spend all by time working on supporting new hardware that I don't get paid for. As a result, builds generally only get put up if:

• They're for an official Espruino board
  
• They're for a board where the owners pay me to keep builds running
  
• There's a lot of community involvement/support - eg. ESP8266/ESP32
  
• They were there for historical or educational reasons - eg. Microbit
  

BTW Interesting info about flash endurance for both nrf51 and 52 is here https://devzone.nordicsemi.com/f/nordic-q-a/15798/flash-endurance-on-nrf52 it is 'only' 20/10K erase cycles, not 100K as I hoped. I like the idea there about rewriting previous data with zeroes (which comes free, it is the erase that counts) and finding first nonzero data.
EDIT: however when reading more, there is nWRITE parameter which is limit of number of writes into the same 1024 byte block which is 181 so looks like you are even not supposed to write 256 32bit values one by one.
EDIT2:look like this differs significantly between nrf51 and 52. nrf51 family has this limit 2 per 32bit word, nrf52832 has this 181 per 1024 block. So the 'clearing invalid data to zeroes' way makes sense only on nrf51.

I like the idea there about rewriting previous data with zeroes

That's basically what Storage does for you :) Not everything, but it sets a flag on a block of data to say 'this one's invalid'.

Interesting about the write endurance stats though.

Interesting... Thank you guys for your clarification. I'm trying to come to a conclusion if it is actually rational to use internal nrf5 flash for this king of things I wanted to use it for, e.g. backing up some readings (daily or hourly).

Giving that:

• we have only 10k write/erase cycles
  
• I've managed to allocate only 14 pages
  
• there is a wear leveling mechanism so that all 14 pages are evenly used even if I write 1 word
  
  

Does it mean that I'll have about 10000 * (14 * 1024 (page size)) / 4 (bytes) = 35840000 write cycles for 1 word?

Something tells me that it is not right or very hard to achieve giving that wear leveling won't be ideal. Also, I'm not quite sure if my assumption is correct - 10000 write/erase cycles is per 1 bit (e.g. not per page or minimum write operation length).

If 10000 write/erase cycles is actually per 1 page (e.g. minimum write operation length is 1024 bytes), then it would be a different figure: 10000 * 14 = 140000 write cycles for 1 word.

I also not quite sure if flash page size on low level is the same as page size of Espruino Flash module. Looks like ppl on that nordicsemi thread mention that page size is 512.

What's your thoughts on that guys? If it is 35m write/erase cycles, then it is acceptable even if we store a figure every hour. If it is only 140000 write cycles, then it could be problematic.

Mon 2019.04.15

Back in the eighties, when we had the luxury of doubling our NOVRAM capacity to a whopping 1024 bytes, we made sure we wrote an entire page to maximize the number of bytes that would get written in one cycle. For a 4 page 512 byte chip, four arrays of 32 bytes each made a page, so we would fill those over time in RAM, then write that page P0. On the second write, we advanced to the next page P1, and the third and fourth the same way P2 & P3. On the fifth page write, we started over, erasing the first page written P0. In this way, we used one cycle per write for a 128 byte page rather than 128 cycles for each of the individual 128 bytes.


The Manufacturer Guarantee

The rough rule of thumb was one page write each hour for one year, roughly ten thousand.

365 days in a year
24 hours in a day
8760 hours in a year

Therefore, for a ten hour work day, a three year life expectancy.

Moore's law, doubling capacity while halving price every eighteen months, allowed obsolescence to force an upgrade to the hardware, an excuse to then greedily use that extra memory. ;-)

@fanoush

NVMC — Non-volatile memory controller
nWRITE,BLOCK Amount of writes allowed in a block between erase

The 181 value indicates the number of writes before an erase must be performed. One then could write all 256 32 bit words, one by one with an intermediate erase.

@Robin

The 181 value indicates the number of writes before an erase must be performed. One then could write all 256 32 bit words, one by one with an intermediate erase.

Oh I got the numbers wrong. It looks like block is 512 bytes long, not 1024. So I can write it in 128 steps. This now makes sense. So I have 53 spare writes to rewrite some 32bit words second time.

@Vlad
Looks like nrf51822 has 20K writes limit per page and there is no such strange write limit for whole blocks. It says that each individual word can be written two times. In fact when checking documentation only nrf52832 has this. E.g. the 52811 or the newest 52840 has only limit per individual word like nrf51822 .

So as I understand it with nrf51822 you can write each 32bit word twice, then you need to erase whole page and this page erase counts as one from 20K per page.

Thanks everyone.

Just a heads up, I've tried to enable full BLE API support for nrf51822 board (e.g. that are disabled atm by defined(NRF52) and SAVE_ON_FLASH - these are the APIs that allow to create GATT server, in other words to make nrf board to be "central" device), this is not possible with that small flash unfortunately:

STORAGE: 247808 -> 262144
CODE: 110592 -> 256372
CODE AND STORAGE OVERLAP

Effectively nrf51822 boards are forced to be peripheral devices only. That's ok methink as to be "central" is not that important/common for this small devices.

However, I must say I was able to read some data from other BT devices by using advertised services, for example I was able to hook up to a Xiaomi Temperature and Humidity sensor:

NRF.findDevices(function(devices) {
            print("Found");
            if (devices[0] && devices[0].serviceData.fe95) {
                var data = devices[0].serviceData.fe95;
                var flag = data[11];
                print(flag);
                switch (flag) {
                    case 4: {
                        // temp
                        temp = (data[14] | data[15] << 8) / 10;
                        break;
                    }
                    case 6: {
                        // humidity
                        humidity = (data[14] | data[15] << 8) / 10;
                        break;
                    }
                    case 10: {
                        // battery
                        battery = devices[0].serviceData.fe95[14];
                        break;
                    }
                    case 13: {
                        // temp and humidity
                        temp = (data[14] | data[15] << 8) / 10;
                        humidity = (data[16] | data[17] << 8) / 10;
                        break;
                    }
                    default : {
                        print("Unknown flag: " + flag);
                    }
                }
            }
        }, {
            filters: [
                { id: "4c:65:a8:d0:7a:ee public" }
            ],
            timeout: 2000
        }
    );

Perfectly working.

Mon 2019.04.22

Congrats on your success @Vlad !

As you have quite a nicely done snippet block of code there and it doesn't appear that the Espruino site currently has an interface solution for the 'Xiaomi Temperature and Humidity sensor', would it make sense to add that content with a few basic comments and some sample output data in a descriptive unique thread to assist others?

Maybe a post such as 'Accessing data returned from a Xiaomi Temperature and Humidity sensor' beneath:

Home >> Official Espruino Boards >> Interfacing

Having that content in this thread may get lost, and may never become searchable. Having a descriptive title would allow others that use the Xiaomi sensor to quickly find your solution.

                   case 4: {
                        // temp
                        temp = (data[14] | data[15] << 8) / 10;
                        break;
                    }
                    case 6: {
                        // humidity
                        humidity = (data[14] | data[15] << 8) / 10;
                        break;
                    }

You use the same dataset for both temp and humidity, is it correct?

Yes, it is correct. Depending on the flag field (11th element), payload contains temperature, humidity, both temp and hum, battery level.

I've decoded and described Xiaomi protocol in the Gatt Parser project here and here. Most of Xiaomi devices follow this protocol, e.g. MiFlora sensor, Mi Kettle, Mi Scales etc

Thanks @Robin, I've created a post here.