Running code off of an EEPROM

Say you want to save memory storing functions (particularly verbose ones). I plan to do this for answering HTTP requests, since that's something that can involve a considerable amount of difficult-to-shrink code (the strings that need to be returned, mostly), yet is something that likely is not used all that often, so it can afford to drag a little bit.

Let's start by adding this function to our eeprom object:

I2C1.setup({scl:B8,sda:B9});
rom=require("AT24").connect(I2C1,128,512);

rom.r = function (id) {
	var x=this.read(1024+id*4,4);
	if (x[2]!=255) {return eval(this.reads((x[1]+(x[0]<<8)),(x[3]+(x[2]<<8))));}
};

Here, starting at 0x0400, we'll start storing a table of indices for functions stored on the eeprom, 4 bytes per (2 each for starting address, and length). We'll need some helper functions to load the EEPROM, and see what's on it. Setting all 4 bytes of the index entry for a function marks that function ID as unused, though we look at the third byte, because if that's 255, the result is unambiguously invalid.

maxid=255;
ftst=1024;

//getList() returns an object containing one property for each function listed in the function index on the rom, and also prints out it's progress in human readable format to assist the operator in loading the rom.

function getList() {
	var count=0;
	var map={};
	for (var i=0;i<maxid;i++) {
		var b=rom.read(ftst+i*4,4);
		if (b[2]!=255) { //if b[2]==255, length ~= 64K, which is clearly not valid data. 
			count++;
			var a=b[1]+(b[0]<<8);
			var l=b[3]+(b[2]<<8);
			map[i]=[a,l];
			console.log(i+". 0x"+a.toString(16)+" "+l+" bytes."); 
		}
	}
	console.log("Scan complete "+ count+ " functions in index"); 
	return map;
}

//getFunction(id) will return a string containing the code for that function, assuming it exists. 

function getFunction(id) {
	var x=rom.read(ftst+id*4,4);
	if (x[2]!=255) {return rom.reads((x[1]+(x[0]<<8)),(x[3]+(x[2]<<8)));}
}

//isSafe(address, length, map) takes a 'map' object (as returned by getList()), and returns 1 if a function of specified length can be placed in the specified address without overwriting something. 
//If it fails, print the ID of the function that it's got a problem with. 

function isSafe(adr,len,map) {
	var max=len+adr;
	for (var i=0;i<maxid;i++) {
		if (map[i]!=undefined) {
			var a=map[i][0];
			var l=map[i][1];
			if (((a < adr)&&(a+l > adr))||((a > adr)&&(a < max))) {
				console.log("Conflict on function "+i);
				return 0;
			}
		}
	}
	return 1;
}

//addFunction(id, address, function) - This creates a new entry for a function of 'id', located at 'address' in the function index, and writes that and the function (supplied as a string) to the rom, provided that that can be done without overwriting another function.


function addFunction(id,adr,str) {
	console.log("Adding function of length "+str.length+" at address "+adr+" with ID: "+id);
	if (isSafe(adr,str.length,getList())) {
		rom.writel(adr,str);
		var tarr=new Uint8Array(4);
		tarr[0]=(adr>>8);
		tarr[1]=(adr&0xFF);
		tarr[2]=(str.length>>8);
		tarr[3]=(str.length&0xFF);
		rom.writeb(ftst+4*id,tarr);
	} else {
		console.log("Selected location would overlap with other function!");
	}
}

//deleteFunction(id) deletes the function with that ID from the function index. 

function deleteFunction(id) {
	console.log("deleting function: " + id);
	rom.writes(ftst+4*id,"\xFF\xFF\xFF\xFF");
}

//cleanup - remove all the stuff related to this. 
function cleanup() {
  delete getFunction;
  delete isSafe;
  delete deleteFunction;
  delete getList;
  delete addFunction;
  delete maxid;
  delete getFunction;
  delete ftst;
  delete cleanup;
}

And of course, the first function we add can be this one:

addFunction(0,2048,'ftst=1024;maxid=255;function getList(){for(var a=0,d={},c=0;c<maxid;c++){var b=rom.read(ftst+4*c,4);if(255!=b[2]){a++;var e=b[1]+(b[0]<<8),b=b[3]+(b[2]<<8);d[c]=[e,b];console.log(c+". 0x"+e.toString(16)+" "+b+" bytes.")}}console.log(a+" functions in index");return d}function getFunction(a){a=rom.read(ftst+4*a,4);if (a[2]!=255) {return rom.reads(a[1]+(a[0]<<8),a[3]+(a[2]<<8))}function isSafe(a,d,c){d+=a;for(var b=0;b<maxid;b++)if(void 0!=c[b]){var e=c[b][0],f=c[b][1];if(e<a&&e+f>a||e>a&&e<d)return console.log("Conflict on function "+b),0}return 1}function addFunction(a,d,c){console.log("Add fun of length "+c.length+" @  "+d.toString(16)+" ID: "+a);if(isSafe(d,c.length,getList())){rom.writel(d,c);var b=new Uint8Array(4);b[0]=d>>8;b[1]=d&255;b[2]=c.length>>8;b[3]=c.length&255;rom.writeb(ftst+4*a,b)}else console.log("Not enough space there!")}function deleteFunction(a){console.log("deleting function: "+a);rom.writes(ftst+4*a,"\xff\xff\xff\xff")} function cleanup() {delete getFunction;delete isSafe;delete deleteFunction;delete getList;delete addFunction;delete maxid;delete getFunction;delete ftst;delete cleanup;}');

Length is like 1160 bytes.

Now, you may be wondering what the point of the cleanup() function is...

If I have a microcontroller with saved code on it which I use, I will often do something like this for onInit()

function onInit() {
pinMode(C5,'input_pullup'); //this is connected to a button to ground. 
if(!digitalRead(C5)){return;}
...
...
delete onInit();
}

So I can enter maintenance mode by holding down that button on reboot, and in that mode, I can connect to the console, load the utility functions from the eeprom and modify the functions in it, and then call cleanup(), and then if I need to modify the normal code too, I can call save() and there won't be any cruft from the eeprom maintenance to clog up the saved code.

Inspires me for something: put like the Apache Documents folder (hierarchy) onto an SD cart - static context, templates, (rendering) functions (like taglibs as used in .jsp in the J2EE world), and what ever is needed to serve an http request(ed html page) - then add some execution container onto Espruino, which able to load it dynamically and temporarily from the SD card and create the html response. To start simple, adding a templating framework - such as the one from _ (underscore.js) would go quite far for that. In _ the templates are actually compiled into anonymous javascript functions... (currently dealing with backbone, underscore, jQuery, and... - and got to like underscore a lot... it is quite a cool js lib for doing a lot of things... of course, also things in a browser context... but that part could be dropped...).

;-)

That's really neat!

Just had an idea (it's pretty nasty), but if you created your own reset() function and also created your own require and Modules.addCached (which was basically the same as your addFunction) then you could probably use the Web IDE to automatically load all the modules straight into the eeprom.

I'm going to have to implement some support for the DS2431 and similar EEPROMs. While they're not that big it'd be amazingly handy to be able to solder on a simple 2 or 3 wire device and get some nonvolatile storage. Because they don't draw much power you could just wire them between any available IO as well.

Oooo! Neat! I didn't know those existed - you know how I am about eeproms!

I'll order some of those one-wire EEPROMs tonight and see if I can get a module written up :-)
Any thoughts about what the right part for me to target is?

Thanks!

Just did a quick look and if you check farnell for 3 wire through-hole EEPROMS you get:

• DS2431+ - 1Kbit, £1.55
  
• DS24B33+ - 4Kbit, £2.26
  
• DS2430A+ - 256 bit, £2.58
  
  

So I'd say it's close between the first two. This page implies that the DS2433 it may be EOL, but I think the DS24B33 isn't.

Hopefully they're very similar in terms of protocol though.

Microchip do some really nice, really cheap ones, but while there are supposed to be through-hole versions available, you don't seem to be able to buy any of them :(

It'd be a very handy little add-on board though.

Ordered a DS24B33 and DS28EC20 (20kbit). The cost per bit is...

$3 for 4 kbit?
In I2C, $3 will get you 512 kbit eeproms.
And in SD cards, $3 will get you 33,554,432 kbits (ie, 4 GB)

@Gordon, when creating this support, could you make it FRAM compatible? FRAMs do not need wait state(s) as EEPROMs do. See 256-Kbit (32 K × 8) Serial (SPI) F-RAM - Ferroelecric RAM as an alternative to EEPROM.

I thought you'd already made a module for the FRAM?

As I understood it, he's just talking about adding a module for it, which I've volunteered to do for the one-wire ones.

Do you have the beginnings of an FRAM module, at least? I'd be happy to reformat it as a module, but I don't have any of the chips on hand to test with. I'd like to make all the EEPROM modules behave as close to identically as possible for the user.

@DrAzzy, yes, but only on a very low level: read and write bytes at a memory location. Basically, the serial FRAM/MRAM behaves like the serial EEPROM - except there are no wait states for write. No wait states for writes and unlimited write cycles would open up new application files: some memory segments in the whole memory space could be FRAM/MRAM and would serve as persistency with the simple option of having or moving data there.

For my own use I have started writing an (string) object store with garbage collect, but no file system yet. As I understand your intention is to have a file system with files on a (what ever) device that you can access serially. Correct?

Oh, no, I'm not trying to make a file system or anything - this is just a system to store snippets of code on eeprom, and run them from there (as a way of saving precious espruino memory)

The modules I'm referring to are the AT24 and AT25 modules, which provide the basics of interacting with those - read as bytes or string, write as bytes or string. Counting the FRAM modules, we've got 4 options for chips that do essentially the same thing (byte-addressed non-volatile storage). I figure the modules for these all ought to behave the same way, to minimize confusion.

FRAM's lack of delay on writing, and unlimited write cycles do open up some new possibilities, although it's hard to wear out EEPROMs in realistic lengths of time, unless you're writing almost constantly - 100k writes is still a lot of writes...

What chips are you using, anyway?

FRAM

As @DrAzzy says, that'd be another module. The reason I want to support these EEPROMs is that (while expensive) they're fantastically simple to wire up - needing only 2 wires connected to any 2 pins.

I'd like to make all the EEPROM modules behave as close to identically

Yes, that'd be great. Ideally just read/write/getMemorySize - I guess it'd be better if we could stick with strings or arrays rather than having to implement both every time? I guess strings are probably easiest from the point of view of executing code?

I guess if I made E.asString and E.asArrayBuffer it would really help matters?

At some point in the future I could make a 'fake eeprom' that did the same via inbuilt flash memory too.

Perhaps then the filesystem you'd posted up earlier could be a module that worked on top of any EEPROM?

I understand... so we go after exactly the same problem... ;-)...

The details about the FRAM chip and its feature are described at the bottom of the very first post of the thread.

My problem raised from the fact that Espruino has quite limited memory. In my case I wanted to write a calibration / recalibration function for the resistive touch screen. Calibration is done by placing graphical X's in - for example 20 x 20 px - squares in all four corners and the center of the display and have the user to tap them. Based on the read resistances and the given screen size the, the mapping x/y-resistances of the touch screen to the x/y-coordinates of the display is adjusted. Before writing the X markers, the current display buffer content has to be saved and after using the X markers restored. On a ILI9341 controller - with 262K colors - each pixel uses 16 bits: 5 Red, 6 Green, 5 Blue. For 5 markers of 20 x 20 px square, 5 * 20 * 20 * 2 = 4kBytes - at least - have to be saved and restored (see post of Resistive Touchscreen directly (no touch controller) thread). It is 4kB when hving packed the color bits into tow bytes with shift/unshift/mask operations for the storing and using. Using one byte for each color makes the processing simpler, but the amount of data grows by 50% to 6kBytes.

In one of my code versions I show one calibration X and square after the other to reduce the amount of bytes to 400 respective 600 bytes at a time. I didn't want to put aside 600 bytes just in case for a user wants to recalibrate the touch screen, because some of my apps in which I planned to use the touch screen, have already run out of memory without that UI component imbedded... so I was thinking about a temporary storage. Of course, there are also savings in these apps by breaking them up into modules and load them minified. Btw, this approach of creating modules is helpful regarding memory consumption reduction in all apps, with the downside of running out of Google closure compiler services, which are a part of the IDE's uploading to the board (see too many compiles performed recently. Try again later).

I knew about serial EEPROMs, but knew also about the EEPOM limitations and cumbersome page writing. This made me look for some serial RAM, since I did not need the persistency aspect. Just by accident I stumbled on FRAM/MRAM technology. Even though - as just said - I did not need persistency, I liked it, because it would give me both: fast unlimited, simple RAM like read and write, and at the same time persistence, for - for example - storing any dynamic configuration... So I got myself 2 chips... of which one I fried after I just got it working :(((8888 . So I unsoldered the 1st one and soldered the 2nd one and rewired it and it working ever since.

I was asking for compatibility, because the the FRAM commands include EEPROM unique commands for the possibility to replace EEPROMs w/ FRAMs without having to change the driver software. This must have been a very early though, because EEPROM's capacity and capacity by price left FRAMs/MRAMs technology in the dark - especially FRAM. For MRAM it looks a bit better, but I could only find large capacity chips with Ball packaging... even though http://www.everspin.com has also 256Kb SMT chips - to mention the smallest. Btw, there are various parallel interfaces to these chips as well as the SPI interface... with a serial speed up to impressive 40MHz - no wait states in any operation.

With a FRAM chip working, I began the development of a memory manager - the (string) object store - which allows storing and retrieving strings in the serial FRAM. To not get lost in the details right from the beginning and slowed down by always pushing to the board, I developed a code reference model in the browser in javascript with visualization... which I attach to this post and you can play with in a browser. It is a single html file with embedded javascript. The nice thing is that the memory matrix and its content is displayed with a color scheme to see what is going on. In the code, the bare metal is to the left, and the visualization code is to the right about half a page indented. For now I have some happy paths, nothing optimized yet, and no error messaging... My plan is to move that later onto Espruino with the low level things into assembler/compiled functions.

Basically, the memory has 3 segments:

1. descriptor for size and state
   
2. object storage space, growing from the bottom towards the top
   
3. object pointer space, growing from the top down towards the bottom
   
   

When segment 2 and 3 touch each other - in other words - memory rans out of new space on attempt of storing an object, a garbage collect of segment 2 happens with the hope to reclaim not anymore used space and complete the store operation. If after garbage collection there is still not enough space, the storing process ran definitely out of memory.

Usage example of the high-level browser app as implemented in the attached html file:

Step 1. After loading you see a - (shot) S01:
It shows a memory of 256 bytes / cells in 16 cols / 16 rows. The first 10 bytes are 5 double byte values are the memory descriptor, which describes the configuration and current usage of the memory.

@ 0000: 000A - length of the descriptor in bytes
@ 0002: 0100 - size of the memory (addr of last byte + 1) and begin of pointer heap
@ 0004: 000A - addr of first free byte for object storage / object data heap
@ 0006: 0100 - addr of last used object pointer / object pointer heap and object id
@ 0008: 0000 - addr of last freed object pointer (begin of chain of freed object pointers)

Step 2. After storing (writing) the first string - ABCD - to the memory - S02:
Enter a string ABCD in the Value field and press write button.
The store operation returns the object's id 00FE in the memory in ID, with which it can be retrieved, updated, and deleted.
(Check of space to trigger already implemented garbage collection and out of memory detection not yet implemented. They are the next thing).

@ 0004: 0013 - addr of the 1st new free byte for object storage
@ 0006: 00FE - addr of object pointer and object id as returned by the write operation

@ 000A..0012 - stored object containing the string ABCD and admin information
@ 000A: 00 - 1 byte object type: 00 = String (for now only strings are handled)
@ 000B..000C: 0006 - 2 bytes length of the String + 2 (includes the 2 bytes of objecte pointer, see 000D..000E)
@ 000D..000E: 00FE - addr of object pointer and object id (used for sweep in garbage collect, can be used for validation of obj addr on access with 1/65536 error probability)
@ 000F..0012: A B C D - the actual object value / string
@ 00FE: 000A: addr of object data at object pointer address / object id

Step 3. After storing (writing) a second string - efghijklmn - to the memory - S03:
Enter a string efghijlkmn in the Value field and press write button.
The store operation returns the object's id 00FC in the memory in ID, with which it can be retrieved, updated, and deleted.

@ 0004: 0022 - addr of the 1st new free byte for object storage
@ 0006: 00FC - addr of object pointer and object id as returned by the write operation

Attachements:

1. framAz.html - initial setup of the memory manager
   
2. mm.html - most recent version of memory manager with one shot and step-wise regression test (just click and run in browser)
   
3. series of screen shots
   

5 Attachments

• framAz.html
• mm.html
• 
• 
• 

Step 4. After storing (writing) a third string - 1234567890 - to the memory - S04:
Enter a string 1234567890 in the Value field and press write button.
The store operation returns the object's id 00FA in the memory in ID, with which it can be retrieved, updated, and deleted.

Step 5. After retrieving (read) of the string with id - 00FC - ('efghijklmn') from memory - S05:
Enter the object ID 00FC in the ID field and press read button.
The read operation returns the (string) object value efghijklmn with the ID from memory.
(Currently no crosscheck takes place wether the obj ID is in the 3rd segment of the memory, is even, and matches with the id in the object data space.)

Step 6. After deleting (delete) of object with id - 00FC - ('efghijklmn') from memory - S06:
Enter the object ID 00FC in the ID field and press delete button.
The delete operation returns the (string) object value efghijklmn with the ID as deleted from memory.
(Currently no crosscheck takes place wether the obj ID is in the 3rd segment of the memory, is even, and matches with the id in the object data space.)

@ 0008: 00FC - addr of last freed object pointer / object ID (begin of chain of freed object pointers)
@ 00FC: 0000 - addr of next (previously) freed object pointer / object ID (0000 = none, end of chain, last one in chain)
On new writes, last freed object ID are reused until all are gone.

@ 0013..0021 - stored object still there, but type is overwritten with 80 to indicate deleted object
@ 0013: 80 - 1 byte in object data space indicating deleted object / garbage collectable.

3 Attachments

**Step 7. After updating with (string) value of 678 of obj with id 00FA - S07:**
Enter a string *678 in Value field and object id 00FA in the ID filed and press update button.
The update can update the object in place since it is shorter and leaves enough of unused in-place space for putting the admin info there.
No change in the object pointers, only length and object data, and not anymore used fraction. Same sized objects are also updated in place.

@ 0022..0029 - updated object
@ 0022: 00 - (string) object type - UNCHANGED
@ 0023..0024: 05 - new length information
@ 0025..0026: 00FA - object ptr / id - UNCHANGED
@ 0027..0029: 6 7 8 - updated object value

@ 002A..0030 - 'new' unused object as 'built' from the not anymore used in place space
@ 002A: 80 - indicator of a new unused object (space)
@ 002B..002C: 0004 - length of unused object (space) - Note: no object id / ptr there
@ 003D..0030: 4 bytes from the old object data - UNCHANGED

@ 00FA..00FB: 0022 - object data addr at obj ptr / id - UNCHANGED

**Step 8. After updating with (string) value of 1234 of obj with id 00FA - S08:**
Enter a string *1234 in Value field and object id 00FA in the ID filed and press update button.
The update cannot update the object in place anymore since it longer. The old object space is marked as unused,
a new space is claimed, and the object data addr at the object pointer / id is updated.

@ 0031..0039 - updated object
@ 0022..0029 - old object value marked as deleted / garbage collectible

@ 00FA..00FB: 0031 - new object data addr at obj ptr / id

Step 9. After garbage collection - S09:
Press gc button

All unused object space are purge by moving remaining active object next to each other, and the object data pointers are updated.

Another needed feature I did not implement yet is the data type 01: *keyed* or named (string) objects. Because after a reset, the application has to be able to retrieve object ids in order to re-access them. This feature can be implemented in a first step by storing an array of object ids in a (string) object as the very first object, because the id of this TOC (table of content) (string) object's id is always size of the memory - 2.

So much for now. Some more doc will follow / updated.

3 Attachments

@DrAzzy - sorry, I got the parts and couldn't wait: http://forum.espruino.com/conversations/260971/

I'd be interested to see if it works for you?

@allObjects - this is what I meant... So ideally you'd have two parts: One that accesses the EEPROM/FRAM/MRAM/etc, and another that handles storage and memory management of objects - so that for instance you could use your memory manager on @DrAzzy's EEPROM.

Hopefully when we've got a few memory drivers that work the same way, we can maybe pull your code into a module that will work with them?

Exactly: the ability to retrieve previously stored strings and the ability to temporarily store strings. The pubs I made is to give you some head start of the logic... I wanted to explore the techniques/logic in a higher lever environment. An intermediate layer is the management of the memory, and the bottom layer / foundation is the actual device driver. The basic functions are:

1. write(string) returning ID (addr)
   
2. read(addr) returning string
   
3. update(addr,string) returning nothing
   
4. delete(addr) returning deleted
   
5. writeKeyed(key,string) returning ID (addr) --- key is a string
   
6. readKeyed(key) returning ID (addr) [and string]
   
7. [updateKeyed(key,string)]
   
8. [deleteKeyed(key) returning ID (addr) [and string]
   
   

The functions in [square brackets] are just combinations of non-bracketed ones.

Wow, allObjects - that string data store is something that I'd need to study for a while to figure out - it pretty much went over my head on the first read.

So - looking at the datasheet for that FRAM chip, it sounds like it's meant to be a drop-in replacement for the AT25. The same manufacturer also makes one that uses I2C and looks to a drop-in replacement for the AT24. (The FM24V02), and Fujitsu's MB85RC256V looks to use the same pinout and is probably the same way.

Also, there ARE larger FRAM's in the same series - there are FM2xV05 (512), V10 (1mbit) and V20 (2mbit) available in SOIC-8 package, albeit at a truly eyewatering price.

Try using my AT25 module to interface with your FRAM and see if it works (the writes will of course be slow, but if all that's needed is simplifying the write process and removing the delay, that makes converting the module to FRAM trivial. My module will NOT work on the V10 and V2o ones, but support for this in both AT25 and 24 is coming with the eeprom overhaul I mentioned.

For the new eeprom modules (requires recent v72 and the new eeprom modules)

function getList() {
	var count=0;
	var map={};
	for (var i=0;i<maxid;i++) {
		var b=rom.read(ftst+i*4,4);
		if (b[2]!=255) { //if b[2]==255, length ~= 64K, which is clearly not valid data. 
			count++;
			var a=b[1]+(b[0]<<8);
			var l=b[3]+(b[2]<<8);
			map[i]=[a,l];
			console.log(i+". 0x"+a.toString(16)+" "+l+" bytes."); 
		}
	}
	console.log("Scan complete "+ count+ " functions in index"); 
	return map;
}

//getFunction(id) will return a string containing the code for that function, assuming it exists. 

function getFunction(id) {
	var x=rom.read(ftst+id*4,4);
	if (x[2]!=255) {return E.toString(rom.read((x[1]+(x[0]<<8)),(x[3]+(x[2]<<8))));}
}

//isSafe(address, length, map) takes a 'map' object (as returned by getList()), and returns 1 if a function of specified length can be placed in the specified address without overwriting something. 
//If it fails, print the ID of the function that it's got a problem with. 

function isSafe(adr,len,map) {
	var max=len+adr;
	for (var i=0;i<maxid;i++) {
		if (map[i]!=undefined) {
			var a=map[i][0];
			var l=map[i][1];
			if (((a < adr)&&(a+l > adr))||((a > adr)&&(a < max))) {
				console.log("Conflict on function "+i);
				return 0;
			}
		}
	}
	return 1;
}

//addFunction(id, address, function) - This creates a new entry for a function of 'id', located at 'address' in the function index, and writes that and the function (supplied as a string) to the rom, provided that that can be done without overwriting another function.


function addFunction(id,adr,str) {
	console.log("Adding function of length "+str.length+" at address "+adr+" with ID: "+id);
	if (isSafe(adr,str.length,getList())) {
		rom.write(adr,str);
		rom.write(ftst+4*id,E.toString((adr>>8),(adr&0xFF),(str.length>>8),(str.length&0xFF)));
	} else {
		console.log("Selected location would overlap with other function!");
	}
}

//deleteFunction(id) deletes the function with that ID from the function index. 

function deleteFunction(id) {
	console.log("deleting function: " + id);
	rom.write(ftst+4*id,"ÿÿÿÿ");
}

ÿ
//cleanup - remove all the stuff related to this.
function cleanup() {
  delete getFunction;
  delete isSafe;
  delete deleteFunction;
  delete getList;
  delete addFunction;
  delete maxid;
  delete getFunction;
  delete ftst;
  delete cleanup;
}

'maxid=255;ftst=1024;function addFunction(a,d,b){console.log("Adding function of length "+b.length+" at address "+d+" with ID: "+a);isSafe(d,b.length,getList())?(rom.write(d,b),rom.write(ftst+4*a,E.toString(d>>8,d&255,b.length>>8,b.length&255))):console.log("Selected location would overlap with other function!")}function deleteFunction(a){console.log("deleting function: "+a);rom.write(ftst+4*a,"ÿÿÿÿ")}function getList(){for(var a=0,d={},b=0;b<maxid;b++){var c=rom.read(ftst+4*b,4);if(255!=c[2]){a++;var e=c[1]+(c[0]<<8),c=c[3]+(c[2]<<8);d[b]=[e,c];console.log(b+". 0x"+e.toString(16)+" "+c+" bytes.")}}console.log("Scan complete "+a+" functions in index");return d}function getFunction(a){a=rom.read(ftst+4*a,4);if(255!=a[2])return E.toString(rom.read(a[1]+(a[0]<<8),a[3]+(a[2]<<8)))}function isSafe(a,d,b){d+=a;for(var c=0;c<maxid;c++)if(void 0!=b[c]){var e=b[c][0],f=b[c][1];if(e<a&&e+f>a||e>a&&e<d)return console.log("Conflict on function "+c),0}return 1}function cleanup(){delete getFunction;delete isSafe;delete deleteFunction;delete getList;delete addFunction;delete maxid;delete getFunction;delete ftst;delete cleanup};'

... string data store ... went over my head on the first read

No wonder, you were at that time deep down in the details of the HW of EEPROMS. Coming back up into application (close) layers, it will be a breeze for you (and you may tell me where my doc needs to be fixed/enhanced, since I'm not of English mother tongue nor eloquent writer... JS is closer to me than English).

over my head

I'm fairing about the same way when looking on things coming out of your fine 'micro brewery' - on first reads. After taking the time diving into it, I get the exquisite taste. Thanks for your contributions!

the writes will of course be slow

Speed in accessing FRAM/MRAM depends on the needs of the application. To get/set config data and user data, the current speed is good enough, because the amount of data is not that critical - and there is most of the time little visibility to the user - if there is even one - to notice and complain about sluggishness. Speed is required only in graphics because of the amount of data and because visibility. My code to communicate with the FRAM was quite similar to your EEPROM code, even with the size limit because of (simplified) 2 bytes addressing with (singed) int. I can easily see speeding up the memory read/write process by using "compiled"; option, or moving the critical code into assembler all together... especially the part that shows patterns for what DMA was implemented: get a particular amount data at a particular address from one SPI (or other wise connected) device to another one (and back), like in my graphical app where screen buffer has to be saved for an overlay/pop-up - and then restored. For fast handling of those transfers, some chips have the hold pin, which keeps state, while the partial data just read is written to the other device... I was thinking along these lines to have some software DMA module implemented:

// devA & devB setup & connected, such as LCD display, (serial) RAM/EEPROM, etc.
// both implementing same kind of read/write interface
var dmaAB = require("DMA").connect(devA,devB,memBufSize); // w/ req/conn pattern
var DMA = require("DMA"); var dmaAB = new DMA(....); // w/ oo/class req/conn 
dmaAB.xfer(0,addr1,bytes,addr2); // xfers bytes in memBufSize chunks from A to B
dmaAB.xfer(1,addr2,bytes,addr1); // xfers bytes in memBufSize chunks from B to A

I can picture to use DMA on an intermediating Espruino in the data line between just any 'devices': sensors and internet... etc. I can also picture an option where the dma is setup to be trigger by a sourcing device to transfer the data to the sinking one... (...with the classical 'null' device...

Piggy-backing existing (module) code with extending or overwriting is always an option, and if this is not good enough or too cumbersome, I could see different implementations for a particular layer in the existing software stack... assuming it is layered/a stack and not a monolith... ;-). On a micro - on the other hand - a few terse, dedicated modules go further than larger, elaborate multi-purpose components. Therefore, a good and easy way to compose all kinds of needed specific functions and just those form very smaller ones is key, and require("module") is the key enabler for that.

Try using my AT25 module to interface with your FRAM

My FRAM is soldered onto my Espruino board. Will re-jumper-cable it and and do some testing, and also look at my x-code for the string memory. The string memory x-code needs transformation towards the available memory read/write interface with its data converters before I go and move it onto Espruino. I'd like to have also a test-bed/test for the sting memory store. And last but not least, implement an optional key function: store and read back by key rather than physical address (related object id)... even if it is just to get those physical addresses of the objects on init after a (re)boot.

The new AT24/25 modules will fix the slow writes problem on FRAM - just pass 0 as the page size. I just (now) fixed a problem with FRAM writes in the new modules that I introduced last night, they're up on my github now https://github.com/SpenceKonde/EspruinoDocs/tree/master/devices

Don't use the old AT25 modules with your FRAM, write performance will suck :-P

The idea of SPI DMA is interesting - I didn't realize that the STM32 chips supported it until I saw how the TV display code was done. I wonder how much it would buy us in terms of performance? Since I use a B&W LCD at 64x128 px, I don't have as much of a problem getting the data to it.

... I just had an idea (not necessarily a good one) - say you stored the entire blob of data to send to the display in the FRAM. Connect SO of FRAM to SI of display, with tristate buffer in the middle, then you could send the address, open the tristate buffer and assert screen's CS, and then read the command from EEPROM back straight into the display.

@allObjects, your English is excellent, hence your JS must be exquisite!

@DrAzzy, looked at the module and wondered why for each byte the address has to be written - have to say that I have not studied the EEPROM api. For FRAM resetting the address for each byte is not required - see](http://forum.espruino.com/comments/11937103/). Are you referring to that when talking about? Yep, the page size of 0 takes brilliantly care 0f skipping the wait cycles.

connecting SO of FRAM to SI of display...

You just invented Super-Direct Memory Access! I did not think about that... for cases where the memory is equally or larger in size, this really works: preparing the next image/page in the memory and then update the display at once. For my case, the FRAM I have is a bit too small: 32,768 bytes FRAM vs. 172,800 byte display buffer (=230 x 320 pixel * 2bytes in ILI9341... and spending a lake of tears for 6 FRAMS and also giving up 6 CS lines, will become a sea of tears... nope. I need only a few KB to ship back and forth areas of the display. For that, I have to set two different addresses, and also connect SO of display to SI of FRAM. Super Direct DMA for save and restore... really cool idea.

What I already thought of - but not had the time this morning to write about - was to have a DMA Hub... inspired by the 0 and 1 to indicate source and destination in my envisioned A_to_B API. Since the process would anyway not be multi-threaded, I could just add any communicating device to a hub, and then trigger the transfers between the *from-to*s with device#X to device#Y. For streaming devices, the address would be -1 (MSB = 1) or something other detectable (with 2 byte address and MSBit of MSByte a control bit, 32KB is the max addressable... just what my FRAM gives me. Note that not all devices have a single dimension address, even though they may support continuous read and write across dimensions, such as the ILI9341 display controler. This display controller requires to provide row and column address, but does auto increment/reset columns and rows....

In terms of 'software DMA' - there are actually the 'pipes' that you can use for files, that you should be able to apply to SPI - at least for writing. Not that I've tried that though! It could end up being quite interesting.

Hardware DMA would be cool - I'd been meaning to pull the DMA code from the TV stuff out into something more general purpose. I wonder how useful it is for stuff like LCDs though, as you still have the problem of not having enough RAM.

I do have an interesting solution to that - but I'll stick another post up about it,