Module for Winbond W25Q SPI flash memory

I have developed a module for interfacing Winbond serial flash (w25Q series). have successfully tested it with a W25Q80BV (8 Mbit) chip. On an Espruino Pico, for many applications this chip makes a great alternative to hooking up an SD card, since the chip is tiny and costs only a few cents.

My module supports erasing the chip, erasing a sector, writing a page, sequential writing and sequential reading. There is no documentation yet, but essential comments in the source code. You should be able to use it if you generally know how flash memory works.

I put the code on github for now. We can add it to the Espruino module library once it reaches a more mature state.

Does the w25q behave differently from the AT25-series? http://www.espruino.com/AT25

Looked at the code in git... there is a lot of wait for ready, practically in every loop, even micro loop down to a byte ( EDIT: misread - see post #6) . I'not familiar enough with the chip and whether it could be done a differently - less frequently - to avoid that back and forth communication. I understand that structuring software interrupt / event driven adds complexity. But could that resolve some of the issues you're having with the waveforms?

@allObjects: are you sure you were looking at the right repository? :) waitReady is not called in any loop. It's only called at the end of writing operations (when flashing a page, when erasing a sector and when erasing the chip). That's it. As long as you only read, waitReady will not ever be called at all. But for writing it's neccessary, because otherwise you might get corrupt data:

Poll to check that erase/program are done. Unlike a read, programming can take tens of microseconds to milliseconds. A chip-level erase can take hundreds of milliseconds. If you execute an instruction before it is done, the erase does not finish! Write your memory drivers to poll the chip until it is done, even if it seems to be more complex and requires more CPU power.
(Source: The Positive Rail - Debugging the Winbond W25Q80BV)

The waitReady function itself is a loop. It could be optimized so that the polling happens in a setInterval function and not the main loop, so it becomes non-blocking and Espruino can do other stuff in the background. But then again, it adds a great deal of complexity, because your application would have to deal with callback functions and asynchronity. To be realistic, erasing the chip isn't something you do all the time, and flashing a page is fast enough to wait for it :)

It's most likely not related to my waveform issues, since in that application I am only reading (once I have flashed my test signal) and there's no ready-polling when reading.

@DrAzzy: I wasn't familiar with the AT25, as I'm still new to the microcontroller world. You shocked me a bit (I thought I may have re-invented a wheel that already existed in the Espruino universe). But to my relief, those seem to be two different wheels. From what I see in the datasheet and AT25.js, the protocol is simpler, the handling of the CS pin is different, commands are different, and the AT25 has only 6 instructions while the W25Q has over 30.

Also chip sizes are different. Most AT25 are in the kilobyte range and W25Q in the megabyte range.

@Dennis, I misread the code. I was wrong. The wait is only called in the .finish() that concludes a write of bytes. Nice of this implementation is that the chip can be asked and when ready move on. Some implementations justs make sure to be time-wise on the safe side with a kill time.

Did you know that FRAM/MRAM technology does not a wait or erase? ...and it is fast: 40MHz clock rate. I 'glued' such a chip on my classical Espruino 1.3 board. The chip is pin AND command compatible with the standard EEPROM chips... even though the commands are not needed. Large capacities are though not in hacker standard package and therefore not as prevalent as EEPROMS. FRAM/MRAM is more like a non-volatile RAM that does not backup power.

@allObjects: I also made sure to allow continuous sequential operations (as opposed to bulk operations).

Yes, I read about FRAM and I even have one lying around that I ordered from Adafruit (32 KByte). Pretty cool technology and definately much easier to integrate. But they are so much more expensive that they're not an alternative for most projects requiring more than a few kilobytes. Low quality audio would set me back 3.50$ per second.

Wow, that's pretty good value for money memory... Thanks for posting the module up - the streaming mode you suggest looks great for handling waveforms. If you can tweak the read function such that it takes an argument for the number of bytes to read, it might well work with .pipe on http/Serial and similar as well.

<3

I notice this module still hasn't been added to the official repo of modules.
https://github.com/espruino/EspruinoDocs/tree/master/devices

Is the module in a state where it would be ready for that? If so, we should get it in so others can start using it.

Yes, it'd be a really good idea to get it in - this totally slipped off my radar.

@Dennis, what do you think?

IMO it'd be nice if read and write worked a bit like the AT24/AT25/DS2xxx modules though:

flash.read(address,bytes)
flash.write(address,data)

Then at least users could choose different modules while using the same code - although obviously the eraseChip/erasePage stuff still needs to be exposed.

I'd like to expose Espruino's internal flash memory somehow too, and it'd be nice to use the same API for that. As it's got variable size pages, I wonder whether something like this might work:

flash.getPage(address) -> { start:address, end:address, size:int }
flash.erasePage(address);

In your case I guess:

// assuming you can only erase blocks of 16 pages at once?

Flash.prototype.getPage = function(address) {
  var pageSize = 256*16;
  return { start: address & ~(pageSize-1), size: pageSize }
}

Flash.prototype.erase16Pages = function(address) {
  var pageNumber = address >> 12; // 256*16
  // ...
}

That way it's relatively easy for code to see which page it needs to erase, and the same API should be transferrable to other flash memory.

I've been absent from the Espruino world for some time as my baby daughter was hospitalized with a dangerous virus. She'll be alright.

My Flash module hasn't changed much, but it's working. Only proper documentation is missing. Developers who want to use the module can already do so by simply copying the file to the modules folder of Espruino's project management folder. It can then be used by a simple require call and the WebIDE will fetch it from there.

@Gordon, your assumption is correct: it's only possible to erase a whole sector of 16 pages at once, but not a single page. Erasing simply resets all bits to 1. Writing data can set bits to 0 but not the other way round, and you can start sequential writing anywhere within a page. The data is kept in a buffer and only actually written when the operation is finished by calling finish. When crossing a page boundary, you must call finish and start a new write operation.

I see the benefits of making the module compatible with others. But I don't like those functions that read and write a whole chunk. They could be added for the sake of compatibility, but I would like to also keep the more fine-grain functions for sequential reading/writing (and maybe add those to the other modules if they don't exist). In many cases that's more efficient than allocating a buffer, filling it and then shoveling data from the buffer to the flash in another loop.

The code above looks wrong. I think it should look like this (not tested):

Flash.prototype.getPage = function(address) {
  return {start: address >> 8, size: 256};
}

Flash.prototype.erase16Pages = function(address) {
  var pageNumber = (address >> 8) & 0xFFFFFFF0;
  // ...
}

who want to use the module can already do so by simply copying the file to the modules folder

Actually just require("https://raw.githubusercontent.com/pastaclub/espruino-w25q/master/W25Q.js") should work too :)

Ahh - the page + erase thing makes life a bit more painful. If you could only erase a whole sector at once I was effectively treating that as the 'page size', but the requirement to call finish after each page causes problems.

The issue is the execution speed of Espruino - repeatedly calling write for each byte will be painfully slow - but being able to shovel buffers worth of data around tends to be a lot faster if it's possible...

Why would that be? Let's say we want to write a whole page and set every byte to its relative address. The bulk write solution would allocate an array buffer of 256 bytes, iterate over the buffer in a loop to set the data and then call the bulk write function. That function would iterate over the buffer again and write it to SPI. By contrast, with fine-grain functions, you would not even allocate a buffer and have only one loop which writes directly to SPI. I don't see why this should be slower? The user can even directly call spi.send and only use the module to initiate and finish the write operation.

By the way, an improvement I can think of would be to replace this code:

Flash.prototype.send = function(data) {
  // sends data and returns result
  return this.spi.send(data);
};

by this code in the constructor:

function Flash(spi, csPin) {
  this.write = this.spi.send;
  ...

Treating a whole sector as a page is an expensive workaround, because it reduces the number of pages by factor 16. After all, once you have erased a sector, you have 16 empty pages and you don't have to write them all at once. You can even write single bytes to anywhere you like. The only thing you cannot do is overwrite data in a location that was not previously erased (but "previously" does not have to mean "just before the write"). Thus, treating a whole sector as a page creates many limitations. A nicer approach would be smarter code which - when data within a page needs to be overwritten - copies the page to a free page. When free pages become scarce, it could relocate pages to free up sectors and erase them. But that functionality would essentially be a file system. It would be possible to implement such a file system and abstract all the complexity away from the user so that the flash behaves like a RAM... but then it would be very inefficient, so I assume that for typical Espruino applications the user should keep in mind it's a flash and deal with that complexity. If that's not an option, they should be using a non-volatile RAM instead.

That function would iterate over the buffer again and write it to SPI.

That's the issue here. I notice your code effectively does this:

 for (var i=0; i<arrayBuffer.length; i++) this.spi.write(arrayBuffer[i]);

but the send/write functions can take arrays, so you can do this:

this.spi.write(arrayBuffer);

All the iteration is then done in native code, so it's very fast. If you're generating the data byte by byte it's less of an issue, but let's say you got it from Waveform, or it came in via serial/USB - you actually have no need to access the individual bytes from JS, you can just pass the string or arraybuffer around which is much faster.

I intended an API that let users:

• Read a chunk of data from anywhere - ideally there'd be a way of just streaming data out as well. From that POV flash.read(bytes, address) makes more sense as Address could then be undefined when streaming data out.
  
• Find the sizes and addresses of blocks of memory that can only be erased in one go - sectors in this case - maybe the choice of name getPagewas confusing.
  
• Write a chunk of data absolutely anywhere - the function would handle pages, so the user would not have to care about when to call finish. That could probably have bytes/address arguments swapped as well, to allow streaming more easily.
  
  

As I understand it, nothing actually stops you writing the same page twice without erasing - just as long as you don't write 0s over bytes that were already 0?

A nicer approach would be smarter code which - when data within a page needs to be overwritten - copies the page to a free page

Yes, I've been planning on a simple 'fake EEPROM' module that does something like that. Ideally it would sit on top of the Flash module, using an API that was the same for all different flash devices - hence having functions like erase16Pages aren't a huge help, as the STM32's flash definitely doesn't have the 16 page limitation.

the send/write functions can take arrays

Ah okay, that makes sense. So we shall have

• a function to read a chunk
  
• a function to write a chunk
  
• a function to initiate a sequential reading operation
  
• a function to initiate a sequential writing operation
  
• a function to sequentially read / write a byte (optional, since it's just an SPI call)
  
• a function to finish a sequential operation
  
• a function for erasing (less than 16 pages is not possible with the Winbond flash)
  
  

What should the names and signatures of those be?

As I understand it, nothing actually stops you writing the same page twice without erasing - just as long as you don't write 0s over bytes that were already 0?

Correct! I tested this and it works fine. You can see it as writing a part of a page.
(but it should read: "as long as you don't write ones over bits that were already zero")

I started using this module for my project-works great. Initially I called the read function in a loop to read a page, then I found out that you can call the seek function once and consecutive reads will advance automatically. Much faster!

W25Q.prototype.readPage = function (page) {
  var x = new Uint8Array(256);
  this.seek(page, 0);
  for (i = 0; i < 256; i++) {
    x[i] = this.spi.send(0);
  }
  return x;
}

Thank you for pointing this out. I don't usually read big chunks, so I never wondered about this. I committed this to the repository for people who may find it useful.

It seems that function calls are very inefficient (otherwise this wouldn't be much faster than calling the read function in a loop). Do you know any way in Espruino's JavaScript to make it an inline call like it's possible in C?

Another idea I had was changing the read method like this:

Flash.prototype.read = this.spi.send;

i.e. making the read property point directly to the property of the spi class. This seems a bit unclean but it may work and actually be faster. I don't have that chip right now, but maybe you can test.

Looks like chip is not available anymore ?

It seems like quite a few of these (maybe not the 8MBit) are still in production? http://www.winbond.com/hq/product/code-storage-flash-memory/serial-nor-flash/?__locale=en

It's not specifically calling functions, it's just that JS execution speed as a whole isn't that fast.

Realistically you want to ask Espruino to do the work for you if at all possible.

Changing readPage to something like:

Flash.prototype.readPage = function(pageNumber) {
  this.seek(pageNumber, 0);
  return this.spi.send({data: 0, count:256});
}

Should be tidier, and an awful lot faster. writePage could be speeded up in a similar way as well I think.

Is there any interest in getting this brought onto the Espruino website as a module so you can require(W25Q)?

Definitely a lot faster this way:
old: 8028 ms
new: 1543 ms
https://github.com/atmelino/MPPT/blob/master/version2_0/Espruino/develop/extFlash/projects/noyito_W25Q64.js

W25Q chips are widely available, and the instructions for the W25Q chips are pretty much the same
If you scroll to the bottom of this page:
https://github.com/Marzogh/SPIMemory/releases/tag/v3.3.0
inside the folder
SPIMemory-3.3.0/SPIMemory-3.3.0/extras
there is a spreadsheet
Winbond Flash Instructions - Comparison.xlsx
with a comparison of the instructions.

I'll be happy to bring the W25Q module onto the Espruino web site (credit belongs to Dennis) when the project works (lots of changes at the moment) and then I can also add a INA3221 module (triple I2C voltage and current sensor) and a DS1307 module. If you need the modules more quickly, you can find them under the first link :-)

I have been absent from Espruino for a while, but these chips are still around. Just recently I was looking for a cheap solution to hook up a non-volatile memory in the megabyte range to a circuit and mouser quickly brought these up again.

Should I add people to the github repository so you can make the neccessary changes?

Thanks - up to you really...

Maybe @atmelino could issue a PR for your repo, and also the EspruinoDocs one - just to keep everything in sync? I think once the module is on the main Espruino site, that'll probably end up being the main place any future contributions get pushed (first) though.

ok, so I created a fork, put in my changes, and created a pull request, according to the tutorial at
https://www.youtube.com/watch?v=e3bjQX9jIBk

Great! Seems like it all went well and just got merged into https://github.com/pastaclub/espruino-w25q - do you want me to bring that into EspruinoDocs now?