-
the send/write functions can take arrays
Ah okay, that makes sense. So we shall have
- a function to read a chunk
- a function to write a chunk
- a function to initiate a sequential reading operation
- a function to initiate a sequential writing operation
- a function to sequentially read / write a byte (optional, since it's just an SPI call)
- a function to finish a sequential operation
- a function for erasing (less than 16 pages is not possible with the Winbond flash)
What should the names and signatures of those be?
As I understand it, nothing actually stops you writing the same page twice without erasing - just as long as you don't write 0s over bytes that were already 0?
Correct! I tested this and it works fine. You can see it as writing a part of a page.
(but it should read: "as long as you don't write ones over bits that were already zero") - a function to read a chunk
-
Why would that be? Let's say we want to write a whole page and set every byte to its relative address. The bulk write solution would allocate an array buffer of 256 bytes, iterate over the buffer in a loop to set the data and then call the bulk write function. That function would iterate over the buffer again and write it to SPI. By contrast, with fine-grain functions, you would not even allocate a buffer and have only one loop which writes directly to SPI. I don't see why this should be slower? The user can even directly call spi.send and only use the module to initiate and finish the write operation.
By the way, an improvement I can think of would be to replace this code:
Flash.prototype.send = function(data) { // sends data and returns result return this.spi.send(data); };
by this code in the constructor:
function Flash(spi, csPin) { this.write = this.spi.send; ...
Treating a whole sector as a page is an expensive workaround, because it reduces the number of pages by factor 16. After all, once you have erased a sector, you have 16 empty pages and you don't have to write them all at once. You can even write single bytes to anywhere you like. The only thing you cannot do is overwrite data in a location that was not previously erased (but "previously" does not have to mean "just before the write"). Thus, treating a whole sector as a page creates many limitations. A nicer approach would be smarter code which - when data within a page needs to be overwritten - copies the page to a free page. When free pages become scarce, it could relocate pages to free up sectors and erase them. But that functionality would essentially be a file system. It would be possible to implement such a file system and abstract all the complexity away from the user so that the flash behaves like a RAM... but then it would be very inefficient, so I assume that for typical Espruino applications the user should keep in mind it's a flash and deal with that complexity. If that's not an option, they should be using a non-volatile RAM instead.
-
I've been absent from the Espruino world for some time as my baby daughter was hospitalized with a dangerous virus. She'll be alright.
My Flash module hasn't changed much, but it's working. Only proper documentation is missing. Developers who want to use the module can already do so by simply copying the file to the modules folder of Espruino's project management folder. It can then be used by a simple
require
call and the WebIDE will fetch it from there.@Gordon, your assumption is correct: it's only possible to erase a whole sector of 16 pages at once, but not a single page. Erasing simply resets all bits to 1. Writing data can set bits to 0 but not the other way round, and you can start sequential writing anywhere within a page. The data is kept in a buffer and only actually written when the operation is finished by calling
finish
. When crossing a page boundary, you must callfinish
and start a new write operation.I see the benefits of making the module compatible with others. But I don't like those functions that read and write a whole chunk. They could be added for the sake of compatibility, but I would like to also keep the more fine-grain functions for sequential reading/writing (and maybe add those to the other modules if they don't exist). In many cases that's more efficient than allocating a buffer, filling it and then shoveling data from the buffer to the flash in another loop.
The code above looks wrong. I think it should look like this (not tested):
Flash.prototype.getPage = function(address) { return {start: address >> 8, size: 256}; } Flash.prototype.erase16Pages = function(address) { var pageNumber = (address >> 8) & 0xFFFFFFF0; // ... }
-
-
The existing functions could take additional optional arguments, and they could do the same as before if these arguments are omitted.
I'm not so much in favor of having a multitude of functions doing essentially the same thing in different ways; I find flexible calling patterns more elegant. But it's not that important, and if dedicated functions are more efficient, maybe that's the way to go.
-
I have one Pico which has severe problems with the USB serial interface. Data always gets corrupted both ways. With another Pico, that doesn't happen.
The Pico itself seems to work fine, but it's almost impossible to work with it over USB. Transfering a program with the Web IDE (at 9600 baud) corrupts data in 9 of 10 cases and then the program kind of runs but crashes in different places due to syntax errors, undefined functions and properties etc. When I enable "Throttle send" in the Web IDE, uploading a 100 lines of code takes 15 to 20 seconds but it works most of the time. But then again, subsequent communication through the console gets corrupted in both directions. I'm posting some examples below.
I had originally set up my waveform flash streaming stuff on a breadboard and then decided to put all the components onto a PCB. I took a new Pico for that and left the old one on the breadboard so I could trace the wires which I needed to solder onto the PCB. Unfortunately I didn't test that Pico before, since I did not expect such problems. Since I didn't have enough stackable pinheaders, I soldered the Pico onto the PCB, so now I cannot easily replace it with another one.
What should I do?
_____ _ | __|___ ___ ___ _ _|_|___ ___ | __|_ -| . | _| | | | | . | |_____|___| _|_| |___|_|_|_|___| |_| http://espruino.com 1v79 Copyright 2015 G.Williams >ules.removeAllCached(); =undefined Uncaught Error: Field or method "removeAllCached" does not already exist, and can't create it on undefined at line 1 col 5 ules.removeAllCached(); ^
_____ _ | __|___ ___ ___ _ _|_|___ ___ | __|_ -| . | _| | | | | . | |_____|___| _|_| |___|_|_|_|___| |_| http://espruino.com 1v79 Copyright 2015 G.Williams >echo(0); Uncaught Error: Field or method "finish" does not already exist, and can't create it on undefined at line 3 col 67 ...is.spi.write(data)};ototype.finish=function(){digitalWrite(t... ^
in function called from system Uncaught SyntaxError: Got ':' expected ')' at line 1 col 325 ...t(a[2].substr(d-2))/60)*(-1:1),lon:(parseInt(a[4].substr(0,e... ^
Uncaught Error: Field or method "seek" does not already exist, and can't create it on undefined at line 16 col 8 flash.seek(0, 0); @4Î /aÎ ñÿÿÿline 1 col 6 play(); ^
> play() =undefined > dump() […] function playÎ 4Î tÎ ñÿÿtion(b) { b.set(audioCache);
{ "time": "14:2ì| Î ì|d{ \9 Ð| ñÿÿÿPÏ "satellites": 0, "altitude": NaN } { "time": "14:26:54", "lat": NaN, "lon": NaN, "fix": 0, "satellites": 0, "altitude"Î \{ "time": "14:26:55", "lat": NaN, "lon": NaN, "fix": 0, "satellites": 0, "altitude": NaN }
-
I have bought a couple of these I2C 128x32 OLED displays. They almost work with the SSD1306 module, however every other line is missing (i.e. pixels for y=0 are not displayed, pixels for y=1 appear in what is actually y=0, pixels for y=2 are not displayed, pixels for y=3 are displayed in what is actually y=1 and so on, pixels for y=63 appear in the last line which is actually y=31).
I had a look at the SSD1306 module which has the screen size hard-coded (maybe not the best idea) to 128x64. But if I change it to 128x32 (which would be correct), I get the exact same output, only a few lines further down. It seems that these particular displays need to be interfaced like 128x64 pixel displays but you need to transform the y coordinate of every pixel in the buffer to 2*y+1. What is the best way to patch the modules to achieve this?
-
This is my code:
var pos; var w; function play() { refillBuffer = function(b) { b.set(flash.send(b)); pos += b.length; }; bufferEventHandler = function(b) { if (pos >= len) { // loop sample flash.seek(0, 0); pos = 0; } refillBuffer(b); }; stop(); w = new Waveform(bufSize, {doubleBuffer:true}); pos = len; bufferEventHandler(w.buffer); bufferEventHandler(w.buffer2); w.on("buffer", bufferEventHandler); analogWrite(B1, 0.5, {freq:100000}); w.startOutput(B1, rate, {repeat:true}); } function stop() { if (w && w.running) w.stop(); }
I'm not using the same cheat. Do you think it would make a huge difference? In any case the code must accomplish both (fetch new data and copy it) in the time window between two buffer events, so as long as it's fast enough, the order of those should not matter, and if it's not fast enough, you have a problem anyways.
I just tried 44.1 kHz because it's such a nice omnipresent number (almost 42). I didn't try higher ones, maybe they would work as well. The nice thing is that you could reproduce higher frequencies. For audio (where the 8bit resolution limits quality anyways), this already covers the audible frequency range. Higher sampling rates for audio are mainly relevant in recording (for oversampling or less steep low pass cutoff before the A/D converer) but realtime recording is certainly no fun with this slow-writing flash memory.
What I didn't consider yet is the possibility that I'm mistaken about the 44.1 kHz and I might actually already be getting significant glitches which I simply don't hear because my test signal (a sawtooth frequency sweep) doesn't reveal them. I don't assume so (since it really sounds quite clean) but it might be the case. But then again, for my audio stuff it wouldn't even matter as long as it's not noticeable :) For other applications one should double check.
What's the maximum SPI baud rate for the Pico? The Winbond flash works up to 104 MHz. And by the way, it supports synchronous transfers of up to 4 bit on parallel data pins per 1 SPI controller (but I don't think the microcontroller can handle that).
-
@allObjects: I also made sure to allow continuous sequential operations (as opposed to bulk operations).
Yes, I read about FRAM and I even have one lying around that I ordered from Adafruit (32 KByte). Pretty cool technology and definately much easier to integrate. But they are so much more expensive that they're not an alternative for most projects requiring more than a few kilobytes. Low quality audio would set me back 3.50$ per second.
-
Oh, garbage collection... yes, that could be the reason for some oddities.
@Gordon, one little tweak with (hopefully) a big impact could be to change the
SPI.write
function so that you can pass a target buffer to it and it will write the results into that buffer instead of returning a new one. -
@DrAzzy: I wasn't familiar with the AT25, as I'm still new to the microcontroller world. You shocked me a bit (I thought I may have re-invented a wheel that already existed in the Espruino universe). But to my relief, those seem to be two different wheels. From what I see in the datasheet and AT25.js, the protocol is simpler, the handling of the CS pin is different, commands are different, and the AT25 has only 6 instructions while the W25Q has over 30.
Also chip sizes are different. Most AT25 are in the kilobyte range and W25Q in the megabyte range.
-
@allObjects: are you sure you were looking at the right repository? :) waitReady is not called in any loop. It's only called at the end of writing operations (when flashing a page, when erasing a sector and when erasing the chip). That's it. As long as you only read, waitReady will not ever be called at all. But for writing it's neccessary, because otherwise you might get corrupt data:
Poll to check that erase/program are done. Unlike a read, programming can take tens of microseconds to milliseconds. A chip-level erase can take hundreds of milliseconds. If you execute an instruction before it is done, the erase does not finish! Write your memory drivers to poll the chip until it is done, even if it seems to be more complex and requires more CPU power.
(Source: The Positive Rail - Debugging the Winbond W25Q80BV)The waitReady function itself is a loop. It could be optimized so that the polling happens in a setInterval function and not the main loop, so it becomes non-blocking and Espruino can do other stuff in the background. But then again, it adds a great deal of complexity, because your application would have to deal with callback functions and asynchronity. To be realistic, erasing the chip isn't something you do all the time, and flashing a page is fast enough to wait for it :)
It's most likely not related to my waveform issues, since in that application I am only reading (once I have flashed my test signal) and there's no ready-polling when reading.
-
I got it to work! I posted my findings about waveform streaming here and I also published my module for interfacing Winbond flash memory.
-
For a while I have played around with streaming waveforms for audio output ("streaming" means having two buffers and continuously re-filling one while the other is playing). Here are my findings.
I first streamed from an SD card. Now on the Pico, I stream from an external SPI flash memory, using my home-grown Winbond W25Q module. Comparing the two methods, I can say that streaming from flash works better.
Buffers can be much smaller. For streaming at 11 KHz from the SD, I needed a buffer size of 2048 samples. When streaming from the SPI flash, 64 samples are fine.
While the highest sampling rate I achieved with SD card was between 12 and 15 KHz, I can stream at 44.1 KHz from the flash RAM.
I'm not sure why exactly this is, but I assume that there is a considerable overhead for dealing with the filesystem on the SD card. I also found that reading sequentially from the flash works like a charm, while re-positiong (i.e. moving the read pointer to a different memory address) is much more likely to produce glitches, so that seems to take more time.
It seems that reading from the SD takes quite some fixed amount of time (plus the variable time depending on the chunk size), which would explain why buffers need to be so much bigger.
Another interesting find is that (at least for audio) glitches can be more acceptable with flash memory, because of their pattern (length of the glitch and interval of occurrence). When streaming from SD with big buffers, you get glitches which are typically tens of milliseconds long (so they sound wrong and disturbing), and they happen a few times per second, which is also an annoying interval. When streaming from flash with small buffers, you get very short glitches (sounding like a click) but so often that they are no longer perceived as individual glitches, but as a background hum with a constant frequency. You can tune that frequency (sampling rate divided by buffer size) and you can also tune its amplitude (trade-off against performance). Audio quality at 8bit is not great anyways, and a little crackling well below the volume of the payload signal is hardly audible, so you can choose to accept controlled buffer underruns for the benefit of less memory impact or higher sampling rate.
Now what's interesting is that (regardless of the streaming source) certain combinations of sampling rates and buffer sizes seem to work well while others don't. One would expect that increasing the buffer size would generally improve performance (i.e. lessen glitches due to buffer underruns), but that is not always the case. On the contrary, I even found that bigger buffers can make it worse. It's not that hard to find a working configuration for a given application through experimentation, nevertheless, it would be interesting to know why Espruino behaves that way. I can imagine that the time needed for a buffer-refill is not a linear function of the buffer size, but a more staircase-shaped one, but that's just speculation. Does anybody know more?
-
I have developed a module for interfacing Winbond serial flash (w25Q series). have successfully tested it with a W25Q80BV (8 Mbit) chip. On an Espruino Pico, for many applications this chip makes a great alternative to hooking up an SD card, since the chip is tiny and costs only a few cents.
My module supports erasing the chip, erasing a sector, writing a page, sequential writing and sequential reading. There is no documentation yet, but essential comments in the source code. You should be able to use it if you generally know how flash memory works.
I put the code on github for now. We can add it to the Espruino module library once it reaches a more mature state.
-
Cool, I didn't know it was that simple (and that
spi.send
can return arrays). But if I understand correctly, there is no way to shovel the data from SPI directly into the buffer of the waveform (waveforms bring their own buffers), so I always have to copy it? And I also have to temporarily allocate the space for the outgoing zeros, so effectively I need 3 times as much memory as the received payload? -
I want to stream a waveform from an external SPI flash RAM (Winbond). Once the RAM has been prepared, reading back sequential data into is just a loop of
SPI2.send(0)
instructions whose return values are sequentially written into a uInt8Array.As mentioned in the docs, a function containing SPI calls cannot be compiled... how can I speed it up?
-
-
-
Thank you guys, the idea with the motion sensor is brilliant! I think this is just what I need. So Espruino can sleep most of the time and wake up when it's moved, to obtain a new position. What a pity that all the interesting components have to be ordered from China which takes so long...
@allObjects, can you point me to how to feed the last position to the GPS and make it initialize faster? I wasn't able to find that...
-
I'm experimenting with a ublox GPS module for a reverse geocache. Obviously the GPS module does not work inside most buildings. What would be a good approach to balance between these 3 requirements?
- user should be able to quickly obtain a position without a delay
- position should be more or less accurate (i.e. not too old cached data)
- power consumption should be minimized to allow for long battery lifetimes
The typical use-case would be that the user leaves the device lying around for days or weeks and then suddenly needs a position quickly.
We could run in deep sleep and periodically wake up to look for GPS signals, we could cache the last determined position, we should invalidate it (based on lifetime? moving speed during capture?, ...) but I'm not really sure which algorithm makes most sense.
- user should be able to quickly obtain a position without a delay
-
-
Thanks for that hint, maybe I should think about a soft power button then. I hadn't considered that yet, because it might drain the battery.
But that could also mean that I have to switch power for peripherals with a transistor since they might drain the battery much faster than just the Espruino in deep sleep.
-
I want to connect a LiPo battery and a power switch to an EspruinoBoard 1.4. I don't want to use the JST connector. Where should I connect the switched power supply? To the GND and VBat pins of the board? Or to the back of the JST connector?
As I understand the documentation, there is a fuse between the battery connector and the VBat pin. If that is the case, connecting the battery to VBat would circumvent the fuse, right?
Another question: if I solder a MAX1551 to the designated position on the board, I understand that I can charge the battery through the USB connector. Doesn't that mean that the board will always be on during charging? Otherwise the switch would have to be inserted AFTER the charger chip...
That is very interesting! I remember I looked at the code for a long time and didn't quite understand this aspect, but since it's also not documented how exactly that event is thrown, I assumed it to be correct.
I will try that with the Flash RAM when I have time and let you know. It should improve things even more there because of the smaller buffer sizes.
By the way, it seems a bit clumsy that we need an if-branch to evaluate which buffer to use. We could store references to the buffers in a 2-element array and then reduce that function to:
but it would be even cooler if the native Waveform object had built-in properties that point to the current and the other buffer.