Uint8Array over SPI

Hi,
while I am fiddling around with driving an epaper display with Espruino Wifi I'm still having issues with writing data. Just to be sure:

Is this:

var a=0x0;
for(let i = 0; i < _bufferSize; i++) SPI.write(a);

different from that?

var a=0x0;
SPI.write(new Uint8Array(_bufferSize).fill(a));

First one actually writes the RAM of the paper correctly and after update it will display correcty, but its slow (3.5s for 5800bytes).

Second one is fast, but seemingly doesn't update the RAM of the paper, because after update it still displays the same.

In the reference it says:

For maximum speeds, please pass either Strings or Typed Arrays as arguments.

Sat 2020.02.01

What is the value of '_bufferSize'?

'different from that?'

Yes, check the MDN example below

'but seemingly doesn't update the RAM of the paper'

Might be SPI timing or how the display handles a specified number of inboound chars. Without the datasheets, and all of the source code, not attempting a guess.

Writing an effective forum post that will get viewed and generate desired results

Have these resources been found, good examples on UintxxArray() usage

http://forum.espruino.com/conversations/1237/
spi.send with Uint16Array

http://forum.espruino.com/conversations/258049/
Uint8Array - is it really an array? And what about the other Uint#Arrays?

It appears the array is initialized with one char, not every location

Check the example using method fill()

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray/fill

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array

_bufferSize is the length of the graphics buffer, in my case 5808 bytes (176*264/8).

So the code is this (inside a lot more code):

IL91874.prototype.grfxBlackWhite = function () {  
  var _display = this;
  var g = Graphics.createArrayBuffer(
            this.display.displaySizeX,
            this.display.displaySizeY,
            this.display.bpp,
            {msb: true}
          );
  g.clear = function(clearColor){    
    new Uint8Array(this.buffer).fill(clearColor);print(this);
  };
  g.flip = function(callback){     
	//this does not work
	_display.wCmd(_display.display.CMD.DATA_START_TRANSMISSION_1);
	_display.wData(this.buffer); 
    
	//this works
    //_display.wCmd(_display.display.CMD.DATA_START_TRANSMISSION_1);
    //for (let i=0;i<this.buffer.length;i++) _display.wData(this.buffer[i]);
     
    _display.waitBusy(callback);
  };
  return g;
};
IL91874.prototype.wCmd = function (command) {
  digitalWrite(this.dcPin, 0);
  this.spi.write(command, this.cs1Pin);  
};
IL91874.prototype.wData = function (data) {
 digitalWrite(this.dcPin, 1);
 this.spi.write(data, this.cs1Pin);
};

So what it does is, when flip is called, transmit the buffer data to the papers ram.

It is actually only a modification of the SSD1606 module code. In there the flip function it does exactly the same: transmit the buffer data via SPI as an Uint8Array (as I just learned, more a view to the buffer?).
Maybe that's the problem? I'm assuming that what works in the example code also would work in my case.

Otherwise I can only think of a problem in transmitting speed. I played around with the SPI baud rate (set it lower) but to no avail.

I mean, the code does work, but writing 2 buffers (one for red, one for black) takes about 3.5s each + the 15s update cycle of the paper. I think there is potential for speeding up things.

3.5 seconds for 176*264 (/8*9) bits makes about 15k baudrate... indeed very slow...

But I have to break bad new to you, it is what it is, as the specs say: https://www.waveshare.com/2.7inch-e-paper-hat-b.htm : 15secs for a refresh / full update, and it has nothing to do with Espruino or any other driving platform.

Why the technology is so slow? I assume it is inherent to the electrophoresis, which by matter of facts takes time. The discovery - https://en.wikipedia.org/wiki/Electrophoresis - speaks of migrating - vs the term of moving - in a liquid. Particles with different properties and color move on different polarity in different direction with different speed in a liquid, which allow the third color (simplified in a nutshell how ePaper works).

I assume the slave device does a lot of clock stretching, which should be visible by looking at the clock with an oscilloscope. I'm pretty sure that ePaper displays do not have built-in buffer. No need for, because after the pixel (distribution of colored particles in cell) has changed, it it stays. No refresh (of same state) is needed. Even with a built in buffer, it could not be made faster. Yes the update of the buffer could be made faster, and some build it logic with more buffer could figure out what to update and go and update only the changed pixles (cells) and then go to sleep after all is updated.

ePaper are for applications that 'rarely' change the content... rarely compared to how long a content stays displayed until updated.

What is the application you have in mind and why did you choose this display technology versus other available ones?

If the answer is "low power", then you may take a look at this paper: https://www.mdpi.com/2071-1050/7/8/10854/pdf

15secs for a refresh / full update

Yes I know it updates slowly and I understand the technology behind it. And I chose it merely because the technology itself intrigues me.
But: the 15s is only the update of the display itself using the waveforms. It has nothing to do with transferring the data.

I'm pretty sure that ePaper displays do not have built-in buffer.

From what it says in the documentation it actually has two buffers: one for old data and one for new data. I guess it is because the display will do other update waveforms if the pixels didn't change between updates.

I think writing the data in the for loop might be slow additionally because of pulling the pins high/low before writing (overhead?).

I will try to do a time measurement on the Arduino example, just to see what speed the display is capable of receiving. The Arduino example uses 4M baud rate if I see correctly.

Ok, so I did a little more testing and sending the data as Uint8Array only writes the first byte of the array to the papers RAM, see pictures (first (red one) is actual, second is supposed to be).

Strange. Could still be an issue with the papers ability to handle all the data transferred to quickly maybe? I lowered the baud rate to 100. It slows everything else down of course, but sill no success.

Just read the specs for the controller. Seems like it supports clock rates up to 20MHz.

3-wire/4-wire (SPI) serial interface for system configuration: Clock rate up to 20MHz

2 Attachments

Sun 2020.02.02

'sending the data as Uint8Array only writes the first byte of the array'

At L9a console.log( "buf: " + this.buffer ) What is the result? e.g. is the result just one char/byte? . . . and/or. . . convert to other base if necessary
 

Untested solution but, . . .

Checked out the MDN example links in post #2 above?

Was an attempt made to init the Uint8Array using the suggestions, one of which is in post #9 found in the first link of post #2 above, and an attempt to replace L10 with those suggestions in the code block in post #3 above?

When done, what is the output?

Looking at L26 L27 L30 L31, are CS and DS backwards? Should CS be toggling with the data as input, then the data line would only * see * a one byte state change!

@Raik, took a look at the SSD1606 and IL91874 controller doc. Indeed, the controller has data buffers. I could though find not much how they are used. I assume it is for optimization not only for time performance, but also for quality performance, such as managing the ghosting. Reading thru all the possible OTP and runtime settings makes me question the readiness of the technology for easy use. But this does not mean not to be curious why the data transfer is that slow.

The controller has a busy signal output... and looking at the full cycle of an update from powering on the device, preparing it for data reception, to finally shut down again could explain this: shut down has to wait until the controller has updated the display cells. What I did not like in the module was the byte-wise operation of clear display screenbuffer with desired color (.csb(cb,clr)). Going back and forth between applications JS and Espruino firmware for each byte slows things don for sure. Alternative: sacrifice memory for another buffer, fill it with the desired value, and send that to the controller (I do not know if .csb(cb ,clr) is in the path of your code).

Various items I cannot understand:

• In post #1: I would have answered: 'no difference' - different from what @Robin concluded. What I can say for sure is that .fill(fillValue) fills the whole Uint8Array and returns the array. Therefore I say: they are same/there is no difference, except that the latter takes much longer to get the data across (see comment about going back and forth between app JS an Espruino firmware).
  
• In post #6: Why only one byte is set. Could it be that the window and x / y address increment settings got messed up?
  
  

Another comment I'd like to make: I read about controller and displays the documentation mentioned that the OTP / LUT data is (usually) factory set for best display (quality) performance - of the particular display on which the controller is mounted. Messing with that I reserve for myself for a later stage (invoking .ini()).

I have a 1.54" 200x200 tri-color ws display sitting around for a while now, and I noticed the the originally white background 'turned rosy'. May be time has come to fool around with it... to co-bang-head-on-wall ... ;\ or better: |:| (flat forehead).

I'll test again tonight to see (proof ;-)) that the buffer is indeed 5808 bytes of data, not just one byte.
I'll also try switching CS/DC pins and/or modifying the data write functions to set those pins differently.

I read about controller and displays the documentation mentioned that the OTP / LUT data is (usually) factory set for best display (quality) performance.

I noticed that the OTP works better for me, the custom LUTs turn the red into rosy (like your display sitting around).

But: I saw that Ben Krasnow from YT channel Applied Science
made a nice video about hacking the ePaper update rates. That'll be next when getting the data transfer fixed up (if possible). :-)

to co-bang-head-on-wall

Sounds good |:|

This is really odd - I don't suppose you have some kind of oscilloscope you could use to check if the data is being sent?

SPI.write(uint8array)

Should be perfectly fine, and we do it all over the place in different modules.

You could give software SPI a go in case it makes a difference?

I don't suppose you have some kind of oscilloscope you could use to check if the data is being sent?

Nope, I'm not the hardware guy ;-)

Should be perfectly fine

I assumed so as well. I'm quite confident it is an issue with the paper itself.

You could give software SPI a go in case it makes a difference?

I did and it didn't change a thing.

I'll investigate more tonight.

Pretty interesting and revealing this Ben Krasnow presentation. What it kind of confirmed to me is the quick transmission is with 20Mhz to get the data buffers updated, but that transmission follows the update of the display cell by cell with looking up the waveform and pushing it out. The presentation also proves that speed is inverse to display quality (contrast). Btw, very neat oszi - a far cry from what I ever can dream of.

but that transmission follows the update of the display cell by cell with looking up the waveform and pushing it out

Actually, you can (should?) set the waveform on initialization of the display. I guess it is then stored on the chip. In the SSD1606 implementation it is done the same way.
For updating the paper after tranferring the pixel data, you just send one command with no data. In my case it would be

DISPLAY_REFRESH:0x12

I don't think you should (have to) adjust the wave forms: they are factory set when display was tested and initial (test) image data was loaded onto, but for sure you can.

At L9a console.log( "buf: " + this.buffer ) What is the result?

console.log( "buf: " + this.buffer );
buf: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0....

console.log( "buf-length: " + this.buffer.length );
buf-length: 5808

I also tried initializing it as Uint16, same result.

Looking at L26 L27 L30 L31, are CS and DS backwards?

I tried switching hard- and software-wise, with negative result.

So, I think I found the culprit. I (again) had a look at the display driver specs and on page 16 it shows the cycle graph(?) for the 4-wire SPI interface, see attached image.

Below the graph it says:

CSB can be “H”between parameter/ command. And SCL, SDA, D/C are invalid
during CSB=”H”

That and the rising flank of the CSB signal after the 8 databits on the right, leads me to believe that CS needs to be pulled down and up again for every byte written.

So I changed my write function like so:

function wData(data) {
 digitalWrite(dcPin, 1);
 _spi.write(data);
}
 digitalWrite(csPin, 0);
for (let i=0;i<bufferSize;i++) wData(buffer[i]);
 digitalWrite(csPin, 1);

Now this does not work anymore, because CS only changes once right before all the data writing.

Changed it back to:

function wData(data) {
 digitalWrite(dcPin, 1);
 digitalWrite(csPin, 0);
 _spi.write(data);
 digitalWrite(csPin, 1);
} 
for (let i=0;i<bufferSize;i++) wData(buffer[i]); 

immediately starts working.

I assume that

_spi.write(data,csPin);

also pulls down csPin only once.

So now the questions would be: would it be feasible to create a native SPI write function that pulls the cs(nss) pin down for every byte written and does that make sense?

Would my case still benefit speedwise? Otherwise I would be stuck to my single bytes write algorithm and the speed bottleneck it causes.

EDIT: Just stumbled across Espruinos capability of inline C code. I guess that could speed up the data transfer loop...?

1 Attachment

On lower level function paths with multiple writes, manage the writing of the -csPin yourself because you know what is going on. For higher levels, you handle it in the top level function.
If you need to write multiple things, you set -csPin as first thing, then you do the multiple writes without passing the csPin as last argument, and for the last write you pass the csPin. Yes, the idea is that the -csPin stays low until you are done. For a single (occasional) write you pass csPin with every write as last argument.

Yes, inline C (and compiled methods) are faster as long as you have not to resolve references by name. If you can pass all data into the C code at once, complete everything within, that works great. Works especially great for bit manipulations within contiguous Uint8..32Arrays. For calling functions / methods that expect the caller to be in JavaScript adds some overhead... after all, the main is in JavaScript... and if something is too slow and used often, it usually gets into the firmware (and built-in module or even omni-present class/function).

For most situations - especially with Graphics buffer - @Gordon has already taken care of by providing options in the setup. If you though think about partial update out of a graphics buffer, the pulling out and prepping the array for the display, it is for sure helpful (I used it here: Efficiently moving things around in zig-zag Graphics buffer visualized w/ 24/32 bpp displays / neopixel strings).

Mon 2020.02.03

'I guess that could speed up the data transfer loop...?'

Yes, by a small amount. But, . . . isn't the data bit rate set by the setup of SPI? (yes)

http://www.espruino.com/Reference#l_SPI_setup

As the image in #16 is helpful here, has the use of an inexpensive logic analyzer been used?

I thought I had some images of SPI clocking at 100000 baud (without the need for inline' C') in action, but is actually the UART Tx-Rx - similar output post #3 there you'll get the idea though - one way of speeding development here
http://forum.espruino.com/conversations/332295/#comment14688770

What about   SPI.send()   vs   SPI.write() ?

SPI syntax for receive string clarification and sanity check please

Yes, by a small amount. But, . . . isn't the data bit rate set by the setup of SPI? (yes)

Setting the baud rate, doesn't seem to matter much.

//byte by byte transfer
data 1 took: 3.80794239044s @20000000 baud
data 2 took: 3.80806064605s @20000000 baud

data 1 took: 4.26346969604s @1000 baud
data 2 took: 4.26357555389s @1000 baud

data 1 took: 4.26337814331s @100 baud
data 2 took: 4.26357364654s @100 baud

data 1 took: 4.22884178161s @software SPI
data 2 took: 4.22821426391s @software SPI

//byte by byte without switching CS
data 1 took: 3.53529644012s @20000000 baud
data 2 took: 3.53545284271s @20000000 baud

There is definitively switching overhead to the CS pin, as mentioned here.

As the image in #16 is helpful here, has the use of an inexpensive logic analyzer been used?

No. Didn't fiddle with that.

What about SPI.send() vs SPI.write() ?

Tested, works, but same speed