ILI9341 Performance Improvements

Hi

I got a 2.2" 240*320 ILI9341 controlled display and have been really pleased with it (considering its low price) but the current ILI9341 module uses a Graphics Callback to display every pixel which is a bit too slow.

I have implemented a new method in the Controller module:

LCD.drawStringBuffered(str, x, y, fontSize, r, g, b)

This creates a buffer for the string being drawn then sends it to the display all at once. It is working (much faster than pixel by pixel) but I am having issues with the colour (wrong colours being displayed). I think it is something to do with the endianness of the 16bit colour data.

The current module uses the following code when writing 16 bit values (e.g. pixel colour) to the SPI:

spi.write(c>>8,c);

Could this be achieved by changing modifying the spi setup to use an order of 'lsb'. (I did try this but nothing was displayed!)

Anyway I'm pleased with how the screen performs & can work around the invalid colors but it would be great if I could get it functioning correctly. My method is as follows:

  LCD.drawStringBuffered = function(str, x, y, fontSize, r, g, b){
	//Create a dummy Graphics object to get size of string in fontSize
   var g = Graphics.createArrayBuffer(1, 1, 1);
	g.setFontVector(fontSize);
	var width = g.stringWidth(str)
	
	var height = fontSize * 1.3; // this works but not sure how height is related to size for vector font
	var x2 = x + width-1;
	var y2 = y + height-1;
	
	//now the width/height is known actual Graghics object can be created
	g = Graphics.createArrayBuffer(width, height, 16);	
    g.setFontVector(fontSize);
	g.setColor(r, g, b);
	g.drawString(str,0,0);
	
	//write buffer to spi
	ce.reset();
    spi.write(0x2A,dc);
    spi.write(x>>8,x,x2>>8,x2);
    spi.write(0x2B,dc);
    spi.write(y>>8,y,y2>>8,y2);
    spi.write(0x2C,dc);
    spi.write(g.buffer);
    ce.set();
	
	//I tried shifting the buffer by 8 bits - some colors correct but others lost
	//offsetBuffer = new Uint8Array(g.buffer, 1);
	//spi.write(offsetBuffer);
  }

Thanks - that's a really neat idea!

Unfortunately LSB in SPI is for the bit order, not the byte order. I guess there should really be an option when initialising Graphics, but for now you could actually cheat and set the colour that you want right at the start...

setColor takes either r,g,b or the actual bit representation of the colour, and getColor returns the bit representation. You could do setColor/getColor on the first graphics context, and could then flip the bytes around for the next one (the first context would have to be 16 bit too).

  LCD.drawStringBuffered = function(str, x, y, fontSize, r, g, b){
    //Create a dummy Graphics object to get size of string in fontSize
   var g = Graphics.createArrayBuffer(1, 1, 16);
   g.setColor(r,g,b);
   var col = g.getColor();
   col = ((col>>8)&0xFF) | ((col<<8)&0xFF00); // flip bytes
   //....
    g = Graphics.createArrayBuffer(width, height, 16);  
    g.setColor(col);
    // ...

Thanks - that sounds perfect! I'll try it out this evening and may write a completely new ILI9341 module which is backwardly compatible with the existing one but doesn't use any callbacks.

Setting the background colour could be an issue but I could query the display to get the existing background (probably just one pixel needed assuming the background was from a simple fillRect).

The SPI speed should be the only bottleneck now - clear() takes a second or so but then that is 320 * 240 * 16 = 1,228,800 bits!!

An option when initialising Graphics would definitely be cleaner (some day!) but your suggested method should have very little impact on performance.

Speed paid with memory... or memory paid with speed. @DaveNI, do you have some memory profiling for this approach?

@Gordon : I implemented your suggestions at the weekend and am really pleased with the results: Drawing text takes 1/4 of the time taken by the stock ILI9341 module and the colours are now correct - thanks.

I have re-factored my module to make is easier to use but have one question:

Is is possible to get details of the current font from a Graphics object (e.g. is it Bitmap/ Vector/ Custom, vector font size/custom font bitmap etc)? I'm sure this is held within the object but just not surfaced (maybe there a way to access it?).

Without these details I need the user to set the font via a new method: setFont(nameOrSize). This sets the font of the LCD object (using eval to call the setFontNxN() method for custom fonts). I then save the nameOrSize parameter to use when setting the font of my internal buffer Graphics object.

@allObjects : You are 100% correct about memory - even one character in a large font can use a lot of ram. I have taken this into consideration and the connect method now takes an additional parameter: maxBufferSize. If the string requires a buffer larger than this (based on width/height) I draw the characters individually (an individual character can still cause this to be exceeded - if the font is very large).

I will post the code later when fully completed (maybe someone could review the code and if OK add it to the modules library - its a really nice display, just a bit slow at present)

Is is possible to get details of the current font from a Graphics object

I'm afraid not. If you want to maintain backwards compatibility you could do:

var g = ....;
g._setFontVector = g.setFontVector;
g.setFontVector = function(s) { this._setFontVector(s); save s somewhere; }

It'd be great if you could contribute something! As you say, it's a bit too slow right now. I wonder if it can be put into a 4 or 8 bit mode? That would at least double the throughput when clearing.

Ideally they'd have a 'fill the current window with colour x' capability, but I never found one :(

Great idea - that would obviously work for setFontBitmap too. I'll have to think about the custom fonts - if I "override" setCustomFont in a similar way maybe I could trap setFont8x12 etc. (its really cool the way custom fonts are implemented - I was impressed)

Although I've programmed for years (vb, delphi, c# ...) I'm new to javascript (very basic knowledge before getting my Espruino - learning a lot)

Thanks Gordon, "Overiding" the functions was just what I needed to make my buffered ILI9341 module backwardly compatible with the existing one.

The module has two additional methods:

setUseBuffer(true/false): If true then use buffering, otherwise use callbacks (default=true)

setMaxBuffer(size): Sets the maximum size of the graphics buffer (number of 16 bit pixels). This defaults to 2000 (i.e. 4kb, similar to the 1bit PCD8544 84*48 display).

The full string is generally written in one go but if the maxBufferSize will be exceeded it is written one character at a time (It may make more sense to have the maxBufferSize in KB).

The buffer is only used during the drawing process but you may wish to lower it if memory is an issue. (NOTE If set too low, a single character in a large font could exceed maxBufferSize, I suppose I should really revert to non-buffered mode if this arises)

I've attached the module (DL_ILI9341.js) and a simple demo comparing buffered with unbuffered.

Hopefully anyone with one of the displays will find it useful - The demo takes 7.6 seconds unbuffered and 1.7 seconds when using buffering. Just set the appropriate pins in onInit().

2 Attachments

• DL_ILI9341_Test.js
• DL_ILI9341.js

Wow, this looks great - thanks!

Interesting:

  //Added "*1" to prevent minifier removing brackets
  function swap16(val) {
    return ((val & 0xFF) << 8) * 1 | ((val >> 8) & 0xFF) * 1;
  }

Gives me hints for solving a similar problem I face in http://forum.espruino.com/conversations/256957 ,
where minifier removed parentheses and Espruino JS choked on their absence... ;-) ...how can you choke on something that is not there... ok: choking on the statement body not greased with parentheses...

var f0 = function() { console.log("f0()"); };
var f1 = function (b) { if (b) for (...) f0(); } ;

in code in Sandbox read with minification on fails (still?) to NOT execute the for statement for

f1();

I have come across something similar to that - its because the minifier identifies that the final semi-colon in a function is superfluous and removes it. The Espruino interpreter chokes if there is no semicolon after a loop (in my case a while loop). I added a "return 1;" statement after the loop and all was fixed. Hope this helps.

Maybe we should compile a list of things to avoid if you want your code to minify correctly (regardless of whether to problem lies with closure or with Espruino)

@DaveNI do you have an example of the (minified) code that breaks Espruino?

Things like the loop are probably very easy for me to fix - once I know about them!

@Gordon I wasn't suggesting there are many, the interpreter seems extremely "fluent" - I have only come across 3 issues:

The first was when closure decided to insert a labeled statement when I used a break or return within a loop - I can't remember the exact code, but I wouldn't expect labels to be implemented in Espruino and if this does occur it is extremely easy to see within the error message.

The second was as described above - closure identifies that the final semi-colon in a function is superfluous and removes it - Espruino expects a semicolon after a do.. while (and possibly after for loop as @allObjects experienced). The following code is valid but breaks when using simple optimizations

function test(){
       var i=0;
       do{
             console.log(i++);
       }
       while (i < 10);
}
test();

The final problem was with brackets being removed by the minifier - the precedence of operators must be evaluated slightly differently by Espruino. This was probably the most difficult to debug because an error wasn't thrown (correct answer is 256 but returns 0 when minified because outer brackets are removed by minifier )

function swap16(val) {
        return ((val & 0xFF) << 8)  | ((val >> 8) & 0xFF) ;
}
console.log(swap16(1));

I hope this simplified code helps - although minor its the sort of thing that may put beginners off the platform and that would be a pity because it can be extremely enjoyable to work with.

The minifier used by Espruino - Google Closure ( http://closure-compiler.appspot.com/home ) is excellent & I think it makes sense for the IDE to make use of it's API but it is strange at times:

I've just noticed that when I minify the first example above using Simple Optimization but select the "Pretty Print" option the semi-colon isn't removed!

I thought "Pretty Print"only affected white space?

(It still removes the brackets in the second example though.)

Thanks - I really want to get these kind of things sorted - as you say, they're pretty minor (and often easy to fix) but they effect the overall impression of the interpreter. If you paste some JS code in, you expect it to work.

The do...while thing is now fixed, and will be in 1v72. It was actually a symptom of a bigger issue, so I'm really glad to have that fixed.

I'm still working on fixing the precedence thing - it's interesting, as I'd believed that & and | had equal precedence, but it turns out they don't. It seems it's even the same in C as well. I guess I'd never come across it because I tend to bracket things if precedence wasn't obvious - you learn something every day :)

Agreed - I tend to use brackets too - I can usually only remember that * / comes before + -

I could never remember all this accurately: Operator_Precedence

I've just fixed the precedence issue - in the process I found a way to make the interpreter faster and use a bit less stack too. It would be really handy if people could try out this build and see if anything is horribly broken?

It passes all the tests, but part of me thinks it shouldn't :)

Sounds good - I'm currently at work but will definitely try it later this evening.

Will test this build and see what it does to me. I had the challenge with left-right-shift (<<,>>), added parentheses to overcome the issue, but when minified, it removed them because of precedence they are not needed.

It is great to have all these tools - and their view and implementation of javascript - but if they are not on the same page what language definition is, it becomes Russian roulette whether will work or not. As much I like the 'adamant' - to the error - ;-) reminder to add semicolons in IDE, I'm not expecting the Espruino interpreter to depend on that (I mean, if it means to save a lot - with the definition of 'a lot' by @Gordon's judgement - then so be it).

The reason I plea for as much a possible compatibility with JS in the Web world is because of the huge pool of people familiar with the JS world from there. There exists still this perception JS is not a language for serious... and it is complicated and a night mare... even more so among programmers of other languages.

Making it work the way users are, is the best and helps to establish a good reputation.

@Gordon I tested 1v72 - it seems fine, I tried a few projects and did not get any errors.

Both issues appear to be fixed and it performs better than 1v71 but still slightly slower than 1v70 (by about 10% as you said earlier).

I really appreciate how quickly you managed to sort these issues out.

@Gordon - Further to my reply above I have noticed an issue with 1v72 (it may also happen with 1v71, I haven't had a chance to try). Its regarding the RTC after a hard reset.

Power up Espruino and try this code (it should take 3 and a bit seconds):

function test(){
     for (var i=0;i<10000;i++){
        var j = new Array(10000);
        j.fill(0xFF);
      }
}
var t = getTime();
test();
console.log(getTime()-t);

Now disconnect IDE, hard reset & run again - zero duration!

(I hope the code above has no syntax errors, I cant run it just at present)

If you keep querying getTime() I think you'll find it reads the same value all the time?

I've had this - but not specifically after a hard reset.

I think it happens when the device has lost power, but not completely gone to 0v - eg. if you unplug it and re-plug it.

The low speed oscillator stops, but the registers are still set up - so when Espruino starts up it thinks everything is working great and doesn't reset the RTC - but actually the oscillator isn't running any more.

I came across the following link about speeding up SPI and was wondering if something similar could be implemented in Espruino (maybe it already is - write vs send?).

http://developer.mbed.org/users/Sissors/code/BurstSPI/

I thought it could speed up writing large amounts of data to the ILI9341 but not too sure if it is relevant?

Yes, that's what SPI.write does already I'm afraid :)

With a bit of work, writes could be made a bit faster by being able to do it in the background, but I think you'll still be disappointed. The best thing to do would be to build the driver into Espruino itself (or to be able to make it from native code).