Cozuino ? Espruino-Cozmo ? ( facial expressions lib for Pixar-like characters faces on tiny OLED screen )

Hi there !

If you ever heard about 'Cozmo', it's a tiny robot that does stuff ( & that seems quite capable seeing what it has embedded for doing so) & that costs $$$ ( 200 if I'm right ).

A while ago, I've stumbled across an instructable project ( gotta retrieve the link .. ) taking on the 'movement' side of things of a little robot, but that only used images to display few faces inspired from Cozmo's.

Having some fun with OLED screens & Espruino, It seemed a good idea to try getting those cute expressions onto our favorite uC :)

I started with 2 rectangles, one for each eye
As images/sprites: heavy for Espruino's ( but can't fit all of them on esp8266's )
As objects with 2 eye rectangles with relative coords for a center point, a little heavy since we have many expressions, but maybe doable if storing the expressions in memory & retrieving them later on ?

I'm currently able to display each of the facial expressions ok, so the next steps are:

• animating the 'center point' position, to have both eyes move around the screen
  
• tweening the transition from a facial expressions rectangle's coords to another ( to get a 'morphing ' eyes shapes animation )
  
• getting round corners for those rectangles to achieve a nice smooth look
  
• being able to change the one color used for the onscreen shapes ( to still get 'instant' refresh but express 'emotion' through color as well as facial expressions & their roundness )
  
• being able to have one or two additional colors relative to the current one ( little brighter & little darker ), to get even smoother using a tiny color gradient ( more advanced gradient & even 'spotlight'/'shades' in a corner of the eye(s) 'd be awesome, but where's not quite there yet I guess .. )
  
  

What we currently have:

• POC code in html5 canvas & Espruino for animating a pixel's movement across the screen( very basic unoptimized tweening )
  
• POC code to display facial expressions in html5 canvas ( allows to keep all of them & get quicker debug for drawings )
  
• POC code to 'emulate' needed features of Espruino's Graphics context via html5 canvas ( very early stage, but helps to later copy/paste code to Espruino directly )
  
• POC code to draw curves on html5 canvas, not yet tested on Espruino
  
  

If anyone's adventurous/skilled/interested/free-timed enough, I'll be more than happy to update the project & the ~'lib/api' part & fork advances in the facial expressions part as well :)

Also, it crossed my mind to have a 'fake' screen ( using html5 canvas running from local server & communicating via serial connection to get updates ), but this 'd prevent being able to use the Espruino IDE at the same time ( or someone has a neat trick to forward serial comm to Espruino IDE while connected to other server itself talking web sockets with a webpage ? ), hence the idea of an html5 canvas embed within Espruino IDE could make a lot of sense there ( as well as for other stuff than a fake screen, ex: simulating a board, attached components .. oops, it's getting bigger & monstrous, sorry ;p -> but 'plugins' acting as fake connected devices may make sense for quick testing ? )

Since the code is quite big, I only post links to the two files for now:

• the facial expressions exported via Illustrator script & tweeting tests as well as console pixel preview: https://github.com/stephaneAG/Espruino-Cozmo/blob/master/expressionsTests.md
  
• round corners tests, to be added to code using Graphics once I'm sure of the best way to do so ( goal is to directly use the custom illustrator export script to get the curves right as we did for the rectangles ): https://github.com/stephaneAG/Espruino-Cozmo/blob/master/roundCornersTests.md
  
  

and the direct jsbins:

• expressions: https://jsbin.com/deritexoya/edit?html,css,js,console,output
  -rounded corners: https://jsbin.com/kovihocugi/edit?html,css,js,console,output
  
  

on the attached images: the gib one is one of the original Cozmo expression ( a screenshot of the robot's screen ), while the second is a quick save of a vector ( .ai ), and the goal is to reproduce & morph between those on an OLED with Espruino :)

2 Attachments

That'd be really cool. I guess the fastest thing might be to just store the polylines as Uint8Arrays and then put those into g.fillPoly? Assuming you were only drawing regular polygons? If they were irregular in one dimension only you might be able to get away with rotating the graphics and still using fillPoly though?

I think one thing we're missing is a fast way to interpolate between two arrays though, which would allow for smoother animations. I guess that one might actually be a good use for the E.compiledC...

;p

understood & agreed for the Uint8Arrays -> g.fillPoly
not sure I catch you on polygons "irregular" in one dimension .. ?

got it for the fast interpolation way -> could it mean implementing the tweeing fcns in E.compiledC & passing two Uint8Arrays as fcn args ? ..

not sure I catch you on polygons "irregular" in one dimension .. ?

Yes, not explained well! Espruino's polygon drawing works in a pretty basic way: https://github.com/espruino/Espruino/blob/master/libs/graphics/graphics.c#L385

Basically: For every Y value, a start and end X coordinate is calculated. Espruino then fills in using those coordinates.

It means that if there are separate areas that needed to be filled on a row of pixels, they'll just be filled between.

As a result:

            XX
 XXX        XXX          XXXXX
 X XX      X  X          XX  XXXXXXXXXXXXXX
 X  XX     X  XX          X          XXXX
 XX  XX   X    X          X       XXXX
  X    XXXX    XX         X     XX
  X     XX      X         X      XXX
  X             X         X        XXXX
  X              X        X           XXX
  X              X        X       XXXXXXXX
  X              XX      XX  XXXXX
  X    XXXXXXXXXXXX      XXXX
   XXXXX
                             Renders fine
      Doesn't render
        correctly

I guess maybe at some point someone might like to improve the algorithm so it can handle that first case ;) There are slightly more complex ways of rendering polygons that would work better.

could it mean implementing the tweeing fcns in E.compiledC & passing two Uint8Arrays as fcn args ?

Yes - that's probably the most sensible option - maybe 4 arguments: destination,source1,source2,data_length+tween_amount<<16

If you need a hand with that let me know and I can do some code.

Hi there !

'll be back on the subject later this week ( I hope :) ), although I've just added the "partial flip" we talked about aside a little benchmarking on the subject ( see http://forum.espruino.com/conversations/269808/#comment14607110)