How to use the Bangle.js Layout library

Hi! With the new Bangle.js 2 coming along the screen sizes/numbers of buttons are different, and it's meant that most apps are going to need a bit of fiddling to work correctly. Rather than having two versions of the same app (or lots of 'if' statements) I thought it'd be good to have a layout library that places everything on the screen in the right place.

THE TUTORIAL IS NOW ONLINE:

http://www.espruino.com/Bangle.js+Layout

This looks pretty neat, hopefully I'll have some time to play with it this weekend.

Would it be bloaty to add something like this (untested):

  Layout.prototype.setLabel = function(id, label) {
      var l = this._l;
      if (l[id].label !== label) {
          this.clear(id);
          l[id].label = label;
          render(id);
      }
  }

(or even add it to txt/btn elements, so you could just do layout.date.setLabel('New label'))

Just because I expect otherwise the compare-clear-update-render code will probably show up in pretty much all apps anyway.

Looks nice and useful. Is there also color attribute? Or one could separate layouts per color and do g.setColor between.

EDIT: oh yes, there is col,bgCol documented in https://raw.githubusercontent.com/espruino/BangleApps/master/modules/Layout.js

However now I see there is lot of "Bangle." calls inside so it is not so universal and reusable for other devices.

Would it be bloaty to add setLabel

I think it could be a good one to add, yes. It's could also provide some extra logic in there as sometimes new text might change the width of the field which could move other things.

there is lot of "Bangle." calls inside

It's only to handle the touchscreen so they could always be put inside if (global.Bangle) ...

If you're wondering about it for other smartwatches I'd definitely say you should try including jswrap_bangle in your builds for those so the Bangle.js stuff is exposed

edit: I should also add that I'd like to make the layout/rendering for this really fast, so when we get something that works well for what we need it for I may well build it into the firmware - and then it could make sense to use it because it'll actually be faster that issuing draw calls from JS.

Gave it a spin, and it works pretty nice. Definitely a huge improvement over fiddling with pixels by hand.

One minor annoyance I ran into:
The smoothest way to simply update a bunch of labels with minimal flicker seems to be setting fillx: true and bgCol: g.theme.bg for every element and then do layout.update();layout.render(), but that clutters up the layout object.
(It annoyed me enough that I wrote some code to add those properties afterwards)
But maybe once it's baked into the firmware it will be fast enough that we can just do layout.clear();layout.update();layout.render() without too much flicker?

I haven't looked too deeply at the new layout library yet, but maybe this will serve as some inspiration...

A while back I was experimenting with a layout library of my own. I never quite got it to the point where I was happy releasing it to the world, but it did have some interesting features. Most notably:

• A flexbox-inspired layout system, with column and row support
  
• Support for elements with dependencies between their width and their height (eg. wrapping text)
  
• Easy to extend with new types of elements
  
• Automatic lazy rerendering - it would detect which elements were new or different each frame (by way of hashing them) and only clear / redraw those elements
  
  

Usage looked like this:

let canvas = trui.canvas();

canvas.render(
  trui.column({x: 0, y: 24, width: 240, height: 216, children: [
    {flex: 1},
    trui.padding({left: 10, right: 10, child:
      trui.text({font: "Vector", fitWidth: true, text: `${hours}:${padNum(d.getMinutes(), 2)}`})
    }),
    {flex: 1},
    trui.row({children: [
      {flex: 1},
      trui.text({font: "Vector", fontSize: 28, text: meridian}),
      {flex: 1},
      trui.text({font: "Vector", fontSize: 28, text: padNum(d.getSeconds(), 2)}),
      {flex: 1}
    ]}),
    {flex: 1},
    trui.wrappingText({font: "Vector", fontSize: 18, lineHeight: 30, align: 0, text: locale.date(d,false)}),
  ]})
);

(Each {flex: 1} there is a flexible spacer)

If anyone would like to see it in action, they can try it here: https://www.espruino.com/ide/?gist=bde5d085b2723a87e93f55411c03983a

@NebbishHacker, tried to run your Web-linked code in the emulator and get:

>
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v10.9 (c) 2021 G.Williams
>
>Execution Interrupted
at line 210 col 4
  });
   ^
in function called from line 215 col 4
})();
   ^
New interpreter error: LOW_MEMORY,MEMORY
>Uncaught Error: Cannot read property 'canvas' of undefined
 at line 221 col 18
let canvas = trui.canvas();
                 ^
>Uncaught Error: Cannot read property 'render' of undefined
 at line 235 col 9
  canvas.render(
        ^
in function "draw" called from line 265 col 6
draw();
     ^
in function "tick" called from line 276 col 6
tick();
     ^
>

What do I miss here?

(See no reason why it should not run in the emulator.)

It does on my site and look's very nice.

1 Attachment

The problem was my use of a particularly long immediately invoked function expression - it only worked for me because I had enabled "Pretokenise code before upload" in the web ide settings.

I've tweaked it so it should work now regardless.

@NebbishHacker, Ic, thank you. I faced a similar issue with my http://www.espruino.com/ui supporting touch screen and soft keyboard (for screenshots see: http://forum.espruino.com/conversations/292990/) when defining a layout. For the framework code I could have only limited modules 'inline'. Placing the layout in a function instead of in immediate execution helped in my case, since the definitions were simpler than your module inline code. (The ui does though a bit more than rendering... and, of course, it was a larger display with much more items on it and the layout is position and size-wise custom/hard coded to the device).

I like the lazy part very much!

Modified line # 227 to

  let hours = (d.getSeconds() % 2) ? 12 : 6; // ((d.getHours() + 11) % 12) + 1;

to showcase nicely the gracious handling of different long - too long - line(s)... see attached screen shots ;)

1 Attachment

Nice - I like the idea of the lazy rendering too (with the CRC to check for changes). Although it looks like you don't use it in that example? While it's super easy for the user, I wonder if it'd just make things too slow to try and handle it automatically?

I wonder if there's anything that we could pull across? I think the row/col layout and 'flex' areas are pretty similar to what Layout does with {fillx:true} so hopefully feature-wise they are reasonably comparable there.

@rigrig the changing values is something so common we definitely need a nice, fast, easy way of handling it. Just thinking here but I guess there are two main solutions:

• Add layout.rerender - so you set modified:true on any item that has changed, and it then automatically updates the layout for those and redraws anything that changed.
  
• Make it so you specify the layout with the maximum amount of text you expect in any particular field, then you only call update once at the start (then fillx wouldn't be needed). I guess maybe .render should automatically clear the area regardless of bgCol, which would tidy things up there too?
  

Add layout.rerender - so you set modified:true on any item that has changed, and it then automatically updates the layout for those and redraws anything that changed.

I guess that sounds ideal, but what if changing some text means the item changes size, so adjacent (non-modified) elements would have to move?

Make it so you specify the layout with the maximum amount of text you expect in any particular field

That might be a bit tricky with Vector fonts, as it depends on the actual characters.
I'd really like to see fitWidth/fitHeight being pulled across, maybe even implicit:

{type:"txt", font:"10%", label:"Vector:10%, automatic width" },
{type:"txt", width:"120", label:"Vector, fitted to be 120px width" },
{type:"txt", font:"10%", width:"120", label:"Vector:10%, cropped/padded to 120px" },

maybe .render should automatically clear the area regardless of bgCol

Or maybe make bgCol default to g.theme.bg when omitted, and you can set it explicitly to null/false to prevent clearing?

Another wishlist item: specify width and height in percentages (of whole screen).

{type:"txt", width:"50%", label:"Vector, fitted to 120px on Bangle.js 1" },

I like the idea of the lazy rendering too (with the CRC to check for changes). Although it looks like you don't use it in that example?

My example does use lazy rendering! Each time canvas.render is called, the only things that get redrawn are those that don't already have their hash recorded. The thing to pay attention to is the rects property of the canvas, which acts as a map between the hashes of the elements currently on screen and the areas they occupy.

I wonder if it'd just make things too slow to try and handle it automatically

The hashing is pretty fast, since it just delegates to E.toJS and E.CRC32, and the rest of the logic honestly isn't too complicated. I'd say it's at least worth investigating.

The Vector Clock app I just submitted also uses very similar lazy rendering logic, without all the layout stuff.

I think the row/col layout and 'flex' areas are pretty similar to what Layout does with {fillx:true} so hopefully feature-wise they are reasonably comparable there.

Correct me if I'm wrong, but it sounds like my 'flex' system is a bit more powerful? I'm guessing that only one element can be set to {fillx:true} in a row or column, whereas in my system the remaining space can be divided between any number of flexible elements, in proportion to their flex numbers. This means, for example, you could allocate 1/3 of the space in a row to one element and 2/3 to another by giving the first one a flex number of "1" and the second a flex number of "2".

Add layout.rerender - so you set modified:true

This will still require the developer to keep a prevVal to compare against.
I'd much rather the layout manger figured out when an item had been changed / not changed.
It would make the apps much easier to write.
Also on the B2 I have noticed you dont get flicker so you can code as if its not a problem.
On the B1 you either have to use buffered screen writing (uses CPU) and memory or minimise flicker by shadow variables for the previous value to compare against.

Since there appears to be some interest in lazy rendering, I've created pull request implementing it in the Layout library.

Just pass the third argument {lazy: true} to the layout constructor, and updates are as easy as:

layout.time.label = require("locale").time(d,1);
layout.update(); // Only needed if there's a chance that something moved
layout.render();

Demo here: https://www.espruino.com/ide/?gist=cc5f26ad897ecaa531b698781d47964a#

Edit: I see my demo has the same issue as before. If it gives you any errors enable "Pretokenise code before upload" in the web ide settings.

what if changing some text means the item changes size

Yes, that would be a pain, however I imagine for the vast majority of apps you probably don't want things to start moving around on the screen when text changes? It's almost a benefit if stuff stays in the same place unless it is explicitly updated.

Vector font width

Yes, this does make a lot of sense. I was avoiding it because I assumed it would be a slow iterative thing, but @NebbishHacker has a really neat trick there to do it quick.

It does make me wonder whether if no font is specified, it almost makes sense to choose the font automatically? Like use Vector for >20px, and then 6x8:2, 6x8, 4x6 depending on the width available?

I guess you don't want the font changing each time but if the font selection were done during 'update' it wouldn't be a huge problem.

I'm guessing that only one element can be set to {fillx:true} in a row or column

No, you can have more than one in layout. The way it's implemented it'd be trivial to make fillx/filly an integer though. That would make a lot of sense.

My example does use lazy rendering!

Sorry, I didn't notice that! I just saw the UI 'tree' being recreated each call and assumed it wasn't :)

Thanks for the PR - I'm a bit busy at the moment but I'll try and have a proper play with it at some point. I'll post some comments on the PR itself though

I imagine for the vast majority of apps you probably don't want things to start moving around on the screen when text changes?

Having things move around on screen when the text changes was pretty much the entire original motivation behind my 'trui' library 🙂

I think for most apps the question is how well they can predict the maximum size of each element - if each piece of text has a predictable maximum size they can calculate the layout once and leave it fixed, but if there's a chance of the text overflowing its original bounds that risks breaking the layout. This can especially be an issue due to localization, if the localized strings end up being longer than the app developer was imagining.

No, you can have more than one in layout.

I stand corrected!

Just merged your changes! I've got some tweaks to enable fillx/y with ints which I'll get in too.

But yes, I think ideally the layout library needs to handle both cases, but I hope in a lot of apps things will stay in the same place most of the time.

Just a quick question on lazy rendering, if I update a layout txt with the exact same text, will this cause a screen update on a render pass?

Just need to know whether I need to check elsewhere in the code if the data has changed before updating screen or if I can just set screen with all current data and have the lazy renderer decide what work actually needs done?

if I update a layout txt with the exact same text, will this cause a screen update on a render pass

As long as the contents of the layout element are identical the lazy renderer should be able to avoid re-rendering it.

I can just set screen with all current data and have the lazy renderer decide what work actually needs done

That's the idea :)

One caveat is that the lazy rendering algorithm itself has a small performance cost, so while it can help a great deal with reducing flicking on the Bangle 1, it's not always that much faster than a full re-render.

I have been looking into the layout library and to meet my aspirations I would like to add an option to have an imagebutton. I am happy to do the work and submit a pull request, but as this could be done in a number of ways I thought it would be a good idea to float the idea and perhaps get a consensus on how it is best implemented. In my mind I can see three options:

1. A new object type specifically for imagebutton
   
2. Extend the existing btn to accept an image source and/or a label
   
3. Extend the existing img to accept a callback on touch
   
   

I would be interested in any views/ preferences on how this should be implemented?

Hi! That sounds like a great idea - it's something I've been considering too.

I'd say option 2: Extend the existing btn

If you could issue a PR that'd be awesome.

Main reason for choosing Option 2 is the idea is to extend Layout such that on Bangle.js 1 you can use the up/down buttons to cycle through the selected buttons and then use the middle button to select (allowing even touch-based apps to be used with the original Bangle). It'll be a lot easier to do that if there's only one 'btn' type).

An even more general approach would be to allow btn to contain other elements as children.

@NebbishHacker in effect an equivalent of "v" or "h" but with a touch callback and possibly an outline round the whole thing. I guess the implementation would have "btn" contain a single layout object which could be a "v" or "h" to allow stacking. Certainly more flexible, but makes the simplest used case of a button with just a label more complicated to implement.

@Gordon how does this sound to you?

Well, in new Espruino firmwares (cutting edge, or 2v11 when released) you can actually put a binary image inside a text string like this, and alignment and newlines are handled correctly. There's a new 'stringMetrics' too which gives you width/height of a string (even including inline images) and I'll move the layout over to using that soon.

So in a way, you can actually do the 'simple' option right now - just set the label to the "\0" plus the image. If you want text under, do "\0"+image+"\nMy Label".

However probably one big use case is going to be toggle buttons - so press once and the image changes to 'on', again for 'off', so probably something that allows you to change the image easily is better.

So yeah, maybe we just look for a c in the button object containing a single layout, and if so we make the size whatever the size of the layout object is inside it, plus some padding? It shouldn't need much in the way of code changes at all.

You can still do the 'quick' image icon, or can do it properly