Touchscreen resolution / coordinates

Hi all,

I'm currently working on a smooth snake game for the lovely bangle watch, and with most games, precise controls are key! I'm using the onTouch event to change the Snake's direction, but it felt weird, and then after debugging I found out that the maximum x value I can receive in the onTouch event is not 176 (as you might expect due to the screen resolution) but 191.

Now I wonder what is the actual resolution of the touchscreen, and is there a function to fetch it dynamically (so I can develop the game device-agnostic)?

Thank you so much! Happy to join the community!
~momo

Hi - that is interesting - the values should be scaled such that it matches the screen - if you plot pixels where you get the event they should be roughly under your finger?

So I'd expect 176x176 but I guess you just have to deal with occasionally out of range values. I can always add cropping in the firmware

Might be related to my post, I can definitely confirm the 191 as max outmost coordinate.

http://forum.espruino.com/conversations/371220/#comment16336127

I've added

console.log(tap.x,tap.y);

after

Bangle.on('drag', function (tap) {

in https://github.com/espruino/BangleApps/blob/master/apps/tinydraw/app.js and receiving coordinates from x=8..191 and y=5..188 .
Maybe some kind of "touchscreen calibration" is needed...

Awesome input! @Poolitzer I didn't see your thread, otherwise I would've added it there. It's definitely interesting what @HilmarSt added as well, because 191-8 = 188-5 = 183 (!= 176 but at least the same for both coordinates)

I think this is best fixed in the underlying system, shall I create a github issue or so? For now I'll hardcode the 8..191 & 5..188 then!

Btw did a little test myself with

Bangle.on('touch', (button, xy) => {
  console.log("coords are", xy);
});

and after spamming the touchscreen edges with my finger it looks like

x ∈ [1,191]
y ∈ [0,191]

Also tested, my ranges are:
x ∈ [0,191]
y ∈ [0,189]

range_x = [50,50];
range_y = [50,50];
Bangle.on('touch', (button, xy) => {
  
  if (xy.x < range_x[0])
    range_x[0] = xy.x;
  if (xy.x > range_x[1])
    range_x[1] = xy.x;

    if (xy.y < range_y[0])
    range_y[0] = xy.y;
  if (xy.y > range_y[1])
    range_y[1] = xy.y;

      console.log("ranges are", range_x, range_y);
});

How does the coordinate reported match up with where your finger is? Is it actually underneath?

My concern with clipping the coordinates is that the touchscreen is probably slightly bigger than the LCD - so you'd expect you might be able to touch around the edges.

If we clip this down in the firmware then it's kind of an own goal. It means that there's now less useful touch area - less scrolling, less ability to press around the edge of the screen, and less accurate swipe recognition (when that actually starts getting used).

If we clip this down in the firmware ...

I would not clip this in the firmware, rather map the 191 to 176 (or 175?).
Or do a touchscreen calibration if not all watches go up to 191?

I have some problems in the GPS time app when trying to touch the button areas on the right side: I have to touch the screen nearly in the middle to have a button press reaction, maybe the layout clips itself at 176?

So you're saying that if you use the 'TinyDraw' app and a stylus, what you draw does not appear under the stylus at all as you approach the right edge?

We do already map the touch coordinates into what seemed to be the right range on the watches I have (the displays reported 0..159 in each axis). Interestingly though, if you're seeing 0..191 then 191*160/176 = 174, so I wonder whether at some point SMA have updated the touchscreen controller firmware such that it gives the correct XY values, and now the adjustment I made previously is making the coordinates too big.

If that's the case then I guess calibration may be in order.

if you use the 'TinyDraw' app and a stylus, what you draw does not appear under the stylus

I've tested TinyDraw and it worked well (the paint does appear under the finger).
It seems that there's a touch sensitive area 176..191 outside of the LCD?

Is it possible to write a widget to show the actual coordinates or draw a "visual feedback" like in the developer options of android? That would help to understand the behaviour of the buttons in the GPS Time app.

Which IC and which bus (SPI / I2C) is used for the touch screen?

How does the coordinate reported match up with where your finger is? Is it actually underneath?

I've made a 3x3 launcher, and was misfiring the launcher on the middle column of apps, mainly because as my range on x was 0-191, the column was a few pixels (around 5?) more to the left, and another 5 or so pixels short, making it that the last 10 or so pixels was already for the next column.

Something like the attached image, where green was the drawn and expected grid, red was the touch feedback. Certainly not as precise but definitely noticeable, and when I compensated for it, it started behaving much better.

1 Attachment

It seems that there's a touch sensitive area 176..191 outside of the LCD?

Yes, this is what I was trying to say above. So clipping coordinates just removes a potentially useful area for scrolling, and scaling would mean the pixel reported was no longer under the finger.

Is it possible to write a widget to show the actual coordinates or draw a "visual feedback"

Yes - but I feel like if you're developing the app you could just insert a pretty much one-liner to do that yourself?

Which IC and which bus (SPI / I2C) is used for the touch screen?

It's software - there's another post on the forum quite recently about that - but if you're digging that deep you need to start looking at jswrap_bangle.c - you can't reliably access it from JS.

@diego please could you try TinyDraw and see what you find? What you're suggesting definitely seems like the coordinates need scaling, but it'd be nice to know for sure since it seems from @HilmarSt and my tests that they don't.

@Gordon just tested, and it worked fine. I don't have a stylus so you'll have to take this with a grain of salt, :P

After this, I've made a second test with the following codes:

Bangle.on("touch", function (button, tap) {
  g.setColor("#000");
  g.fillCircle(tap.x, tap.y, 5);
});

  Bangle.on("drag", function (tap) {
    g.setColor("#000");
    g.fillCircle(tap.x, tap.y, 5);
  });

What I've found is: drag is really consistent, touch has a few odd positions here and there.

Thanks - so I think we're saying that actually the coordinates are about right? The issue is that they go past the edge of the screen if your finger is towards the edge of it? Maybe that's just something that should be documented rather than 'fixed'?

The tap position is interesting though - the tap is reported from the touchscreen as a separate event. However, it's possible in the firmware the coordinates are getting overwritten with the drag coordinates, which might be messing with the position?

Yes, I think that's it: there is more touch area to the right of the screen that I can reach with touch or drag, but is seems to be definitely offscreen.

And then there is the other thing with touch, where sometimes the touch is a little bit off the touch point, that's what misguided me to think I needed to readjust the positions, some confirmation bias I guess.

I've noticed a similar issue here - the lowest y value (at the top of the screen) I'm able to get is ~5 (though anything lower than 7 is rare), and that's with me tapping the bezel quite a bit above the screen.

I think this might be causing me some issues with tapping on the Messages app icon that appears in the widgets toolbar

I can get all values from 0,0 to 192,192 (y=192 is hard to get, y=191 relatively easy).
For my fingers top left display corner is at 5,15 and bottom right at 176,185. So some "free" space round the display.
Subjectively the circles always appear a bit too far down.

I have just done an experiment with my stopwatch touch app.

I have changed my on(touch) handler to clip the xy.x and xy.y values to within the values of 0->g.getWidth() and 0->g.getHeight().

Bangle.on('touch', function(button, xy) {
  var x = xy.x;
  var y = xy.y;

  // adjust for outside the dimension of the screen
  if (y > h) { print("y adjusted to h"); y = h; }
  if (y < 0) { print("y adjusted to 0"); y = 0; }
  if (x > w) { print("x adjusted to w"); x = w; }
  if (x < 0) { print("x adjusted to 0"); x = 0; }

   etc

Remember when we create buttons we are going to do it on the basis of how wide / high the screen is. So this is on the basis of what is returned from g.getWidth() g.getHeight().

I now find my play button is a lot more responsive near the right edge of the screen.

I also see the following in the console when I click on rightmost part of the screen near the bottom.

 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v12.32 (c) 2021 G.Williams
y adjusted to h
x adjusted to w
x adjusted to w
y adjusted to h
y adjusted to h
x adjusted to w
x adjusted to w
x adjusted to w
x adjusted to w
x adjusted to w
x adjusted to w
x adjusted to w
x adjusted to w
x adjusted to w
x adjusted to w
x adjusted to w
x adjusted to w
y adjusted to h
y adjusted to h
x adjusted to w

To me it looks like on(touch) returns coordinates outside the values returned by g.getWidth() and g.getHeight(). I would argue that clipping is the correct thing to do.

I would never expect ontouch to return an x or y outside of the size of the screen reported by g.getWidth() and g.getHeight().

Sorry for the delay, I missed this. There was a big discussion about this a few weeks ago and it's tricky as you don't really want to reduce the usable touchscreen area (especially where gestures/scrolling is concerned) - but as you say apps expect coordinates in the screen range.

I guess maybe the thing to do is to add clipping for 'touch' coordinates only (at least for the moment).

edit: I've added this to the latest firmware build now