Micro Chat (in progress)

I've been playing around with building a chat ui for connecting to Espruino boards.

https://benjaminbenben.com/microchat/

It's not quite ready, and it's broken in a few ways but I wanted to share it so that I don't disappear down a rabbit hole with it!

When you add a device and it's like a "contact", and then you can start a session which feels like a "chat". Currently I'm showing a terminal session in the ui, but I've got plans to make it more message based (I've got a pretty neat wrapper for rpc functions).

Just now I'm showing the device name, but I'm planning to make that editable so you can have "My Pink Neopixel Puck" and some notes about what you've got attached. Because it's url based, you can bookmark/link a particular device.

So far, the UX seems to work pretty well. It feels quite natural to browse devices without opening up the BLE dialog. And when you decide to connect, I'm able to limit the dialog so it only shows the device you're looking for. Reconnecting on reload feels pretty natural. It sounds like there's some device api in the pipeline that might make it even cleaner.

I'm thinking this UI might give a pretty good route for extending beyond repl interactions. You could have something like slack slash commands to perform more complex tasks.

With the implementation, I started with with puck.js library, though I ended up extracting bits of that into my own Socket class so that I could separate devices from creating sockets. It's a bit hairy at the moment, but seems like it's going to work okay.

2 Attachments

I really like the UI. Is that custom made or some library? I need to learn more about css. Checked the functionality and it is nice not needing to reconnect. It would be nice to store the connection somehow even after the browser is reloaded. I wonder if thats possible.
Whats the goal of this project? Is it easy communication with multiple devices?

Thanks! The UI style is from TailwindCSS, and there's some React code for managing the state of the app.

There's a persistent connection behind a chrome flag which I think would allow it persist when reloaded, but I've not looked into it yet.

The goal of the project is a bit odd (I wasn't sure if this was the best place to post it in fact). I started off making a remote control for my TV, then I liked the idea of locking to a device, so had a bit of a hack and ended up with this.

Bit more hacking/fixing, you can set the name of devices now too.

1 Attachment

Nice! That looks great! I wonder whether you could do a string replace on certain emojis, so 💡 could go to LED.set() and similar?

This would actually be really cool for robots - you could connect with the phone, then send commands like 'forward'/'left'/etc or even just the emoji arrows. My son would love it :)

But yes, there's a navigator.bluetooth.getDevices API (iirc) where you can get previously connected devices which you can connect to without the BLE menu, so that could actually be a pretty neat addition. The IDE uses it - I didn't think it was behind a flag but maybe i just turned the flag on and then forgot ;)

But actually talking of Slack, I guess potentially you could have something where you sign in to slack on an old Android phone, connect your Espruino device, and then you can just type @phone turn XYZ on and it'll magically do it when you're at work (I'm not sure if slack has an API or you'd need a bookmarklet though?).

Thanks @Gordon!! (Hope all is good!)

Oh man, I do love a good emoji replacement. I'll have a think about that! 🤔

For something like the robot commands, I'm wondering if there's some approach like telegram bots where you could display buttons in the chat dialog. Though maybe your son might prefer typing it!

Ah great! I'll enable the flag and play with navigator.bluetooth.getDevices.

With the IDE, is there a way to link to it and have it connect to a specified device? I'd love to add a button from this ui which is "open this device in the IDE". If that's not possible, is that something I could add a PR for?

Oooooo, slacking a puck could be amazing!!

Wow, the telegram bot thing would be cool. I guess there could be a library for Espruino that allowed you to do prompt("Light on?",{buttons:["yes","no"]}).then(result => ... ), and you had some magic that detected what it output and put buttons on the chat dialog?

My son's still at the 'poke the picture' stage, so the easier the better really :)

With the IDE, is there a way to link to it and have it connect to a specified device?

Interestingly no, there isn't! There's already 'upload', and it's smart enough to connect to the emulator:

https://www.espruino.com/ide/?emulator&upload&code=print(%27Hello%27)

But having the ability to specify an actual device would be very cool, and a PR would be great! In getPorts there's already the ability to specify a port that the IDE should automatically connect to, so it might be as easy as modifying getPorts in serial.js to check window.location.query and set the relevant field if the name matches :)

Also, we currently only have upload, and not a connect option :)

Just to add, we made him a little 'perseverance rover' and the weekend, so finding a way to control it is on my mind at the moment - although it'll probably just be a modified version of https://banglejs.com/apps/#smartibot

I just got carried away with the Slack Bookmarklet (seemed the easiest solution): http://forum.espruino.com/conversations/360430/

With the IDE, that's great! Thanks for the pointers, I'll have a dig and try and raise a PR. upload seems fine for my use case, I could just print "Hello from MicroChat" or something.

TIL about the emulator - that's amazing! wow.

With the prompt & buttons. I've been playing with a (sketchy atm) host/puck rpc wrapper. It's pretty similar to Puck.eval(…), but it lets you do async listeners and handles errors.

It lets you write browser code like:

await rpc('new Promise(resolve => setWatch(() => resolve(), BTN))')

console.log('Battery level: ' + await rpc('E.getBattery()'))

try { 
  await rpc('E.getButtery()') 
} catch(e) {
   // Error - getButtery is not defined
}

I'm still playing around, but I'm hoping that it'll make it easier to spread code between a page & microcontroller. So for chat button prompt for "light-on: yes/no" could be in react/telegram/wherever.

The other option I explored was puck = new PuckWorker('puck-script.js'), which would feel like a web-worker, but be running on a puck. You get a nice pattern for messaging, and puck-script.js would be espruino-only code, which might make it easier for more complex usage. Though I guess it's a bit of a different usage to having longer-lived scripts running.

Aw, his own rover sounds so much fun! (Perseverance is amazing 🤩).

Gonna check out your bookmarklet!

Nice! actually I guess the Puck.js library could be extended to return promises - it'd make an awful lot more sense. Although running promises on the Puck itself too is really neat, and neatly wraps around any calls to Espruino libraries that use Promises internally.

It was always my intention to make UART.js (that thing I started in the Oxford hack day) use Promises, and to generally fix the dodgy APIs inherited from Puck.js. Sadly I never got around to it though :(

The PuckWorker is a neat abstraction - although as you say maybe it actually makes things a bit harder as you've now got two separate files.

Just a random thought, but it seems this works:

function getBattery() { 
  return E.getBattery() 
}
Puck.eval("("+getBattery.toString()+")()")

So you could actually write JS code for Espruino alongside that for the browser (without having to encase it inside a string). It's nice from the point of view of syntax highlighting and linting anyway.

Ah yep, a promise interface for the Puck.js library eval could be sweet - and I guess it would work alongside the callback option, so wouldn't break anything. Is that another thing that I could PR? I've never contributed code to Espruino, and I'd really like to.

I guess the slight difference with the wrapper is that I can run async tasks on the puck (like waiting for button presses/states), I'm not sure how I'd do that with Puck.eval.

Ah, UART.js seems where my PuckSocket thing is headed. I'll take more of a look. Have you done much with the streams api? It seems a slightly odd api, but might fit pretty well.

Ah, that function serialising is cool! The result reminds me a bit of comlink.

Is that another thing that I could PR?

Yes, absolutely! That'd be amazing! https://github.com/espruino/EspruinoWebTools is where Puck.js and UART.js are hosted...

The library actually doesn't handle errors all that well, so it'd be amazing if it could for example reject Process.eval's promise if invalid JSON was returned (eg an error message).

But actually UART.js might be the thing to improve. If the connection type could be set explicitly in order to avoid the popup (eg if you don't care about serial) I feel like it'd be the best thing to point new users at (if only because Puck.js as a library name is super confusing if you don't have a puck!).

I can run async tasks on the puck

Yes, that's a good point. It almost looks from how you use rpc that it could replace Puck.eval, but maybe it's better to keep it as a separate function, if that were something you wanted to add?

The Puck library is definitely missing a nice way to handle getting async data easily - eg from sensors where you might have sensor coming in all the time... Even a Puck.on('x',callback) that actually added an event handler inside Espruino to E.on('x',...) would be nifty.

Streams API could be a good thing to implement - especially where there's the 'connection' object. The write/eval stuff works well for simple sites and is a good way to get started, but it might be worth having something more flexible.

Comlink looks neat! Actually I wonder if you could go one step further and have a general purpose worker script that you could include, and then just call with the function that you wanted to be executed in a worker...