ub.js, a Web of Things hub

Hi everyone,

I've spent the last (long) while designing and building some Node.js based software that works as a Web of Things hub. It's designed to connect lots of devices (built or bought) and services together using Web standards (WebSockets, TCP sockets, JSON, HTTP, etc.) as opposed to proprietary formats and APIs.

Why not Internet of Things? Because it's not about the underlying connections (WiFi, Zigbee, serial); they can be anything. ub is a layer on top of that.

The best feature is that its main client is based on Espruino! Espruino allows you to have JavaScript from the browser to the device, so messages don't need to be reformatted or translated to go from one end of the system to the other. It's also free, open source and actively seeking contributions.

That sounded a lot more like an advert than I wanted it to! It's part of my PhD project so while it works, it's not bug free yet. I'd be really interested in your feedback as it's primarily aimed at people like you, and was originally built for Espruino devices.

Here's the link and please let me know what you think: https://github.com/ub-js/ub.js

Thanks!

Uh, did I miss something? what does it do?

All the details are in the link (I hope). It passes messages between devices, so say you want to turn on all the lights in your house (for example), you can send a single message to the hub with a CSS selector in it to turn them all on.

It's deliberately very versatile. It's a framework and a few libraries to enable you to build large projects in a relatively simple way, in a way that is at least semi standard and open.

To explain a bit more, it has all the tools to make an Espruino (or Node.sj) based device like a light, which you then give an ID and a number of classes, like you would if you were adding an image to a Web page.

Then you can make a Web page or app that controls that light using the ub.js Web UI library, so all you have to do in your JavaScript is initialise ub.js and call ub('#light_id').css('power', 'on') and the light turns on. All the connectivity and messaging is taken care of by the Node.js app.

That kind of implementation is fairly simple for most people here to do (or at least figure out how to do), but ub scales well. So you could instead do ub('.light').css('power', 'on') and control thousands of lights if you wanted to. You can also do things like ub('.light:not(.ceiling)') to select all the lights not on the ceiling.

ub also uses formats, protocols and approaches which make it more compatible with the Web as it already is. Rather than connecting to a device, the device integrates with the Web as it is. It's a subtle difference, but I think it's an important one.

I hope that makes it a bit clearer. There are also a few working examples in the GitHub and I plan to add more when I can.

@AlexOwen cool stuff !

I have multiple ESP8266 and started to use MQTT for communication. Looks like ub.js can save me a lot of time. I will definitely try it.

Have you thought about a solution that needs no hard coded SSID and PASSWORD ?

Something like this: ESP starts as AP if there is no valid Wifi connection and is able to be setup by a station that connects and provides Wifi connection data ?

Sounds great - using selectors to control things is really neat.

Something that would also be fun (if you get time ;) is to add Bluetooth LE support (Probably NPM's bleat package). There are starting to be a lot more BLE devices available, and the protocol is rigid enough that you should be able to control a lot of proprietary devices quite easily.

@MaBe, I had a version that did that but it kept running out of memory. I think with the new 1600 jsvar build it might just work so I might give it another go.

@Gordon that's a good idea, thanks. I'm not sure if I have any BLE devices but I can probably emulate them somehow.

@AlexOwen like to run some test with this version - can you make it available on github ?

Wow, CSS(D) is pretty cool. It solves an issue with the hone home automation hub I've been building in node-red (selections for ui groups) in a very neat way. I might give this a try.

Have you used node-red at all? It's very good at making it easy to make connections between network and hardware protocols, but it's up to you to create the core that it's all talking to. Ub.js seems like it might make for an excellent core for my home automation stuff, an improvement over what I've patched together myself.

I'm honestly still having trouble understanding what this is, or rather, where it fits in... yet other people seem to get it and think it's cool...

Can anyone help me out here? Assume I have a location, and I've got some sensors and some devices that I want to control over the network. Presently, what I would do is put an ESP8266 or Espruino+ESP8266 node near each pile of devices, and program them to have a page that returns status as json, and takes commands via a GET request (eg, 192.168.2.19/ICBM/launch.cmd ), and then to tie everything together, I'd put a page on the webserver stuffed behind the bookshelf in the living room that brought all these controls together on one page, and drove everything with XHRs (using the PHP cache module to store semi-persistent information like when it last talked to sensors or whatever).

So considering that example, what would be different if I was using this?

As I see it, it provides a standard interface between all your devices. You don't have to manually create all the pages, commands, etc. Whenever you come up with a new sensor, you hook its io into ub, and then controlling it or reading from it becomes a one-liner from any other device connected.

@MaBe, I'll get it up once I have a chance to test it and see if it fits with the new build. If it doesn't I'll PM the changes to you so you can give it a shot yourself.

@the1laz, thanks, CSS(D) is a WIP for sure, but hopefully it's useful. I've looked at Node-RED a while ago, but it seems to have come on a lot since then. Thanks for the suggestion, I'll take another look.

If you end up using ub then please let me know, and also if you decide not to for any reason (even if it's one I can't fix. Any feedback will help immensely with my PhD (which is what this is largely in aid of at the moment).

@DrAzzy, it's like @the1laz said, it's designed to hopefully make things a bit simpler after the initial deployment while still retaining the ability to script everything to the minutest detail if you want to. It's also looking forward to when there are potentially hundreds of connected devices in an environment, in which case controlling them from a webpage becomes cumbersome. Instead you can control sets of devices which you can define using CSS selectors, which are already pretty standard.

It's also about integrating with the Web, so in your example you've created a resource (launch.cmd) behind a URI, which works for direct control, but with CSS(D) and ub you don't need to know that the resource exists. You can send a command to an environment and anything that understands it will respond and react. It's a loose vs strong coupling thing.

You can also easily to M2M so you never need to interact with the system at all.

An example of where it's good is one I have working in my room as we speak. I have a Web page where I can set a temperature range, then when the temperature drops below the lower limit, anything that has the 'heater' class turns on and warms the room up until it reaches the upper limit, then turns off. Because the devices are stored as DOM objects it's easy to add a CSS(D) property to say, for example, how powerful each heater is (e.g. power: 3000w). Then if I want to keep my energy usage below a certain level I can only turn on anything that has a 'heater' class and has a power level below 2 kilowatts.

However this also highlights one of the things that needs to be worked out with CSS(D), which is when I use the 'power' attribute do I mean on/off or a power rating, and can some objects be half on. As I said it's a WIP and any contributions to the code or design choices are greatly appreciated.

@AlexOwen I'm pretty keen to try it out, but I just finished building a 3D printer so... I might not get to it right away. :)

EDIT: Printing enclosure parts for sensor nodes running Espruino!

@MaBe it would seem I included the access point creation after all, I think it must have just about fitted after the increase from 1200 to 1400 jsvars on the ESP8266. The code runs from line 7 in ub.wifi.js and is called if the device fails to connect to the hard coded access point 5 times in a row. It'll create an AP which has a name the same as the ID of the device.

Currently you have to connect to the AP it creates, go to the IP for the device, and enter (in plain text (unfortunately) the new AP and password. It should save the new AP and password once you've submitted them on the page, but it won't survive a reboot.

However, in my tests I can't seem to connect to the AP of the ESP8266 once it's created. I don't know if it's a bug in the code or the hardware misbehaving, or (less likely) a bug in Espruino.

@AlexOwen

from 1200 to 1400 jsvars on the ESP8266

some ESPs at my side are running with 1v85.MaBe_master_db6edcd ( latest pull )
with 1700 vars, 10.000 heap, GRAPHICS and CRYPTO

(in plain text unfortunately)

crypto is now included

but it won't survive a reboot.

use Wifi.save()

a bug in the code or the hardware misbehaving ....

Will run some test on 1v85 to clarify that AP is working as needed.

Ooooh, we're up to 1700 JSVars on ESP? That is awesome!

Where can we find this 1700 JSVar build? Awesome indeed!

@Ollie This is no official release - just a travis build with the latest commit - so you are on you own !

espruino_esp8266_user1.bin 29-Apr-2016 10:17 450K

and this is it :

 ____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v85 Copyright 2016 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:512/512, manuf 0xe0 chip 0x4016
>process.memory();
={ "free": 1678, "usage": 22, "total": 1700, "history": 0 }
>require("ESP8266").getState()
={
  "sdkVersion": "1.5.0",
  "cpuFrequency": 160, "freeHeap": 10160, "maxCon": 10,
  "flashMap": "4MB:512/512",
  "flashKB": 4096,
  "flashChip": "0xe0 0x4016"
 }
> 

Thanks @MaBe, I'll catch you in Gitter to ask you about it, rather than hijack this thread :)