Shiny Web UIs

Yeah - I thought he was talking about the ESP8266 when I said that. Sorry about that.

Yes, of course can Espruino deliver just everything (for example, over http). An interval 5+ seconds gets a lot done... But when it comes to 'higher' frequencies, ...because of the threadedness I was thinking about a combination of just from local (Plain Browser over HTTP/sockets or WebIDE-like Browser plug-in) or a WebSite (allowing CORS). Quite a while ago same topic came up and posted solutions where quite nice... (offloading Espruino and let Espruiono just do what it does best).

I'm Mike from ThingStudio, and I'd just thought I'd weigh in with our view of how to do IoT UI's

-We've opted for MQTT as a first transport (which is already in place, btw), as it seems to have the most traction and the correct symantics for IoT.

-This also means that we aren't in the business of creating special libraries for every platform out there.

-We also decided on JSON payloads for MQTT, as this gives us the ability to send and recieve objects in a single message, and display them simply in the UI.

-We also opted for HMTL5 as the best fully cross-platform UI tool. Mobile is really, really important, but I also need to be able to make 'Mission Control'.

-Being able to share and publish apps is important. We do that to some extent already, but we will have fully shareable webapps shortly.

-Unlike, say, freeboard, we can work with or without an intermediate logic layer, such as node red.

-We are currenly deficient in pre-styled widgets, and we are working hard on that. However, you can already create ANY widget of your own with just some simple HTML. We believe that while it should be easy for you to get up and running quickly with professional looking widgets, you should be able to create your own widgets, which is why we started there.

-We think you should be able to control and monitor multiple devices on one screen, or have many people view on device. This is inherent in MQTT, but just saying...

Security and privacy are important, so we don't, and can't see your data, that is run on a websocket between your MQTT broker and your browser.

That's about it, if you like what we say, go kick the tires, http://www.thingstud.io, we are still in Alpha, but working hard...

Mike

Security and privacy are important, so we don't, and can't see your data, that is run on a websocket between your MQTT broker and your browser.

@mkarliner, does this statemet imply that there is some secure layer in place (or shell arround the MQTT broker) that manages the communication between the display device - browser - and the MQTT broker and as well helps monitor and control the devices that (can successfully) talk to the MQTT broker?

The communication between the browser and such a layer can be managed already now - becauyse the browser isn't as resource strapped as the sensor devices 'out there', but adding some of the same securing/privatising(encription) features to the devices creates quite a challenge.

I'm really comparing us to other approaches to IoT platforms where your MQTT (or other transport) traffic is sent up to a cloud service, which you can then access via API to get plots or stored data or whatever. All we do is serve up UI's and don't have any access to any of your actual IoT data. As far as encryption between the ThingStudio webapp and the broker, we support secure websockets (TLS), so that data can be protected. If you want to keep data safe and your devices don't have the resource to do encryption, then the advantage of our approach is that you can have your MQTT data going over a private network, just so long as it provides a outgoing gateway so you can access the ThingStudio servers for the webapp.

The attached diagram is probably easier to understand

1 Attachment

A wee update blynk.cc went live on App store a few days ago it crashes all the time :( & as @Gordon has said their API sucks (my words not his!) evothings.com this hasn't moved on much since I first discovered this a long while back.

This brings me to @mkarliner thingstud.io thought as Mark say's the UI isn't that sexy at the moment it's the easy is's to integrate too. All the services I've listed he's the only one who's bothered to come to talk to us.

@mkarliner one question for you? have you considered offering MQTT hosting as you've solved one part of the problem but left many with a fence they rather not jump e.g. setting up an MQTT broker and I can't find a free cloud service and HiveMQ is near 90 EUR a year!

Ic. So you are in the space of getting easy to use UI IDE with a given - but extendable - catalog of components and services for setting up implementions going. A clear focus. The UI components and solutions - even pulled from thingStudio - have though to communicated with MQTT and have to support CORS according to your graphic.

The ThingStudio components for communication with (customer) MQTT and DATA realm have to be able to satisfy push and pull needs... and the push has to happen on arrival of new data from devices and storing in the cloud.

Did you take a look at Bayeux Specification - and language bindings?

Bayeux is a bit dated, but did quite recently a very good, fast, and secure job for me, easy to understand, easy to implment in the apps - jsavascript client-side / java server-side - and uses just solid plain http(s) technology requiring NIO on the (web/application) server side, though. Even though the spec is somewhat dated and NIO is not new as well, some of the top middle ware provider only just 'recently' implemented NIO support accross their middle ware http(s) protocol connected servers. I other words, the things may communicate through what ever channel and means to get the data stored in the cloud using MQTT, but the UI - which is more or less a reporting ((single page?) Web) app - does not need the use of MQTT versatility: two plain http(s) connections and available data access in NIO server do the pub-sub job just fine and secure as any other secure Web app - and yielded for me in 2010 subsecond response times from client to client, on small server, 'low' bandwidth, connected hardware without any tech stretch (yetty and Java, no Web sockets, just https).

To not go for Java, node.js as server environment may work as well, don't though not know how NIO pattern works there, because I cannot see any kinds of goSleep() nor * on(event,wakeup())* for that purpose (push()). node.js' option of multiple cores does not help, nor OS level multi-threading. http://jxcore.com - can handle such constructs and is in addition not Google Chrome V8 restricted - a huge factor in enterprise wide decision making. Note, NIO is not to be confused with threads from node.js thread-pool to move on serving other requests while waiting for, for example, data returned from a database, and then resuming this thread and completing the request.

What the hell is MQTT, and why is it any better than just firing off HTTP requests like the whole world understands?

For a supposedly hot technology, it's got a mighty thin wikipedia entry. Is anyone actually using it, or are startups just building ghost-town IoT platforms with it?

@DrAzzy agree about http. But I've come across a lot folk who can do embedded but don't know where to start with building web apps. I've helped quite a few in the electricimp & node-red communities do it and it would be great to point them at a simple App builder that talked to their device.

MQTT was invented by IBM to allow constrained devices talk to industrial SCADA systems. It's been around for ages & is widely supported. One reason I assume Gordon added a library for it.

@DrAzzy it's a publish-subscribe thing... So you connect to an MQTT server, and can push an ID+data, or can choose to subscribe to an ID, in which case you get sent any data that arrives for that ID.

I guess it's nice in that there are already MQTT servers with debugging, so you can pretty much just focus on doing the embedded side of things.

My main gripe is it's designed as a lightweight protocol, but there's no support for encryption. Their answer is 'just do it over TLS' but then suddenly there's a massive overhead and you might as well have just been sending human readable JSON :( End result is you end up having a separate MQTT server on a secure network that can then encrypt and store/push the data somewhere else.

Now if someone did a magic box that bridged secure MQTT over TLS with MQTT on a low-power (one-way) radio then I'd be really interested. But because MQTT is totally insecure I'm not sure how that'd work - when I've asked they just say 'use a secure radio' :/

@mkarliner yes, thanks for getting involved. So in your system you have a server on your local network that takes in MQTT data from something like Espruino, and that then serves up the UI? Or do you serve up the UI from the internet that then communicates with the local MQTT server?

edit: Sorry - I see what happens now. Local MQTT server with Websocket support, and then the UI served off of your server. Quite a neat idea.

To be fair Alex from Evothings did send me a one-sentence e-mail... He said if I sent him a free 'Espurino' board he'd add an example. Problem is, looking at their examples I'm just totally unconvinced their stuff does anything useful. For instance this one.

So you've got all that code, you have to download an app, program it all up with their custom API, manually enter an IP address and actually you could have just done it all from Espruino without any apps with:

function onPageRequest(req, res) {
  var a = url.parse(req.url, true);
  res.writeHead(200, {'Content-Type': 'text/html'});
  res.end('<html><body><a href="?led=1">on</a><br/><a href="?led=0">off</a></body></html>');
  if ("led" in a.query) digitalWrite(LED1, a.query["led"]);
}
require("http").createServer(onPageReque­st).listen(80);

I think as @LawrenceGrif says, an MQTT Library for Arduino/Espruino does give people who don't know anything about web technologies a good way to push their data to a server.

While I guess quite a lot of developers see making a webpage that pushes data into a database as a 10 minute job, for someone who has only ever done embedded that's actually a pretty steep learning curve.

Getting back to UIs though, it feels like the sentiment here is "UIs are easy, the hard part is all the communications under the hood". I'm totally unconvinced by that - if anything what I see from Blynk, Thingstudio, Evothings, etc is that even people who have a product whose whole focus is the UI still don't have very professional UIs (sorry Mike).

I guess the argument is "well everyone's going to want their own customised UI anyway" - but it seems to me that you have to be a HTML/CSS/JS ninja and a graphic designer to actually do that.

@Gordon thanks for the heads-up and very thoughtful reply. Agreed about needing front end ninja skill. In many web agencies you should see the bun fights between UX/UI and Devs. over pixel perfect UI elements!!

As for coms from browser to device having done Ajax polling for web UIs for years, web sockets have been a joy and rock solid & personally I would go back to Ajax but that said it works.
And extending the http server for web socket support is beyond me!

Secure browser to device connectivity is a must so hears hopping HTTPs isn't too long in coming.

Lawrence

Hopefully WebSockets will come - it's just prioritising work at the moment.... With HTTPS it'd be a necessity given the time taken to negotiate keys - AJAX is about usable for most stuff at the moment, but it definitely won't be over HTTPS! :)

I don't think that we'll be doing MQTT hosting. It's really a core part of what we do not to see your data. However, you have the following options I know about:
-CloudMQTT have a free tier. If you use it, remember that they only support secure websockets, so select that in our 'connection' properties.
-If you are using a Mac, we have created a desktop app that has an MQTT broker included, as well as a serial <-> MQTT bridge for using ThingStudio with plain 'old Arduino's. http://blog.thingstud.io/recipes/how-to-­use-a-plain-ol-arduino-with-thingstudio/­
-I've also written a blog post about setting up a Raspberry Pi with the latest Mosquitto + Node Red. This is a really sweet combo. http://blog.thingstud.io/recipes/how-to-­make-your-raspberry-pi-the-ultimate-iot-­hub/
On windows, Mosquitto supply a pre-built binary.
You can also try the open server at http://test.mosquitto.org/

All of which pretty much explains why we are not providing MQTT :-)

Blog posts on : blog.thingstud.io

Mike

Gordon, as I said, I'd be the first to admit that our UI is not great at the moment, we been concentrating on the architectural aspects of ThingStudio so far. Certainly, UI's are hard, even when using a great service like (ahem) ThingStudio, there is still a lot of work to get a really good UI. Reading this thread inspired me though, and I've started work on the widget system, so here is a teaser image of yesterday's prototype of an SVG widget made in ThingStudio. We'll be providing a set of widgets, but unlike everyone else, you'll also be able to make your own, and we hope the community will take up the challenge. It's still rough, but you can see where we are going. We will also shortly have a new, really spiffy UI for the actual IDE as well.

1 Attachment

I totally get your point about MQTT - did you ever consider just selling a Raspberry Pi with MQTT installed? It might not make you much money but it'd be a good way to get people started with Thingstudio.

The dial looks neat. With the SVG widgets, if Thingstudio could actually export one SVG with the JavaScript needed to drive it, that in itself would be amazingly valuable (even without the websocket code).

Yep. You've got it. I spent about 30 minutes with iDraw (£16) to make the dial, pasted into a ThingStudio screen, and inserted the helpers to animate it. I'm an SVG noob, but it took me about an hour in total. The code to drive it is two lines with ThingStudio. You can actually do it today on ThingStudio, but we don't support these as libraries, you have to paste in the code each time, that's what the new version will allow. I just pasted this into one of my screens on the current version http://www.thingstud.io/screens/MTmhk7Lw­iN58ijKZP, (you'll need to signup to access it), the second slider down will move the pointer.

http://thethingbox.io/ do an SD card with a preloaded node-red, node, and mosquitto, but so far I've not been able to persuade them to upgrade to Mosquitto 1.4, which supports websockets. :-( . I suppose I could go into competition, but I'd rather they just upgraded their build...

Shniny UI - click me to see me alive, what about this? ...adding something to the wheel helps a lot.

Copy below code - or - get it from attached link, save it to your disk as .html, open it in your browser. It works even from file:// protocol. Modify it to display your Espruino captured sonsor data.

<html>
<head>
<title>Shiny UI</title>
<script src="http://ajax.googleapis.com/ajax/lib­s/dojo/1.10.4/dojo/dojo.js"
        data-dojo-config="async: true, parseOnLoad: false"
    ></script>
<script>require(
	[ "dojo/_base/kernel"
	, "dojo/dom"
	, "dojo/parser"
	, "dijit/registry"
	, "dojox/dgauges/components/default/Circul­arLinearGauge"
	],function(base,dom,parser,registry,gaug­e){
	parser.parse();
	var gauge = registry.byId("gauge"), date, value;
	setInterval(function(){ value = (new Date().getSeconds() * 10);
			value = (Math.floor(value / 60) % 2)
				? 60 - (value % 30)
				: value % 30;
			gauge.set("value",24 + value);
		},1000);
	});
</script>
</head><body>
<table><tr><th>Shiny UI<br>Temperature with
	<a href="http://dojotoolkit.org/reference-g­uide/1.10/dojox/dgauges.html"
		>dojo Gauges</a></th></tr><tr><td>

<div    id = "gauge"
        data-dojo-type="dojox/dgauges/components­/default/CircularLinearGauge"
        value="32" minimum="-50" maximum="150"
        style="width:15em; height:15em"
        interactionArea="none" animationDuration=500
    ></div>

</td></tr>
</table>
</body>
</html>

Key in above code are lines 29..34 which declaratively - with plain html - get this gauge defined. But there are also ways to defines gauges programmatically.

Replace lines 16..21 with some push/pull mechanism to get some real data involved.

3 Attachments

• shui.html
• 
• 

@allObjects neat find! dojo looks like a very interesting framework.

Have been googling SVG based widgets there are quite a few libs and this one for knobs http://eskimoblood.github.io/jim-knopf/

And for a bit of fun http://bl.ocks.org/brattonc/5e5ce9beee48­3220e2f6 NB. hit refresh once page loads

And I would love to create something like this http://demo.hmi-project.com/system/ the HIM Project are the goto guys for industrial HMIs.

Yep, dojo is an extremely powerfull set/toolbox of fully integrated frameworks, with full class hierarchy support and many other things that make it an enterprise strong javascript/client/(and server - on node.js) platform. Today's defacto standard 'require.js' IS dojo's original loader/dependecy manager/loader cache (a-la Java classloader). dijit - an many dojox - packages host a sea of widgets. dojo was born about the same time as jQuery, but going fully object-oriented (and keep functional approach where appropriate - hybrid) and M*VC w/ data/object stores it was over the head for usual Web page developer... dojotoolkit is co-built by Alex Russel, now member of Chromium projects (Chrome Browser, Chromium OS).

Btw, if you omit or set interactionArea="gauge", you have your knob at the same time...

Other things I use - for Web mobile - from dojo. Nice show case with source examples...

...and with some real-world data - click me to see current temperature at SFO and another nice example (below code). dojo gauges are composites of backgrounds, scalers, scales, range indicators, and value and text indicators that can be customized in all kinds of ways. When thingStud.io is coming up with a nice Editor/Composer tool of some kind, that would be helpful for end users.

Copy below code - or - get it from attached link, save it to your disk as .html, open it in your browser. It works even from file:// protocol. Modify it to display your Espruino captured sonsor data. Running an HTTP server on Espruino and returning a javascript (callback) costs only a few more bytes than a pure JSON response, but you have not to serve the file nor deal with CORS challenges, and best of all: you can serve the bulk from local just off file. On the other hand, using a (local) HTTP/Apache gives you additional flexibility in modularization and reuse...).

<html>
<head>
<title>Shiny UI</title>
<script src="https://ajax.googleapis.com/ajax/li­bs/dojo/1.10.4/dojo/dojo.js"
        data-dojo-config="async: true, parseOnLoad: false"
    ></script>
<script
<script>require(
	[ "dojo/_base/kernel"
	, "dojo/dom"
	, "dojo/parser"
	, "dijit/registry"
	, "dojox/dgauges/components/default/Circul­arLinearGauge"
	, "dojo/request/script"
	, "dojo/domReady!"
	],function(base,dom,parser,registry,gaug­e,script){
	parser.parse();
	var gauge = registry.byId("gauge"), time = dom.byId("time"), f; 
	var update = function(){ // bbox=minlong,minlat,maxlon,maxlat
		script.get("https://aviationweather.gov/­gis/scripts/MetarJSON.php",
			{ jsonp: "jsonp"
			, query: "bbox=-122.3745,37.6185,-122.3735,37.629­5"
			, preventCache: true
			}).then(function(data){ f = data.features[0];
				time.innerHTML = new Date(f.properties.obsTime);
    			gauge.set("value",Math.round((f.properti­es.temp * 9 / 5 + 32) *10) / 10);
			}, function(err){
				if (confirm("Error! - Retry?\n\nError:\n" 
					+ err)) { location.reload(); }
			});
    };
	update(); setInterval(update,600000);
	});
</script>
</head><body>
<table><tr><th>San Francisco Airport - &deg;F<br><span id="time"
	>Loading...</span></</th></tr><tr><td>

<div    id = "gauge"
        data-dojo-type="dojox/dgauges/components­/default/CircularLinearGauge"
        value="-50" minimum="-50" maximum="150"
        style="width:15em; height:15em"
        interactionArea="none" animationDuration=500
    ></div>

</td></tr><tr><td>
	by <a href="http://dojotoolkit.org/reference-g­uide/1.10/dojox/dgauges.html"
		>dojo Gauges - dojotoolkit.org</a></td></tr><tr><td>
</table>
</body>
</html>

2 Attachments

• shui3.html
• 

Thanks for the pointers guys. I just found jim-knobf myself a couple of minutes ago.
The issues with widget sets are:
1) You can't please all of the people all of the time. Thus selecting one widget set and it's associated API's is not on.
2) We certainly don't want to bloat the system by including large widget libraries of which only a fraction will be used.
3) We think people should be able to make their own.
Thus, we'll be using the graphic elements from things like Jim Knobf, and inserting the real-time update code from ThingStudio, which is minimal, into them. This facility will be available to anyone using ThingStudio, not just us.

2) We certainly don't want to bloat the system by including large widget libraries of which only a fraction will be used.

It all depends how things are built... Any system that does not implement a dependency driven loader, is not modularized along the dependencies, and has no builder, does certainly pull in more then what's needed.

Nobody goes to a (real) library and brings all titles home. A proper (AMD/dependency/loader/builder) setup pulls just the items of interrest at that particular scope of runtime.

Thanks for posting those links up - they look great. Definitely the kind of thing I was thinking of :)

It looks like they could be hard to serve up completely from Espruino, but then that's not the end of the world if you have an internet connection.

Completely agree. However, many systems have some minimum overhead and the granularity of includes varies. We are actively working on trying to get sensible widget support, but I think you'll agree it's not a particularly easy decision.