Espruino-red: running node-red nodes in Espruino

I've been using node-red for a while to act as the "server-side" component of various Espruino nodes I run, but I've been wanting to try and integrate things further, which is is to run actual node-red nodes on Espruino itself. That's espruino-red. If you're interested I wrote a long post on the node-red group: https://groups.google.com/forum/#!topic/node-red/pJXwdl2UMQ8 and there's a pretty detailed readme in the repo: https://github.com/jeelabs/espruino-red
Warning: it's all a proof-of-concept that this stage (but it does really work).

I have been dreaming about this type of visual program composition applied to embedded systems for a long time and I want to try it out to see whether I actually like to use it in reality or whether it's just a misguided dream.

(...quote from node-red group post).

There was a time where this was a great hype... 90'... and it faded, but I still believe it has great value. What I refer to is VisualAge for XYZLangue - an application (programming and generator) product series from big blue IBM.

Visual linking - programming - is great when the interfaces are nicely defined and the linking can generate proper plumbing code. Not so fun is it when timing and circular references come into play. It get's pretty complex - almost like garbage collection - to avoid racing conditions. See attached graphics for a taste of how it looked (taken from IBM Redbook Using VisualAge Smalltalk ObjectExtender, p. 327 (.pdf: p.353).

First attachment has decent complexity, following ones? ...I don't know... :|

It WAS fun! Placing graphically (with the mouse / track pad) a connection triggerd a lot of plumbing in the background... I like to see some of this coming back!

3 Attachments

What I like about node-red's approach is that it's quite high-level plumbing. You don't write algorithms visually, instead those go into "function" nodes as code text. So what you end up visually is large components/modules that you plumb together. It makes it easy to take some value floating around in there, such as a temperature, and wire that up to some new nodes that record it, alert on extremes, chart it, plop it onto a dashboard, etc.

Absolutely... it works well as long as there is no - or only little - dependency between between the individual components.

As soon as flow / sequence is involved, simple scripting / programming is much easier to grok than graphically presented cascaded, dependent event-action links.

Looks great! I've been interested in this kind of thing for a while (I really like the dataflow idea when used in the right place). Node-red's something I was planning on doing stuff with on the Puck.js hub as well.

So at the moment, you have a bunch of yellow espruino nodes, and all of those nodes run on Espruino itself - connected via MQTT?

I don't know how possible it is, but did you consider having just Node-red -> Espruino and Espruino -> Node-red nodes? Then you could decide where the nodes ran based on which side of one of those nodes they were on... It might save you re-implementing a bunch of nodes, specifically for Espruino?

So at the moment, you have a bunch of yellow espruino nodes, and all of those nodes run on Espruino itself - connected via MQTT?

Exactly. The nodes on espruino communicate with nodes in the node-red process (and with nodes on other espruiono boards) via MQTT. Local communication just uses function calls or setTimeout(.., 0) calls to break the tail recursion.

I did consider gateway nodes but I found that I didn't really need that and that it's more elegant to do some magic where the wires are connected internally. I did not consider running arbitrary nodes on espruino for two reasons. One is resource consumption: I'm trying to keep the espruino side small. The other is that I don't see the use for it (I'm probably blind). I went down the sidebar in node-red and asked which of the std nodes I would run on espruino and I came up pretty empty. If I take the example of a DHT21 temperature sensor, do I really want to be able to write one implementation of a node that uses two different DHT21 modules which happen to have the same interface but are implemented very differently in espruino vs node-red? Maybe I'm missing the point?

Yeah, very true. There's probably not too much you want to carry over - and as long as you have an arbitrary script box for Espruino you could work around anything anyway.

This reminds me of remote I/O for industrial control systems, some kinds can manage their own logic when the main processor gets disconnected. Very cool seeing this implemented on inexpensive microcontrollers and open source, user-friendly software.

Just having a look at your code, it shouldn't be too hard to store the node list in flash so as to be able to reload it without the connection to node-red.

Yes. That's correct.

@allObjects, this reminds me of Objective-C and Next Computer's graphical IDE. That was a time when this sort of methodology for application development was all the rage ('90s). I seem to recall some SmallTalk IDEs that also tried to get actual code-writing completely out of the picture.

What failed, and still does, is that none of these schemes simplify away the underlying engineering complexity of any real application, in which case the "difficulty" simply shifts from coding to interface definitions and protocols. Having been down this path, one soon finds that any "real world" application of any meaningful functionality and complexity ends up being just as much work -- or more -- to architect, specify, and implement interface and protocol specs, finding that in many cases it's "just easier" to fall back to actual code (C, C++, O-C, Java, whatever) and bang out something concise and efficient.

It's why these methods faded from the scene.

IMO, the real revolution for software engineering will come with AI, not IDEs and various alternative ways to spec program algorithms, which is really what this sort of thing is. Rather, when we have AI that we can simply describe a problem to, perhaps in some formal syntax, better if we can just talk to it, and it spits out the program.

That would free the designer to focus on features, function, behavior, etc.

The other domain that I think is woefully under-invested-in from the standpoint of software development is failure/defect detection and debugging. So little brainpower, relatively speaking, has been spent on improving and enriching this aspect of code development.

A lot of brainpower has been invested in software testing over the past years. Developing software is very different today from a few years ago, IMHO.

What I like about node-red is that for relatively simple things one can compose visually and that can provide a nice overall overview and structure to what ends up being a bit of a distributed application. It's not a place to do the coding, but one to glue things together and establish the overall flow of events & data. In particular for beginners it's nice environment to try things out quickly.

I'm not sure that node-red is really used for "application development". I'd actually consider it closer to something like ladder logic (which definitely isn't fading from the scene). It's more about having some inputs and outputs and visually and clearly defining the logic connecting them, rather than trying to skip out on coding.

Components with 'Smart connectors' - well defined API's with machine readable meta-data about static and dynamic aspects - allow a platform to do quite some useful plumbing. Almost like advanced code completion in IDEs, circuit design in CAD/CAM, etc, and when there are options/ambiguities, the platform can ask at composition time the user interactively for selection.

Not sure where you are running the mqtt stuff, this is an interesting article using docker on the raspberry pi

https://sample.me.uk/blog/post/rpi_docker_iot?mkt_tok=eyJpIjoiTVdNNE5tSTJPV0l3T1RWaiIsInQiOiJxcmdrck1JRmVLZUpDbVwvNGtuMUZHRTMwRUhcLzY1Zm9CZTY1MDgrWUZONFVUbEEzVUl0MmFpNEt0dmdJbnNlTzhzQXkzSjFWMk5aWGVTWm1McFZqUElsYkx5WlwvYmlNS0hNSXo0OTRTdk1Yaz0ifQ%3D%3D

Interesting! I'm not convinced I want to add docker to my odroid set-ups, but I've thought about it. This is the best article I've seen in terms of explaining how to get docker, thanks!