Architecture: WiFi generic object interface

As requested by @Kolban.

Still not sure on a name, apart from network? As a starting point:

wifi.setPower(onOrOff, function(err) { ... })

Initialise WiFi, or turn it off. That could be handy?

wifi.connect(ssid, key, [options], function(err) { ... });

Connect to the given access point. options could be an object extra stuff like the security type.

The callback is called with err==null on success. Could maybe also return the IP?

wifi.getAPs(function(err, aps) { ... });

Call the callback with a list of access points, of the form aps = [ { ssid, enc, signal_strength, mac_address } ].

The callback is called with err==null on success.

wifi.getConnectedAP(function(err, ap) { ... });

Call the callback with the name of the currently connected access point. The callback is called with err==null on success.

wifi.createAP(ssid, key, channel, enc, function(err) { ... })

Create an access point with the given ssid, key, channel, and encoding. Encoding can be 0, undefined, "open", "wep", "wpa_psk", "wpa2_psk" or "wpa_wpa2_psk".

Example: wifi.createAP("ESP123","HelloWorld",5,"wpa2_psk",print)

wifi.getConnectedDevices(function(err, devices) { ... });

If if AP mode (with wifi.createAP), call the callback with the second argument as an array of { ip, mac } objects - one for each connected device.

wifi.getIP(function(err, ip) { ... });

Call the callback with the current address details: {ip:string, mac:string, ...?}

The callback is called with err==null on success.

2cts: I would not call the parm in the callback err. I'd rather call it somthing like status or info or ...

If returning 0 (zero), or null, or undefined as the only ok values, that works with easy check of negation. If something bad happened, an error object should be returned if there is more information than just a scalar. Returning a simple scalar for error and having to go back and ask for the error is an option, but it costs memory, if the error is not already an anyway-available status property.

Furthermore, I'd prefer an object over a list of parameters.

My comments should be taken with the grain of Espruino salt, because for a resource tight environment other considerations than just object-orientation apply.

@Gordon This is perfect ... exactly what I hope for from an architect. We can now start designing an implementation for this and see what gets uncovered as that progresses.

@allObjects Your thoughts are vital ... this is community stuff. I think I also lean towards and object with properties over parameters ... however ... I also am keen that if an object is passed in that is missing some required "property" that the implementation makes that known rather than breaks or quietly does nothing.

I am also of an opinion that callback functions that are passed an "error" indication ... that the error parameter be the last parameter in the callback function's formal parameter list. So instead of:

function(err, ap)

we would have

function(ap, err)

Of course if there is some existing standard or convention in Espruino, I would follow that and then if I felt strongly, I'd raise a work item request that we address the standard or convention as a whole.

Furthermore, I'd prefer an object over a list of parameters.

Yes, although I think in some cases (args that are always there) it could be preferable to include them? Maybe not though - I'm open to changes.

Of course if there is some existing standard or convention in Espruino, I would follow that

Yeah, it's Node.js. Generally node.js callbacks all have error as the first argument (I think so you can have varying numbers of arguments - including none - and still find the error).

Also, I'd just add:

wifi.setIP(object, callback)

Where object can have ip, mac, subnet, gateway and dns in it (if some are left out they're kept as-is). Basically it matches the object that you get from getIP.

If you just call setIP() it does DHCP - at least that's what WiFi and Ethernet do right now. Not sure if it's worth adding a specific DHCP function though?

for the question of api-style (error first/last, object yes/no) i vote for error first and object-style, it's just easier to follow the node.js culture here

assuming espruino is designed to be single-threaded and mostly asynchronous, than node.js seems to be a good (perfect?) style

@MichaelPralow Awesome ... community choices are (to me) the most important guidance. I am still ignorant on Node.JS so it is incumbent on me to go study that. However if y'all are saying that the defacto story is error code first ... then that has to be the right answer.

A github work item (#589) has been created to track the low level design and implementation.

https://github.com/espruino/Espruino/issues/589

Some comments:
wifi.setPower(onOrOff, function(err) { ... }) sounds like TX power level to me? We need a function to set the wifi mode, I believe, this could include: none (=off), station, ap, station+ap. Or do you want just on/off and then drive the mode by using connect/disconnect/createAP/deleteAP ?

For wifi.connect(ssid, key, [options], function(err) { ... }); I would prefer to see callbacks that get the current state. That allows the app to show whether it's waiting to associate or waiting to get a DHCP lease. Unless you want to roll all that into a global wifi state callback.

wifi.getAPs(function(err, aps) { ... }); I believe this is usually called scan, no? Or put differently, do you expect getAPs to return a list of already-known APs or do you expect it to initiate a scan? If it initiates a scan, can the callback be called multiple times as APs are discovered or only once when the scan finishes?

Why does wifi.getAPs(function(err, aps) { ... }); need a callback as opposed to just returning the info?

wifi.createAP(ssid, key, channel, enc, function(err) { ... }) I see a problem with the channel, which is that in sta+ap mode the ap has to be on the same channel as the sta. Maybe allowing channel 0 to stand for any can make this work.

Additional thoughts:

Being able to set the IP address and associated network parameters (gateway and mask) is needed.

A call is needed to set the hostname, which may be used in DHCP to update DNS and in other local host discovery protocols. The hostname needs to be set before connecting to an AP...

We need calls to disconnect and to shut down the AP.

How about a call to get RSSI and current data rate? Alternatively, getConnectedAP could return that info.

I wonder how wifi and low-power modes should interact. It would be good to have a wifiPowerMode call with an extensible set of values, starting with full power and something mapping to PS-Poll (using the DTIM intervals to power down Wifi). This way there's at least some portable way to state that it's desirable to use less power at the expense of throughput and latency.

One question with the above is what the state at boot should be. Systems like the esp8266 save the wifi config in flash and can automatically connect at boot time. Also, I'd really like to support connecting the IDE to the esprunio via TCP/Wifi and then the espruino has to connect at boot to something on its own (unless that's expected to be driven by startup JS files stored in flash).

@tve Good thoughts ... since there are lots of questions and lots of details ... Ive created a Wiki page we can use to categorize our questions and comments. See the following:

https://github.com/espruino/Espruino/wiki/Generic-WiFi-Design

Feel free to update the page with your own comments and thoughts.

Thanks - I reckon it's important to design things to be flexible enough that they can cope with different forms of WiFi/Ethernet in the future - also that functionality can be added without breaking existing code in the future.

Some of the names are choices because I'm trying to break existing code as little as possible. I try hard not to piss off existing users by making non-backwards-compatible changes. Sometimes that means some things aren't perfect though.

Just looking at createAP makes me think it should take an object, where all but the AP name is pretty much optional.

We need a function to set the wifi mode

Good plan - much nicer than setPower

Why does wifi.getAPs(function(err, aps) { ... }); need a callback as opposed to just returning the info?

Because it's something that takes time, and function calls shouldn't really block? As you suggested above, this is really a scan. Maybe it could be changed, but it is a function that exists now and which is in use by existing Espruino users.

Being able to set the IP address and associated network parameters (gateway and mask) is needed.

Yep, that's where setIP comes in. We could add the ability to set hostname in that as well

We need calls to disconnect and to shut down the AP.

Is this not just setMode you described above?

Connect at boot

Yes, it could look at what's in the variables even before the interpreter starts. USB HID does that at the moment.

Personally I'd say the user should connect themselves in onInit - but having the ability to program the device by WiFi without any of that being involved seems like it's worth it :)

But let's discuss on the Wiki

Just to add to this, I notice in the Wiki there's a 'let's look at how everyone else does this' up the top.

Please also look at how Espruino handles this right now. There's:

• ESP8266 (see bottom of page for API)
  
• CC3000
  
• WIZnet Ethernet
  
• GSM/GPRS libraries.
  
  

They're all relatively similar but not the same - this would be a great opportunity to bring all those properly in line. However plenty of people use Espruino as it is, right now - lets not make changes that force them to rewrite their code for no good reason.

I'm thinking that's the good thing about all this pre-amble discussion. If we discuss the proposed specification for the interface up front and iron it all out (as best we can) before we implement and then ship ... that will make sure we are all in agreement.

As I understand it, if we build this new interface ... that will be a new interface that will not have been coded to previously? It will be a generic interface that all existing boards could choose to implement and at that point, a "WiFi" implementation would then be normalized. I could also imagine that we could choose that the existing implementations remain exactly the same so that there would be TWO ways that WiFi could be used ... one would be the way that exists for boards that already exist ... and the other would be a "common" mechanism that new boards would comply against.

I don't personally consider the ESP8266 support "released" so as yet, all ESP8266 board users are still alpha/beta testers and should expect flux.

It's an option, but I'd like to only have 'one true interface' if at all possible. It's hard to fit everything in Flash on something like the Original Espruino Board as it is - and having 2 ways of doing the same thing just wasted space and confuses people.

ESP8266 board users are still alpha/beta testers and should expect flux.

Absolutely - I meant existing boards like the Pico and the Original Espruino. There are people developing commercial devices with those - changing stuff under their feet will just annoy them.

In my real-life job, when changes are being suggested that might impact folks upstream or downstream we call them "stakeholders" and try and get them involved early and often in things that might impact them. Perhaps if you know of folks who are "important" consumers (and that is an extremely subjective concept) we can make them aware of this work item and see if they would be interested in commenting or getting involved in the decisions? If they say that the impact of changes is not acceptable ... then that's a major consideration. If they say that proposed changes are welcomed ... they may even have important contributions to include.

Maybe drop them a note pointing to this thread and/or the issue?

To be honest @alexanderbrevig is the main one that springs to mind.

Unfortunately there are a lot of people who never come to the forum or even get in touch with me at all - I see their Web IDEs downloading modules from the Espruino website but that's it, as no information gets reported back to me :(

this raises an important question for me - is it possible to work with specific module versions (entangled with firmware if needed) ?

from

var sensor = require("ADNS5050").connect(SPI3, B6); 

to

var sensor = require("ADNS5050", "1.0.2").connect(SPI3, B6); // or '1.0.+' or '1.+' or no version = latest

would be perfect to get more stable code and important hints to the exact cause of "never changed code, but now it explodes" problems

if the version is part of the require api, some sort of header informations or even a module.versions file per project does not matter, it could even be near/identical to node.js/NPM with a package.info file which contains even more nice informations for building, packaging, etc. (e.g. which espruino firmware version this javacode depends on...)

combine that with some "building info" e.g. print out of used firmware/module versions (inside WEB IDE or running espruino as well) :-)

Good post... it would be very useful... e.g. case ESP8266-01 on shim with different versions of AT commands... surely will come next versions... :-)

@profra there are already two distinct modules for the two firmware versions available :(

@MichaelPralow yes, it's a good point. It's been asked before - and might well be worth another post about nice ways to handle this - especially since the modules don't have version numbers at the moment.

For now, as you can type a URL into require, you can actually just pull a module right off GitHub. For instance you could look at the history of something on GitHub, and could then just require a specific version of that file with:

var g = require("https://github.com/espruino/EspruinoDocs/blob/411be7a72bc55b4a51d05f3df5bc8fa80017f3b0/devices/PCD8544.js");

The URL comes from clicking <> by the commit you were interested in.

@Gordon thx for the info, ... i was about to ask you to add an example or tutorial, but i can and will do that myself in the next days

@MichaelPralow that's awesome - thanks!

You might also want to make a note that there's an option in the IDE for whether (and how) the modules you load off the net get minified. The ones that are usually loaded when you type require('module') have been pre-minified.

@Gordon I know, I was the initiator of this new version :-) ... but solution described by @MichaelPralow would be nice, elegant and universal...

Next step to this pulling of a version is adding a config/mapping thing to the IDE to not have it in the code itself...

I think that aiming for separate config for versioning straight away would be better. Considering how widely used the npm approach is, and that espruino is using the same commonjs/require() method of using modules, I think that it would be best to not deviate from that. Also, I think keeping versioning out of Espruino runtime and only in the module-loading code would have a lot of advantages (simpler, doesn't take up flash/ram, no issues with modules on sd cards, new users don't have to deal with it).

I looked through the various libraries linked off the wiki page. What strikes me is that other than the particle.io one none handle the fact that wifi can come up automatically at power-up. They all seem to think that the JS code needs to drive the wifi activity when I think that that's not a good idea for several reasons.

The first is power. In the case of the esp8266 the SDK can initiate a connection before Espruino even starts to run. Depending on the state is was in (e.g. sleep) it shouldn't have to get a fresh association.

The second is that things do get complicated quickly. For example, do you really want the logic to handle state transitions and to retry to connect if the AP is lost in JS? That quickly adds up to quite some code.

Or if power-save sleep results is the loss of a connection, do you want to code the logic to receive a callback and then reconnect in JS?

I would much prefer an interface that prescribes the desired state and that lets some lower device dependent code make it happen and "keep it so" even through restarts. So instead of "connect" I would prefer a "station" call that makes Espruino a station on the specified network (the params are mostly the same). Similarly an "access_point" call that makes Espruino an AP.

How is that different? The "station" call might take some parameters to control how aggressively to reconnect when the AP is lost but the biggest difference is in the semantics. In one case the effect of a call is persistent, in the other it's not.

Maybe this can be implemented in layers so a lower layer with the primitives is available as well, but that does add code and complexity.

We're going properly off-topic here! New thread?

@the1laz how does node.js handle requesting certain versions? Is it all done via npm - or can you require("foo@1.2")?

The issue for me at the moment is that there is no versioning on the modules. I'm not 100% sure there should be either as Git commits maybe make more sense... Someone could easily come up with a historic modules plugin for the Web IDE that let you choose from a list of modules, and then let you choose the git commit from a list (which could also show date and commit message) or maybe even add your own URL if you wanted to do some development work.

On topic again :)

@tve I totally agree here. Espruino's about trying to bring the simplicity of scripting on desktop to an MCU, so dealing with WiFi disconnect shouldn't be something you have to worry about.

Making the connection persistent (at least between soft resets) is vital if there's going to be any OTA programming too.

Only thing I see with connection at startup is it's very likely that someone will do function onInit() { require('http').get("http://foo/i_am_alive); }.

In that case, what do you do? When the ESP8266 boots, it might be trying to connect, but it almost certainly won't have connected when onInit runs.

... and you can't delay onInit - it's very likely someone might want to display Connecting... on an LCD or something.

So you could delay executing any pending network requests until you have a connection I guess? It all gets a bit complicated.

Also, connecting to a WiFi network shouldn't really involve a write to flash memory (but it would have to if it were to be persistent?).

I'd be tempted to say:

• After connection, WiFi should stay connected between soft resets
  
• On a reboot, WiFi isn't connected and needs to be enabled
  
• (possibly) if the user does a save() while WiFi is connected, it could auto-connect at power-on?
  

Sounds like there need to be a couple of simple config options around what to do on reset/reboot 'cause I would definitely want it to always reconnect and you seem to want something different. I think I can cook a proposal up...

For the connection at boot it might be handy to have a onInitialConnect callback that can be used to trigger stuff when a connection is first established after a restart/reset. This would be different from a onStateChange callback that is called everytime something happens and makes more sense to display status or so.