HTTPS support on Pico!

It's still early VERY days, but it's now possible to:

• Check out the HTTPS branch on GitHub
  
• build and flash with make clean;PICO_1V3=1 WIZNET=1 USE_HTTPS=1 make serialflash
  
  

then...

SPI2.setup({ mosi:B15, miso:B14, sck:B13 });
var eth = require("WIZnet").connect(SPI2, B10);
eth.setIP(); // DHCP
require("http").get("https://www.google.com", function(res) {
  res.on('data', function(data) { console.log(data);    });
});

HTTPS support works on:

• Linux, Raspberry Pi, OpenWRT stuff
  
• Pico with WIZnet ethernet
  
  

There are some big issues:

• There's no cleanup, so you can only do a single HTTPS request before it fails right now
  
• You can only do one HTTPS request at once
  
• The network API needs reworking to allow nonblocking connect and name resolution before ESP8266 (AT or native)/CC3000/etc will work. But when that's done, it'll solve a few other problems too!
  
  

This uses a lot of code space, and realistically it's not going to fit on the Original Espruino board unless you're happy using the extra flash memory that isn't supposed to exist (but does).

...termendeous success! Thanks for tackling https... it opens a flood gate for good things. I know it is a bit a stretch for the resources, but it makes the world a much safer place.

This is awesome news!

@Gordon This is really good news! I'm assuming TLS 1.2?

Yes, it's 1.2. mbedtls seems quite flexible so potentially other things could be used though.

Ok, an update on this... It's working great now, and I have just merged HTTPS client support in.

You can get an early build for the Espruino Pico and Wiznet/ESP8266/GSM modules by copying and pasting this into Advanced Flash in the Web IDE's settings:

http://www.espruino.com/files/espruino_1v81.274_pico_1r3_https.bin

And the code works just as before, just add https to the beginning of the URL:

digitalWrite(B9,1); // enable on Pico Shim V2
Serial2.setup(115200, { rx: A3, tx : A2 });
var wifi = require("ESP8266WiFi_0v25").connect(Serial2, function(err) {
  if (err) throw err;
  wifi.reset(function(err) {
    if (err) throw err;
    console.log("Connecting to WiFi");
    wifi.connect("WiFi_name","WiFi_key", function(err) {
      if (err) throw err;
      console.log("Connected");
      // Now you can do something, like an HTTP request
      require("http").get("https://google.com", function(res) {
        console.log("Response: ",res);
        res.on('data', function(d) {
          console.log("--->"+d);
        });
      });
    });
  });
});

However, bad news for those of you thinking of using this on other boards. The TLS spec seems to require that there be 16kB packet sizes, and it looks like you need two buffers. So you need over 32kB of free RAM minimum if you're going to abide by the spec. There's an extension to this where the client can ask for smaller buffers, but it's not guaranteed to work at all.

So it looks like running HTTPS on the ESP8266 is never going to happen (we have 12kB available for all code and variables currently). That'll have to wait for the new one EspressIF are releasing :)

Accidental no-mentioning of CC3000?

Nope... It should work too, but I haven't tested. It just requires a separate build which I couldn't be bothered to put online. You could build it yourself though.

Q: what led you to use mbedtls over some of the other options?

It seemed like it'd be well supported, and also well optimised for ARM (which basically every uC apart from ESP8266 is using). It also exposes SHA/AES/MD5/etc that would be useful to Pico owners in their own right.

I think the only thing stopping a lot of people using PolarSSL was the licence, but since it got acquired by ARM they moved to something far more permissive.

For ESP8266, it could be worth setting MBEDTLS_SSL_MAX_CONTENT_LEN in config.h and actually seeing if it can connect to any services. My guess is that stuff like Twitter won't support it though.

This is great news, Gordon. Thanks!

I'm looking to use the AWS IoT services. The primary protocol is MQTT encrypted with TLS, identity is managed using X.509 certificates (validated via TLS1.2 client authentication mode).

It looks like most of the pieces are in place for this, but I wanted to make sure I'm not missing something in terms of device capability in using the certificates / encryption.

http://docs.aws.amazon.com/iot/latest/developerguide/identity-in-iot.html

Here's the key requirements:

Clients must support all of the following in their TLS implementation to use AWS IoT certificates:

• TLSv1.2
  
• SHA-256 RSA certificate signature validation
  
• One of the following cipher suites: ( found here )
  

Thanks!
Luke

Hi Luke,

That sounds fine to me - the way to find out would be to give it a go :)

Having said that, right now TLS is only enabled for HTTP in the image above. Adding it for raw sockets is just a matter of exposing it at the API level - I'll take a quick look at how to do that now.

Ok, if all goes well, in an hour or so there will be a Pico build here with it in: http://www.espruino.com/binaries/git/commits/4111ee167a43b8b8d80e5579829f562993ff2fe8

require("tls").connect("https://localhost:443", function(res) {
  console.log("Got response: " + JSON.stringify(res));
  res.write("GET / HTTP/1.1\r\n\r\n");
  res.on('data', function(data) { 
	  console.log(">" + data);
	});
});

The example above is a broken HTTP request, but it's enough to prove that socket connections work.

I'm leaving out HTTPS and TLS servers for the moment - I think they're probably a lot less use, and it's going to be more effort to add certificate loading.

Note: These builds still don't verify the certificates. While they'll connect to secure services, all that effort is wasted if someone can spoof the DNS and point you at their own TLS server without you noticing :)

That was fast. Thanks!

I'll have a go at getting it to work. I have a rough idea of what steps I'll need to take:

1. read the root and client certificates from either sd card or flash memory
   
2. Extend the MQTT module to accept these certs in the connect options and then setup a TLS client connection and handle TLS errors.
   
3. Possibly need to expose something to handle server request for client cert or should this be all set?
   
   
   
   

At the moment there's nothing in there for certificates at all, so I'd have to add that.

But for now I think you can get away without modifying the MQTT lib - it was designed to work over things other than network sockets unless someone wanted to (for instance) use some other kind of radio:

  var mqtt = require("MQTT").create();
  mqtt.on('connected', function() {
    mqtt.subscribe("test");
  });
require("tls").connect("mqtt://whatever:1234", function(res) {
  mqtt.connect(res);
});

Just to add - the build didn't work - there's now not enough room in flash memory for the debugging info and TLS - so I'll have to change the build to produce builds without debugging info.

Thanks for working on this, Gordon. I'll have a look at getting the certificates / client authentication working with TLS.

Support for MQTT over tls is big news. It deserves a separate post. I have been waiting this long. Thanks so much!

@Gordon this fantastic put's espruino in league of it's own.
Once done and dusted this worth a news push.

Thanks

Wow, fantastic news!

@ Gordon How much flash space will there be left for the actual program on the Pico after flashing the TLS capable firmware?

Thanks! We'll have to get some examples together of MQTT + TLS... maybe someone wants to write a tutorial? ... :)

Had anyone tried this yet? All the latest builds like this one should have it in now.

The Pico will actually have the same amount of program space available as before - it had some free flash memory before, but now that's mostly taken up with the TLS stuff :)

... just to add - it's definitely worth spreading the news about this, but I should do a 1v82 release first I think.

It'd be good to get some feedback about the current GitHub versions though - there have been some pretty major changes in there to try and get memory usage down, so I'm still a bit paranoid about whether some bugs have crept in :)

Maybe a release candidate for 1v82 could help in getting more people to test it? I.e. an easy to find download that remains stable for a while...

I'm setting up a test today - more soon. I was able to compile and flash to the pico with the RELEASE=1 flag set.

Solved.