HTTPS support on Pico! #131
Replies: 126 comments
-
Posted at 2015-11-17 by @allObjects ...termendeous success! Thanks for tackling https... it opens a flood gate for good things. I know it is a bit a stretch for the resources, but it makes the world a much safer place. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-17 by DrAzzy This is awesome news! |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-17 by d0773d @gordon This is really good news! I'm assuming TLS 1.2? |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-18 by @gfwilliams Yes, it's 1.2. mbedtls seems quite flexible so potentially other things could be used though. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-18 by @gfwilliams Ok, an update on this... It's working great now, and I have just merged HTTPS client support in. You can get an early build for the Espruino Pico and Wiznet/ESP8266/GSM modules by copying and pasting this into
And the code works just as before, just add
However, bad news for those of you thinking of using this on other boards. The TLS spec seems to require that there be 16kB packet sizes, and it looks like you need two buffers. So you need over 32kB of free RAM minimum if you're going to abide by the spec. There's an extension to this where the client can ask for smaller buffers, but it's not guaranteed to work at all. So it looks like running HTTPS on the ESP8266 is never going to happen (we have 12kB available for all code and variables currently). That'll have to wait for the new one EspressIF are releasing :) |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-18 by @allObjects Accidental no-mentioning of CC3000? |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-18 by @gfwilliams Nope... It should work too, but I haven't tested. It just requires a separate build which I couldn't be bothered to put online. You could build it yourself though. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-19 by tve Q: what led you to use mbedtls over some of the other options? |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-19 by @gfwilliams It seemed like it'd be well supported, and also well optimised for ARM (which basically every uC apart from ESP8266 is using). It also exposes SHA/AES/MD5/etc that would be useful to Pico owners in their own right. I think the only thing stopping a lot of people using PolarSSL was the licence, but since it got acquired by ARM they moved to something far more permissive. For ESP8266, it could be worth setting |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-19 by lukevanhorn This is great news, Gordon. Thanks! I'm looking to use the AWS IoT services. The primary protocol is MQTT encrypted with TLS, identity is managed using X.509 certificates (validated via TLS1.2 client authentication mode). It looks like most of the pieces are in place for this, but I wanted to make sure I'm not missing something in terms of device capability in using the certificates / encryption. http://docs.aws.amazon.com/iot/latest/developerguide/identity-in-iot.html Here's the key requirements:
Thanks! |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-19 by @gfwilliams Hi Luke, That sounds fine to me - the way to find out would be to give it a go :) Having said that, right now TLS is only enabled for HTTP in the image above. Adding it for raw sockets is just a matter of exposing it at the API level - I'll take a quick look at how to do that now. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-19 by @gfwilliams Ok, if all goes well, in an hour or so there will be a Pico build here with it in: http://www.espruino.com/binaries/git/commits/4111ee167a43b8b8d80e5579829f562993ff2fe8
The example above is a broken HTTP request, but it's enough to prove that socket connections work. I'm leaving out HTTPS and TLS servers for the moment - I think they're probably a lot less use, and it's going to be more effort to add certificate loading. Note: These builds still don't verify the certificates. While they'll connect to secure services, all that effort is wasted if someone can spoof the DNS and point you at their own TLS server without you noticing :) |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-19 by lukevanhorn That was fast. Thanks! I'll have a go at getting it to work. I have a rough idea of what steps I'll need to take:
|
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-19 by @gfwilliams At the moment there's nothing in there for certificates at all, so I'd have to add that. But for now I think you can get away without modifying the MQTT lib - it was designed to work over things other than network sockets unless someone wanted to (for instance) use some other kind of radio:
|
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-20 by @gfwilliams Just to add - the build didn't work - there's now not enough room in flash memory for the debugging info and TLS - so I'll have to change the build to produce builds without debugging info. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-21 by @gfwilliams Update #2: If you:
Use the Google sheets example, but remove the timeouts:
You'll end up with loads of free memory (~450 vars used as opposed to ~850), as then any functions stored in modules stay defined in flash - so that actually makes HTTPS on the Pico + ESP8266 a whole lot more usable. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-21 by Cale It crashed just the same. I don't know why I keep getting the key mismatch. I have tried the same keys in the tutorial to no avail. I have been able to make it work fine when espruino runs in Ubuntu with the keys/certs I have. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-21 by Cale Im getting
|
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-22 by @gfwilliams You need the latest version of Espruino from here: http://www.espruino.com/binaries/git/commits/master (so not just 1v89 from the website) (give it an hour or two - looks like there are some problems with the build I need to fix)
So when you compile Espruino yourself to run natively in Linux it works? The really odd thing there is that the HTTPS code is identical in the Pico and Linux builds, so it should be doing exactly the same stuff. Can you run the version checking code that's on http://www.espruino.com/ESP8266? I guess it might be that your ESP8266's firmware is out of date, and it's somehow corrupting the data it sends to the server. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-22 by @gfwilliams Ok, it's up there now. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-22 by Cale Linux: Yeah I've never had a problem with it.....
now we have:
and
Right now I am really busy trying to understand exactly how this communication works. Do we really need the cert and key? I have a test client through NodeJs and it is sending fine with one of their known "root" CAs. I think these are them. Much thanks for your help @gordon. This is really going to do well for me once we get this figured out. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-22 by @gfwilliams ESP8266 version seems ok - it's what I'm using here. And that's using the google sheets example code exactly as-is, without any changes? It's frustrating because 0x4290 doesn't even seem to be a valid mbedtls error code :( I guess the lack of memory could potentially be causing an issue if it's having trouble allocating something during the handshake. I'll be releasing that new Web IDE soon which might help out with the low memory. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-23 by Cale I recopied everything exactly from the example:
I'll also put together a push through node app to get me going in the mean time and I'll keep trying to figure this out from my end. Let me know if you can think of anything else I can try on the Pico. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-23 by @gfwilliams Ok, so I do the following:
Wait a minute, and it says:
(so it fails)
So it works - but it's properly tight on memory. What's happening is the first time it's got the command history and other stuff in memory, and it tries to allocate everything it needs but it can't find enough continuous space, even after it's freed everything. Next time the history's already been freed so it's able to 'fill the gaps' while executing, leaving a much larger contiguous amount of memory. I just modified
However I've just pushed the new version of the Web IDE (0.65.2) with the |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-23 by @allObjects fight for the bytes! |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-23 by Cale A minute after gaining wifi connection this all happens right in a row...... no time gap.
Then after another minute:
|
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-24 by Cale I set up my own pass through server app that is getting the information where it needs to go. Sorry @gordon I just do not know what else to do with this. I wonder if this is something to do with my location and country? I did at one time get it to work on the wiznet, but my application would be much nicer with esp8266. I'll keep plugging at it every now and then. Thanks again. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-24 by @gfwilliams I just don't understand how you're getting 'not enough memory' if you're using the exact same code and firmware that I had - I left it running all yesterday and it was going without problems :( Also, with the suggestions in #113 I had loads of free memory available. Glad you got something else sorted - it's just a shame you didn't get the Pico working directly. It could be your location I guess, but the fact you had it going on WIZnet and PC, and that it always complains 'out of memory', makes me think it probably is just a problem with the Pico running out of memory. TLS uses loads of memory if you're trying to do it in a spec compliant way (which I am), and when the Pico's got to communicate with the ESP8266 using JS as well there's not a great deal of memory left over. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2017-04-07 by sureshkm I think, http://forum.espruino.com/conversations/302991/ is similar to the issue discussed here. Did we get any solution? |
Beta Was this translation helpful? Give feedback.
-
Posted at 2017-04-07 by @gfwilliams The issue in your new post is, as it says in the error message, that there's not enough memory. Please can we discuss that on your new post though, rather than in this one? |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-17 by @gfwilliams
It's still early VERY days, but it's now possible to:
make clean;PICO_1V3=1 WIZNET=1 USE_HTTPS=1 make serialflash
then...
HTTPS support works on:
There are some big issues:
This uses a lot of code space, and realistically it's not going to fit on the Original Espruino board unless you're happy using the extra flash memory that isn't supposed to exist (but does).
Beta Was this translation helpful? Give feedback.
All reactions