HTTPS support on Pico! #131
Replies: 126 comments
-
Posted at 2015-11-18 by @gfwilliams Yes, it's 1.2. mbedtls seems quite flexible so potentially other things could be used though. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-18 by @gfwilliams Ok, an update on this... It's working great now, and I have just merged HTTPS client support in. You can get an early build for the Espruino Pico and Wiznet/ESP8266/GSM modules by copying and pasting this into
And the code works just as before, just add
However, bad news for those of you thinking of using this on other boards. The TLS spec seems to require that there be 16kB packet sizes, and it looks like you need two buffers. So you need over 32kB of free RAM minimum if you're going to abide by the spec. There's an extension to this where the client can ask for smaller buffers, but it's not guaranteed to work at all. So it looks like running HTTPS on the ESP8266 is never going to happen (we have 12kB available for all code and variables currently). That'll have to wait for the new one EspressIF are releasing :) |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-18 by @gfwilliams Nope... It should work too, but I haven't tested. It just requires a separate build which I couldn't be bothered to put online. You could build it yourself though. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-19 by @gfwilliams It seemed like it'd be well supported, and also well optimised for ARM (which basically every uC apart from ESP8266 is using). It also exposes SHA/AES/MD5/etc that would be useful to Pico owners in their own right. I think the only thing stopping a lot of people using PolarSSL was the licence, but since it got acquired by ARM they moved to something far more permissive. For ESP8266, it could be worth setting |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-19 by @gfwilliams Hi Luke, That sounds fine to me - the way to find out would be to give it a go :) Having said that, right now TLS is only enabled for HTTP in the image above. Adding it for raw sockets is just a matter of exposing it at the API level - I'll take a quick look at how to do that now. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-19 by @gfwilliams Ok, if all goes well, in an hour or so there will be a Pico build here with it in: http://www.espruino.com/binaries/git/commits/4111ee167a43b8b8d80e5579829f562993ff2fe8
The example above is a broken HTTP request, but it's enough to prove that socket connections work. I'm leaving out HTTPS and TLS servers for the moment - I think they're probably a lot less use, and it's going to be more effort to add certificate loading. Note: These builds still don't verify the certificates. While they'll connect to secure services, all that effort is wasted if someone can spoof the DNS and point you at their own TLS server without you noticing :) |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-19 by @gfwilliams At the moment there's nothing in there for certificates at all, so I'd have to add that. But for now I think you can get away without modifying the MQTT lib - it was designed to work over things other than network sockets unless someone wanted to (for instance) use some other kind of radio:
|
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-20 by @gfwilliams Just to add - the build didn't work - there's now not enough room in flash memory for the debugging info and TLS - so I'll have to change the build to produce builds without debugging info. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-23 by @gfwilliams Thanks! We'll have to get some examples together of MQTT + TLS... maybe someone wants to write a tutorial? ... :) Had anyone tried this yet? All the latest builds like this one should have it in now. The Pico will actually have the same amount of program space available as before - it had some free flash memory before, but now that's mostly taken up with the TLS stuff :) |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-23 by @gfwilliams ... just to add - it's definitely worth spreading the news about this, but I should do a 1v82 release first I think. It'd be good to get some feedback about the current GitHub versions though - there have been some pretty major changes in there to try and get memory usage down, so I'm still a bit paranoid about whether some bugs have crept in :) |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-24 by @gfwilliams I'm not sure many more people would have tried a release candidate than tried the GitHub builds. I just released 1v82 properly - if there's a big problem I can always shift out 1v83 quite quickly. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-25 by @gfwilliams Great! So what exactly is missing from the Espruino Pico implementation? Is it just that there's no check on the certificate so it's not a good idea to connect to AWS, or it's actually that you can't connect to AWS at the moment. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-26 by @gfwilliams Ahh, I'd have to look into that. mbedtls will definitely handle that - I just haven't exposed it. Pull requests welcome :) |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-27 by @gfwilliams Ok, I just filed an issue for this: espruino/Espruino#736 I'm not sure when I'll get around to it, but I'll see what can be done. It doesn't look like it'll be too painful to implement. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-27 by @gfwilliams Ok, just done. For instance the following now works:
If you have the certificates as This will be in http://www.espruino.com/binaries/git/commits/master/ - just give it an hour for the latest build to appear in there. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-02-15 by @gfwilliams You probably just want to put the code in as a JS file, and run Victory as in working on Linux, or working on everything? |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-02-16 by @gfwilliams Hmm, interesting - and you were using the 0.25 firmware on the ESP8266 when it wasn't working? And yes, the way HTTPS is done it'll work on WIZ550io, CC3000, GSM, and anything else that comes along. I have only personally tried it on the WIZ550io though. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-02-17 by @gfwilliams Oh, so you get It could just be a connection issue - I reproduced exactly the same problem when I disconnected the CS pin. There shouldn't be a problem with the Picos - the first run are absolutely fine, they just don't have a fuse on them so you have to be careful not to short the power rails out or the diode fries :) |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-02-18 by @gfwilliams
Does it say anything else? Sounds a bit strange - I wonder if the code has failed to upload properly. I changed the Google Sheets example a bit last week to use The new one should hopefully be a little more reliable from that point of view. It's strange though - I'm not sure I've come across the -7 error before! |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-02-18 by @gfwilliams Actually - just so we know, what was the problem with WIZnet? Was it the CS pin? |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-21 by @gfwilliams Are you able to post up the exact code that you're using so we can give it a try? Also, which espruino version worked, and which didn't? |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-21 by @gfwilliams So even the bare google sheets example isn't working for you with 1v89? Are you getting it halting at:
Like I did when I just tried it, or is it breaking somewhere else? With 1v86, does it work when using the example as-is? I have no idea what error 0x256a is (it doesn't seem to be a valid error code from the Pico's SSL library), however 0x6D00 is:
Which implies that the public key might be of the wrong type (like DSA instead of RSA). I just tried here on a Pico + ESP8266 and the example still works perfectly on 1v86. Having said that there's virtually no free space on the Pico at all when using HTTPS with ESP8266. The Espruino WiFi board will be a better bet there as it's got an extra 32k of RAM - I just tried it and 1v89 works fine (so the crash on the Pico would appear to be some kind of out of memory issue). At some point when things calm down after Puck.js I'm going to move to a compiled implementation of the ESP8266 driver. That should cut down on the RAM usage significantly and will make the Pico a lot better at HTTPS with ESP8266 - but I'm afraid it's likely to be a month or two off. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-21 by @gfwilliams Quick update: The latest build for Espruino Pico has fixed the issue that was causing the Google sheets example to crash. Turns out it was to do with available memory. For some reason newer versions of GCC started allocating a 1kB buffer for the standard libraries. By moving back to the old Sadly I believe the crash was in the TLS libraries - because it was overflowing stack (not allocated memory) it's far more difficult to catch and handle correctly (hence the crash). Also: I've made another change to module loading. It'll require some changes to the IDE to take full advantage of it, but when that's done you'll be able to turn on the 'save on send' option in the IDE on and it'll drastically reduce RAM usage. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-21 by @gfwilliams Update #2: If you:
Use the Google sheets example, but remove the timeouts:
You'll end up with loads of free memory (~450 vars used as opposed to ~850), as then any functions stored in modules stay defined in flash - so that actually makes HTTPS on the Pico + ESP8266 a whole lot more usable. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-22 by @gfwilliams You need the latest version of Espruino from here: http://www.espruino.com/binaries/git/commits/master (so not just 1v89 from the website) (give it an hour or two - looks like there are some problems with the build I need to fix)
So when you compile Espruino yourself to run natively in Linux it works? The really odd thing there is that the HTTPS code is identical in the Pico and Linux builds, so it should be doing exactly the same stuff. Can you run the version checking code that's on http://www.espruino.com/ESP8266? I guess it might be that your ESP8266's firmware is out of date, and it's somehow corrupting the data it sends to the server. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-22 by @gfwilliams Ok, it's up there now. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-22 by @gfwilliams ESP8266 version seems ok - it's what I'm using here. And that's using the google sheets example code exactly as-is, without any changes? It's frustrating because 0x4290 doesn't even seem to be a valid mbedtls error code :( I guess the lack of memory could potentially be causing an issue if it's having trouble allocating something during the handshake. I'll be releasing that new Web IDE soon which might help out with the low memory. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-23 by @gfwilliams Ok, so I do the following:
Wait a minute, and it says:
(so it fails)
So it works - but it's properly tight on memory. What's happening is the first time it's got the command history and other stuff in memory, and it tries to allocate everything it needs but it can't find enough continuous space, even after it's freed everything. Next time the history's already been freed so it's able to 'fill the gaps' while executing, leaving a much larger contiguous amount of memory. I just modified
However I've just pushed the new version of the Web IDE (0.65.2) with the |
Beta Was this translation helpful? Give feedback.
-
Posted at 2016-11-24 by @gfwilliams I just don't understand how you're getting 'not enough memory' if you're using the exact same code and firmware that I had - I left it running all yesterday and it was going without problems :( Also, with the suggestions in #113 I had loads of free memory available. Glad you got something else sorted - it's just a shame you didn't get the Pico working directly. It could be your location I guess, but the fact you had it going on WIZnet and PC, and that it always complains 'out of memory', makes me think it probably is just a problem with the Pico running out of memory. TLS uses loads of memory if you're trying to do it in a spec compliant way (which I am), and when the Pico's got to communicate with the ESP8266 using JS as well there's not a great deal of memory left over. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2017-04-07 by @gfwilliams The issue in your new post is, as it says in the error message, that there's not enough memory. Please can we discuss that on your new post though, rather than in this one? |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-17 by @gfwilliams
It's still early VERY days, but it's now possible to:
make clean;PICO_1V3=1 WIZNET=1 USE_HTTPS=1 make serialflash
then...
HTTPS support works on:
There are some big issues:
This uses a lot of code space, and realistically it's not going to fit on the Original Espruino board unless you're happy using the extra flash memory that isn't supposed to exist (but does).
Beta Was this translation helpful? Give feedback.
All reactions