Have Espruino tweet?

Has anyone managed to have Espruino post a tweet to Twitter?

Now that I've got Espruino Pico with ESP8266 in hand, I'd like to make a coffee pot that tweets.

I understand I could do simple GET and POST request with arguments using built-in http module. But Twitter requires authentification by OAuth.

Could we use OAuth library like this one?
https://code.google.com/p/oauth/source/browse/#svn%2Fcode%2Fjavascript

Reading through how tweet is done in Arduino world, there seems to be two ways:

1. Use intermediate server to hand off the OAuth heavy lifting
   http://playground.arduino.cc/Code/TwitterLibrary
   This is light weight for Arduino code. But requires 3rd party server or set up your own.

2. Implement OAuth request signing on Arduino
   http://www.markkurossi.com/ArduinoTwitter/index.html
   Arduino can directly tweet. But the library is big.

Mmm... the second approach is very attractive for independence.

Would getting oauth.js and sha1.js here work on Espruino mean we could do it?
https://code.google.com/p/oauth/source/browse/#svn/code/javascript

I think I just have to give it a try...

As far as I know, nobody has tweeted yet (at least not since Twitter changed to using OAuth). It sure looks (at least space-wise) like you could fit OAuth and SHA1 into Espruino's memory, so it could well be possible - especially if you choose to minify the code before it's sent to Espruino.

Please give it a go and let us know how you get on - it'd be great if you could... If there is anything in Espruino's JS implementation that causes you problems let me know too - I might be able to do something about it.

Thanks for your encouragement, Gordon!
I gave it a go and faced that "encodeURIComponent" function is missing.

Here's what I did:
Copied sha1.js to Espruino Web IDE and shuffled the order of function definitions a bit, it was sent to Espruino Pico alright. Running sha1_vm_test(); on Pico returned true. Nice ;-)

Then I added codes from oath.js. It needed minor change (move open curly bracket to the same line as function declaration). With that, it was sent to Pico alright. But running testGetSignature() returned Error: Function "encodeURIComponent" not found!

I searched around and found that v8 implements it in javascript
http://v8.googlecode.com/svn/trunk/src/uri.js

Copied it to Web IDE but it refused to send to Pico with message:
"A corn parse for plugin/compiler.js failed. Check the editor window for syntax errors."

There are quite a few "!" marks and these lines are labeled "x"

if (value < 0x10000) {
      %_TwoByteSeqStringSetChar(index++, value, result);
    } else {
      %_TwoByteSeqStringSetChar(index++, (value >> 10) + 0xd7c0, result);
      %_TwoByteSeqStringSetChar(index++, (value & 0x3ff) + 0xdc00, result);
    }

var result = %NewString(array.length, NEW_ONE_BYTE_STRING);
    for (var i = 0; i < array.length; i++) {
      %_OneByteSeqStringSetChar(i, array[i], result);
    }

I'd appreciate any advice to get encodeURIComponent working. I have no idea what these % are doing in javascript...

Thats....

I have no idea how that code could ever work. I can't find any information on that function it's calling either - I suspect it's jumping through a lot of unnecessary hoops there, nor to % being valid in a variable name. Also, in any event, the % functions aren't defined anywhere. I wonder if they're calling out to things only available in V8?

Also, wtf@ %NewString?

It looks like they're making fixed length "strings" and loading them using those setchar functions.... Why are they doing it that way, though? For optimization?

I think that section ought to be totally reimplemented - all the code does is URL encode something (you know how spaces get turned into %20 in URLs - that); It could be a lot smaller than that, I think, particularly since we know the string is 8-bit.

As @DrAzzy says, looks like that's probably not 'real' JavaScript, but is calling into internal V8 functions...

As there's no unicode it's simplified quite a bit - you could probably get away with:

function encodeURIComponent(s) { 
  var ok = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
  var r = "";
  for (var i=0;i<s.length;i++) { 
    if (ok.indexOf(s[i])>=0) r+=s[i];
    else r+= "%"+(256+s.charCodeAt(i)).toString(16).substr(-2);
  }
  return r;
}

Thanks again, DrAzzy and Gordon. I've got one step closer.

OAuth URI encoding seem to have some extra ok characters. And it wants to see upper case letters after %. With this modified encodeURIComponent function, Espruino passed testEncode(). Yeah!

function encodeURIComponent(s) { 
  var ok = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_.~";
  var r = "";
  for (var i=0;i<s.length;i++) { 
    if (ok.indexOf(s[i])>=0) r+=s[i];
    else r+= "%"+(256+s.charCodeAt(i)).toString(16).toUpperCase().substr(-2);
  }
  return r;
}

The next hurdle is regular expression in parseUri function.

    parseUri: function parseUri (str) {
        /* This function was adapted from parseUri 1.2.1
           http://stevenlevithan.com/demo/parseuri/js/assets/parseuri.js
         */
        var o = {key: ["source","protocol","authority","userInfo","user","password","host","port","relative","path","directory","file","query","anchor"],
                 parser: {strict: /^(?:([^:\/?#]+):)?(?:\/\/((?:(([^:@\/]*):?([^:@\/]*))?@)?([^:\/?#]*)(?::(\d*))?))?((((?:[^?#\/]*\/)*)([^?#]*))(?:\?([^#]*))?(?:#(.*))?)/ }};
        var m = o.parser.strict.exec(str);
        var uri = {};
        var i = 14;
        while (i--) uri[o.key[i]] = m[i] || "";
        return uri;
    }

Well, Espruino already has url.parse function so I think I could use it with some key re-mapping. I'm going to try...

Wow, yes, that's going to be more painful! However again I'd guess that the URIs actually passed in are actually really simple and of one particular type?

If url.parse doesn't do it, you might be able to hack something simple up?

My crude re-mapping of url.parse seems to be doing the job. But...
Ouch! I've faced with ERROR: Out of Memory...

Well, as attached, I'm just copying concat codes into Web IDE. I think I'm going to try minify it.

|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v79 Copyright 2015 G.Williams
>echo(0);
=undefined
>testGetParameters();
ERROR: Out of Memory!
=undefined
Uncaught Error: Field or method "length" does not already exist, and can't create it on null
 at line 22 col 29
        if (esp_parse.search.length > 0){
                             ^
in function "" callein function "normalizeUrl" called from line 16 col 74
...tureMethod.normalizeUrl(URL))
                               ^
in function "getBaseString" called from line 2 col 69
...ethod.getBaseString(message);
                               ^
in function "sign" called from line 7 col 69
...ame, accessor).sign(message);
                               ^
in function "sign" called from line 21 col 53
        OAuth.SignatureMethod.sign(message, accessor);
                                                     ^
in function "completeRequest" called from line 33 col 67
...sumerKey: 'CK', token: 'T'});
                               ^
in function "testGetParameters" called from line 1 col 19
testGetParameters();
                   ^
>
Disconnected

1 Attachment

• esptweet.js

That's a lot of code. How much memory do you have left just after uploading it? I can't imagine very much.

Thanks for your interest, DrAzzy!

This is right after uploading the code in #9 without minify.

 1v79 Copyright 2015 G.Williams
>echo(0);
=undefined
>process.memory();
={ "free": 204, "usage": 2836, "total": 3040, "history": 190,
  "stackEndAddress": 536924840, "flash_start": 134217728, "flash_binary_end": 312536, "flash_code_start": 134234112, "flash_length": 393216 }
> 

With minification by Esprima.

>process.memory();
={ "free": 1242, "usage": 1798, "total": 3040, "history": 1219,
  "stackEndAddress": 536924840, "flash_start": 134217728, "flash_binary_end": 312536, "flash_code_start": 134234112, "flash_length": 393216 }
> 

With minification by Closure Simple Optimisation.

>process.memory();
={ "free": 1333, "usage": 1707, "total": 3040, "history": 1218,
  "stackEndAddress": 536924840, "flash_start": 134217728, "flash_binary_end": 312536, "flash_code_start": 134234112, "flash_length": 393216 }
> 

Pretty good improvement ;-) I continue my endeavour.

I've got oauth.js and sha1.js running on Esprino Pico alright! (Well, at least included test functions run ok.)

 1v79 Copyright 2015 G.Williams
>echo(0);
=undefined
>process.memory();
={ "free": 1323, "usage": 1717, "total": 3040, "history": 1229,
  "stackEndAddress": 536924840, "flash_start": 134217728, "flash_binary_end": 312536, "flash_code_start": 134234112, "flash_length": 393216 }
>sha1_vm_test()
=true
>testEncode();
=undefined
>testGetParameters();
=undefined
>testGetBaseString();
=undefined
>testGetSignature();
=undefined
> 

I used minification by Closure Simple Optimisations and have Set Current Time on under Communications setting.

Now I'need to fight with Twitter API and find out what parameters to throw.
This seems to be a good reference to follow.
https://github.com/mogya/tm_twitter_api

1 Attachment

• esptweet.js

Could Espruino handle HTTPS...?

All twitter REST API resource URL are HTTPS, not HTTP...
And I've got this error code returned. Yeah, sure. I called require("http").get() function with a URL starting with "https://"...

Response:  {
  "headers": {
    "content-length": "52",
    "content-type": "application/json;charset=utf-8",
    "date": "Sun, 14 Jun 2015 13:38:10 GMT",
    "server": "tsa_a",
    "set-cookie": "guest_id=v1%3A143428919094643089; Domain=.twitter.com; Path=/; Expires=Tue, 13-Jun-2017 13:38:10 UTC",
    "x-connection-hash": "3e91ae650c8rfb6894d0fc7183fae787",
    "x-response-time": "8"
   },
  "httpVersion": "1.0",
  "statusCode": "403",
  "statusMessage": "Forbidden"
 }
--->{"errors":[{"message":"SSL is required","code":92}]}
> 

Oh, these HTTPS and OAuth security things are so tough for micro controllers! We need a good solution to make IoT inexpensive!

Argh - sorry, I'd assumed that just having OAuth would be enough. At the moment there's no way to do HTTPS. It might be possible to implement natively in the Pico, but the memory + processing requirements are absolutely huge.

It's very frustrating of Twitter - requiring HTTPS seems a bit heavyweight when the information you're sending is going to be displayed publicly anyway!

Well, HTTPS and OAuth seems the de facto standard of web service these days... despite IoT hype. I wonder if a WiFi module with hardware HTTPS and SHA1 capability could come to rescure.

TI's CC3200 has "Hardware Crypto Engine" and also said to include TLS/SSL stacks along side with ARM M4 processor for applications to run.
http://www.ti.com/product/cc3200

Would be interesting if Espruino can run on it!

A few people have asked that, but after the CC3000 I'm not touching TI WiFi with a bargepole. It seems they're great at marketing, probably fine at hardware too, but horrifically bad at drivers/firmware/support.

I think probably more interesting is whether there could be an HTTPS-capable AT-command firmware for the ESP8266. It sure seems very likely (apparently it already has a lot of crypto functionality exposed).

On the Ethernet side, WIZnet have launched the W7500 (with an ARM in it), so there's the possibility of offloading HTTPS there too.

Very interesting to hear about your experience with TI. I hope they have learn lessons and are better now with CC3200/3100. At least they offer free IDE option now.

ESP8266 is indeed very interesting for its price. Although no hardware encryption, its MCU could be good enough to post tweets. There are quite a few discussion threads with mixed outcomes about SSL. I found these most informative:
https://github.com/ParsePlatform/parse-embedded-sdks/issues/5
https://github.com/nodemcu/nodemcu-firmware/issues/134

Have to keep an eye on the progress.

You might look at IFTTT. They've just added a "Maker" channel. URL endpoints to call and webhooks back to your applications, and obviously their Twitter channel handles the oAuth so all heavy lifting is done for you.

Default is HTTPS but the endpoints respond on HTTP also.

@Ollie that's fantastic. I bugged them about doing something similar it a while back, but no reply.

Nice so see they've listened to all the people that have been asking them for an easily hackable endpoint. I'm amazed there was never something there right from the start.

Wait, IFTTT actually works now?

I was never able to make anything work, even simple stuff. I had a little LED (kickstarted for too much $) that plugs into a USB port, and you could control via IFTTT, but it never worked except occasionally (and seemingly randomly) while I was trying to debug it. I gave up.

It's really nice and easy too.

@aerialist_user6911 I've just posted up some example code that'll let you tweet from Espruino

@aerialist_user6911 now the Pico supports HTTPS, I wonder whether your code would finally work?

There's also require("crypto").SHA1 as well which might tidy things up a little?