Espruino 1v63 out now

Hi there,

Just a quick explanation of some of the stuff that's in 1v63:

First the fun stuff...

Streaming File API

While you could load files before, you could only load an entire file into memory - you couldn't load a bit at a time.

Cephdon's done a bunch of work on this, and I just integrated it. We've settled on a nice simple file API:

f = E.openFile("hi.txt", "w");
f.write("Hello World");
f.close();

f = E.openFile("hi.txt", "r");
f.read(5) // .. "Hello"
f.read(5) // .. " Worl"
f.read(5) // .. "d"
f.read(5) // .. undefined
f.close();

I don't like deviating from the standards that everyone else has, but in this case every JavaScript file API I've come across seems unsuitable, so coming up with a nice clean File API seemed the best way forwards.

Piping files

This is where the streaming file API really shines. Because you couldn't load part of a file into memory before, you couldn't do things like serve up a webpage that wouldn't fit into memory. Now it's dead easy:

function onPageRequest(req, res) {
  res.writeHead(200, {'Content-Type': 'video/webm'});
  E.openFile("rick.webm").pipe(res);
}
require('http').createServer(onPageRequest).listen(80);

The code above will serve an arbitrarily-sized file directly from the SD card.

This uses standard JavaScript too, so you can actually make your own endpoints for pipes if you want to. I'll be working on making more of Espruino's built-in classes accept piping in the future - for example the Waveform class could stream sound directly from the SD card, or could even record sound and output it over HTTP!

Better HTTP stream support

Part of the work above involved tweaking the HTTP server and client to allow streaming. They now emit drain events when they're empty, so you can easily not just serve, but compute webpages that would never have fitted into memory before.

For example, perhaps you're making a data logger and storing the last 10,000 samples in RAM. This is fine, but you can't send a webpage full of them, because if you convert that binary data to ascii, it could take up 40,000 bytes and use up all your memory.

Instead, you can use the drain event to send the data in chunks...

var history = new Uint8Array(10000);

// ...

function onPageRequest(req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  var n = 0;
  res.on('drain',function() {
    for (var i=0;i<10;i++)
      res.write(history[n++]+",");
    if (n>=history.length) res.end(" ];");
  });
  res.write("var data = [");
}
require('http').createServer(onPageRequest).listen(80);

Fixes/tweaks

And finally, I guess you won't notice these, but:

• edit(...) is now able to edit functions that are in use in callbacks - for example with setInterval
  
• Sparse arrays - raybellis contributed some code that really helped out here. There was some non-standard behaviour before when adding/removing elements, and it's basically sorted now.
  
• Faster Utility timer (so the max speed for waveforms should have gone up)
  
• Errors on long lines now display nicely, and the pointer to the error location is better aligned in all cases
  
• When you get an error in one function called from another, stack traces are a lot more useful now too
  
• Added HIGH/LOW constants for better Arduino compatibility (a few people had bumped into that)
  
• Improved documentation on SPI/I2C/Serial
  
• CC3000 auto-reconnect - this seems to fix a lot of outstanding WiFi issues. It's been running for 48 hours non-stop not without issues.
  
• Martin's put in a load of time and has fixed a lot of the problems that were plagueing WIZnet Ethernet. It's looking really good now.
  
• E.reverseByte has now been added - it's ends up being quite useful, especially when dealing with displays.
  
• Support for bigger Espruino chips. We've found that a lot of you (maybe all of you) actually have chips on your Espruino boards with 64kB of RAM (even though they're marked as having 48kB). While the existing binary won't use this memory automatically, you can now compile your own binaries (or use someone else's) that will.
  
• Fixed the header sent by the HTTP client when accessing ports that aren't 80
  

OH HELL YEAH!

Awesome changes! Can't wait to try some of this new stuff out. Great to see file piping finally done, and that the internet connectivity finally sounds like it's working. The streaming enables some pretty awesome things.

While the existing binary won't use this memory automatically, you can now compile your own binaries (or use someone else's) that will.

You rang?

http://drazzy.com/espruino/espruino_1v63_espruino_1r3_bigram.bin
http://drazzy.com/espruino/espruino_1v63_espruino_1r3_wiznet_bigram.bin

(2600 jsvar, 64 ram 512 flash) Built with latest git as of like 40 mins ago, which I assume is the v63 release. My builds are untested.

Nice. Thanks!

Just had a quick look on one of my boards and your build seems to work great :)

Great job,
main topics for me are file support and bigram.
I'm working on a kind of datalogger with customized display.
Based on better filesupport I will be able to load some nice gimmicks from file to display. Working on a simple charting Module with lines, bars, points (and pie?)
Bigram will (hopefully)give me more freedom for extended functions.
One question around filehandling.

• Is it possible to read from and write to specified location in a file ?
  

That could be really good - it'd be nice to have a way to display more data than would fit in to RAM.

I also added a BMP Loader Module so you can load bitmaps off the SD card - it might be handy.

About file handling - yes, it's possible. You just use skip to skip ahead to the location you need. If you want to go backwards, you have to re-open the file. The reasoning is that it's something that can be applied to streams.

var f = E.openFile("file.bin","w+");
f.skip(10); // move 10 bytes
f.write("Hello"); 
f.close();

var f = E.openFile("file.bin","r");
f.skip(10); // move 10 bytes
console.log(f.read(5)); // read 5 bytes
f.close();

Note: something's broken in the build so the version that got uploaded is a few days old. I'll be updating 1v63 shortly though.

Ok, fixed now. If you updated to 1v63 before now and the build date under settings->about says '12th May' you'll need to update again...

Wow - great set of additions - excellent work :)