Espruino plugin for Rollup module bundler

@Gordon to get an idea on how it could work please have a look at https://rollupjs.org/repl

It demonstrates rollup running within the browser and its capabilities. It is cca 230k of payload (doesn't contain the minifier though). Uses service worker to cache things for subsequent app loads.

Wow, yeah - that does look really promising. We'd need to ditch their service worker since the IDE has its own, but that should be pretty easy.

If we can get that in the IDE and it's under 500k that'd be perfect. Does it handle require ok? I didn't see that in any of the examples. I saw one of the examples uses Object.freeze which Espruino doesn't have at the moment, and I'm sure there are a few other issues we'd hit as well - it sure seems like it'd be worth it though :)

Does it handle require ok?

Yes, it just needs a plugin to handle that and as the showcase is to actually use the ES6 import they just don't have it in (rollup-plugin-commonjs). I had a look at that and it would require changes to be able to work in the browser but nothing too complicated IMO. Also the terser (uglifyjs-es successor) should not be a big problem to put in.

In order to use the tree shaking functionality though the ES6 import needs to be used (more on that here https://rollupjs.org/guide/en#tree-shaking). That I find quite fine to use in the application code. The modules usually do not import other module parts. We could go even further IMO perhaps as far as saying that const connect = require('SSD1106').connect is the same as import { connect } from 'SSD1106'; (but that is just a thought for the future implications of which would have to be investigated first).

I saw one of the examples uses Object.freeze ...

It is used for the import * as xyz functionality and there the Object.freeze use can be turned off (see an example in the repo at https://github.com/rollup/rollup/blob/c3228cde9f4358ec08c349bef123b3289e33b8d3/test/form/samples/freeze/_config.js)

Actually I just tried import * as name from "module-name"; and it does tree-shaking there based on the usage of the module too - so hopefully require would still allow tree-shaking (just maybe not in all cases).

I think realistically 90% of users will still use require because that's what all the existing Espruino docs use/suggest, so it'd be good if it could be made to work. Seems a shame if after all the work of adding rollup, most people never actually saw the benefits.

But yeah, this all looks really promising, With terser as well this could be a really good system.

so hopefully require would still allow tree-shaking

By default it would not as per https://rollupjs.org/guide/en#tree-shaki­ng. We'd have to devise some hack as I mentioned earlier.

If all rollup integration solves would be those es6 minification issues I'd be cool with that. ;)

It looks like rollup with the commonjs plugin is quite clever. It does tree-shaking out-of-the box. The following outputs are outputs of rollup + commonjs + terser runs inside a browser (1.2MB minified).

Using import:

'main.js:' 
  import { test as printTest } from './lib.js';
  import { run1 as runTest } from './run.js';

  printTest();
  runTest();

'/lib.js:' 
  function test() {
      console.log('test');
  };
  function test1() {
      console.log('test1');
  };
  module.exports = { test, test1 };

'/run.js:' 
  exports.run = function() {
      console.log('run');
  };
  exports.run1 = function() {
      console.log('run1');
  };

'bundle.js:'
(function o(){console.log("test")})(),function(){console.log("run1")}();

Using require (not sure why it adds the dummy module.exports={}) at the end):

'maincjs.js:' 
  const printTest = require('./lib.js').test;
  const runTest = require('./run.js').run1;

  printTest();
  runTest();

'/lib.js:' 
  function test() {
      console.log('test');
  };
  function test1() {
      console.log('test1');
  };
  module.exports = { test, test1 };

'/run.js:' 
  exports.run = function() {
      console.log('run');
  };
  exports.run1 = function() {
      console.log('run1');
  };

'bundle.js:'
(function o(){console.log("test")})(),function(){console.log("run1")}(),module.exports={};

I understand that the Espruino community is diverse. For some the very light weight usage has the most weight, where for others it cannot be heavy enough to produce the most lightest code... I like the word play.

May be now is the time were plugins have to be treated as plugins all the way -not just in regard to the invocation style: plug-ins can be (auto) installed... Would that solve the issue of getting Espruino for the simple - fun - minded started in less than 15 seconds - and for the production minded in stages as needed.

@allObjects I agree (if I got all the words right ;)). This is exactly the reason there is the buildEspruinoConfig (see at github).

It returns ready-to-go rollup config based on minimal input should contain all the necessary rollup plugins configured properly to support the default Espruino environment. I assume we can build it into the IDE so that the experience stays as lightweight as possible.

All that while providing a way for production minded people to tweak the setup or build it completely from scratch to fulfill their special needs.

@Gordon So I managed to put this gist together with the rollup in browser capability for now to play with. The result is ~1.3MB minified (should gzip to at least half of that) and it feels quite snappy once loaded.

https://gist.github.com/opichals/0083c3079268db44997c6dd3969669a7#file-readme-md

Wow, nice - thanks! So with that gist you basically end up with a big JS file that you can just call espruinoRollup on and get a minified, tree-shaken result from?

Only thing I can see at the moment is that we need the modules to be loaded ahead of time because all the filesystem calls there are sync? But even so that could be done using the initial module loading code (it'd just have to be extended to deal with imports).

Only 'local' modules needs to be fed into it through the modules options. The rest is normally downloaded from espruino.com/modules as in the IDE (is uses the opichals/rollup-plugin-espruino-modules to do that).

With regards to module loading it is missing the GitHub and npm repository capabilities mentioned http://www.espruino.com/Modules but those are faily easy to add.

@Gordon It seems shaping up nicely the memfs was patched upstream and I added JSON file import support as well as GitHub url support (to mimic the EspruinoTools:plugins/getGitHub.js).

Just noticed the npm modules support removal so I added a couple of related PRs https://github.com/espruino/EspruinoDocs/pull/456, https://github.com/espruino/EspruinoWebIDE/pull/195.

That's great - thanks! I must have totally forgotten to keep that up to date.

@Gordon There is https://github.com/espruino/EspruinoTools/pull/80 which I'd appreciate getting feedback on. Would that be a way to go to integrate or are other ideas, requirements or comments.

One known issue:

• It looks like the terser config is tool agressive and although I researched the options I didn't find a reasonable way for it to not to strip the onInit() method if not called (having the top_level mangling on at the same time for the best compression ratio).
  
  

Fool the tool! - works even w/ the smartest one! - What would be a the half-liner doing so? ...like a if (getTime()<0) onInit(); as last item in top-level code?

I hope the win using the tool chain exceed by far the losses caused by above code half-line...

@allObjects I actually do such a thing here , but I do it only in the rollup flow and not for the minification only. Also I don't like it too much as it is too big of a hack IMO (also troublesome to remove from the resulting bundle). Perhaps there is another way to make sure this function is never inlined or renamed by changing the terser configuration .

I have yet to try the babel-minify as that one sounds like the one with the most prospective future.

Made some progress, fixed some stuff in rollup-plugin-commonjs and had other problem corrected in terser minifier and now it looks like it is doing what I would expect.

@Gordon The tools PR (https://github.com/espruino/EspruinoTool­s/pull/80 ) is now working in the IDE as well through https://github.com/espruino/EspruinoWebIDE/pull/197.

That's great - thanks! I'm literally only just back from the conference today so am catching up but I'll try and look at this next week.

@Gordon :) Were you on nodeconf.eu? People brought 3 badges from there to our office... I should have gone ;)

I was, yes! Where do you work?

:) I work @ apiary.io (acquired by Oracle almost 2 years ago) based in Prague, Czech Republic

@Gordon I did some more polishing of the PRs and they are from my point of view ready for pre-merge review.

Everything is off by default and needs to be explicitly turned on in IDE settings or in the cli job file.

That's great! Thanks - sorry for not looking at these yet, I've been a bit busy here.

I'm trying to use the new rollup-plugin-espruino-modules (the old one is not API compatible with Rollup anymore).

import { espruinoModules } from 'rollup-plugin-espruino-modules'
import commonjs from 'rollup-plugin-commonjs'
import { terser as terserPlugin } from "rollup-plugin-terser";

export default {
  input: 'src/main.js',
  output: {
    file: 'dist/bundle.js',
    format: 'es'
  },
  plugins: [
    commonjs(),
    terserPlugin({
      compress: true,
      mangle: true
    }),
    espruinoModules({
      board: 'ESP8266_4MB'
    }),
  ],
}

Running rollup fails with this:

Error: ENOENT: no such file or directory, open 'C:\dev-esp\espruino-cc1101-q7rf\modules\.board_ESP8266_4MB.json'

Mod: okay, never mind, I've figured out. I needed to create the modules directory by hand, otherwise it fails. It's probably a bug.

Am I correct that this new module cannot upload the code to the board after bundling? I've seen the older plugin had the port/baud/save options for this, but not this.

Also is it okay to run the Espruino plugin after the Terser like this?

Check this link to get the needed JSON file

http://www.espruino.com/json/