How to use setBootCode?

I'm probably a bit dense today, but I'm not quite seeing how to use E.setBootCode. What I'd like to do is save some low-level modules that are stable to make more space for my own code. For example, I'd like to save the MCP_23008 module, which I have in a file MCP23008.js. And then I'd like to be able to iterate uploading my_project.js which uses the MCP23008 module. How do I do this?
I suppose I start loading MCP23008.js into the IDE, check the Save on send flag, and hit the upload button. Now the module is saved. I can reset() for good measure. But it's not really loaded as a module, right? And when I now upload my_project.js the module registry gets cleared anyway, right? Or do I need to add something like var mcp = require("MCP23008") at the end of the MCP23008.js file to get a reference into a variable? Or...

NB: have you thought about how a target like the esp8266 could use a different bigger flash area for the setBootCode?

... save some low-level modules that are stable to make more space for my own code

The issue is that if you save Modules.addCached("...","...") using setBootCode then it's a string, so any functions defined in it can't be kept in flash. You could do something like this:

global["\xff"].modules = { myModule : (function() {
  exports = {};
  // my module code
  return exports;
})() };

In reality you want a little more fudging, as for some modules you need the global variable 'module'.

I guess it might make sense for the Web IDE to automatically do that instead. Even without saving to flash it's actually more efficient, as the module code can be loaded straight from the input string (so needs less memory and time during the upload).

To be honest I'm not sure why it didn't do that from the beginning.

And when I now upload my_project.js the module registry gets cleared anyway, right?

Yes, Modules.removeAllCached() get called (this could probably just not be sent now, as reset() is usually called before anyway). Nothing stops you from doing Modules.removeAllCached=function(){}; to disable it though.

have you thought about how a target like the esp8266 could use a different bigger flash area for the setBootCode?

Could you not just change the location used for saved code to one that was bigger? That would be by far the easiest way.

I mean, sure, you could add some kind of filesystem, but it's probably not worth the effort.

Is the 512k version of the esp8266 still usable ( @jumjum) reported issues on gitter. (I've cooked my esp01 so I can't test).

If longer supported, there is a 20k block of flash that could be used:

0x07C000	496KB	16KB	512KB flash: Espressif SDK system params, else unused
0x080000	512KB	4KB	Unused

Perhaps this area could be used for E.setBootCode, or to store modules?

The 512KB versions still work fine.
WRT picking a larger chunk of flash, the issue is that the location is #defined and comes from the board definition, so it's the same for all esp8266 variants. On the 512KB variant the current location is the only one available.
One thing I could do is set the location to some "virtual address" and then remap it for the esp8266 in jshFlashRead/jshFlashWrite. That would also allow me to string the current location plus additional small bits on other modules together into 32KB. Not pretty but it would be nice...

I guess so... Not sure exactly where that #define is used, but I'd be happy if you wanted to turn it into jshFlashGetSavedCodeAddr or something?

@wilberforce ,tested new version on ESP8266 512KB
@gordon ,should this be moved to ESP8266 ?

• combined version (espruino_1v86_esp8266_combined_512.bin)­ from http://www.espruino.com/binaries/ works fine.
  
• creating my own combined version (some extensions to handle WS2812 matrix) works fine
  
• flashing from tgz-file in travis does not start
  
• creating my own flash based on last version from github does not start
  
• creating my own tgz without crypto and graphics works fine
  
  

So, there is a problem with my flashing, any idea ?
I'm using the nodemcu firmware programmer with:
boot_v1.4(b1).bin to 0x00000
espruino_esp8266_user1.bin to 0x01000
blank.bin to 0x7e000

@jumjum - it's good to know the release ones work...

Now, I wonder what is up with your versions? Have you got the math lib .o file that was introduced a while back - I guess you must have otherwise it would not build?

The other thing to try would be an erase first, then a flash.

Rather than joining two bits.....

@tve
Here is an idea that would prevent the need to virtual addressing:
For the esp8266 module 512.. Everything stays the same.

If the EPROM is larger, use the current save code location, but extend from 12k to 16k, and keep the compressed code.
Then the next region I mentioned above, 16k could be used for saveBootCode and modules, and this area would not be compressed so the strings are directly usable.