Memory use

Dear Gordon, couldn't you answer to some questions about memory use.

• Is it possible to change the memory distribution between code and data? If I have a little code, but the memory isn't enough for data, is any way to increase a data memory allocation at the expense of the programm memory?
  
• Where is located the cache of modules? It seems that the modules are loaded to data memory.
  
• Is it true that it load all modules (even enclosed) that are in code at once? Can I hold over a loading of module till the its execution? The matter is that when I use a 'require' too often the memory comes to end quickly...
  
• Do minifying and optimization work properly? My WEB IDE can't load a code to Espruino when optimization options are switched on.
  

Hi,

I'll just answer in order:

• code and data are shared already
  
• modules are also shared
  
• yes, it's true. There are other threads on it here but you can do m='module'; ... require(m) to force the web ide not to load modules. Check out the reference for the Modules class - you can easily flush the cached modules
  
• Yes, they work fine on 'simple' optimisations. Your problems may be because you're not using an official board, and nobody seems particularly bothered about fixing the lack of flow control on non-f103 boards.
  
  

By the way, 1v70 will be much more memory efficient when it comes out

Sorry, I've not fully understood...

• What does it mean: "code and data are shared already"? The data and program are stored in memory always separately and there is no way to reallocate the memory, is it true?
  
• I cannot understand where modules are stored... Sometimes when I'd released the cache no free memory cells (that I ask by 'process.memory()') was added. Somethimes per contra very many cells are released by 'Modules.removeCached("xxx")'...
  
• My board is HY_STM32_24ve. So far I can see, it's an f103-boards, isn't it? And no problems must be with minifying and optimization...
  

Code and data use the same memory so there is no need to reallocate.

If you load a module but keep a reference to something it provides, that may stop any memory being freed.

Yes, the hy is an f103 board. However if you connected via the 'serial' usb connector you could have problems, as well as if you turned on 'advanced minification'

Mmm, you speak that the memory is common for data and program. But the 'process.memory()' tells me that there are always 2000 words (4K) allocated for data regardless of code length. STM32VET6 has 64K SRAM, but I can have variables that take up only 4K... It's not very well.

I use modules often inside the subroutines in local variables (for example, var a = requires("aaa")), and I beleave, that, when the programm leave the subroutine, the memory for 'a' is released automatically, isn't it? But I've understood so, that in this case it release only the reference to cache, and after leaving of function I must remove the cache. Is it true?

Yes, I use an usb connection by VCP. Is that a reason? And what can I do with this?
Yes, 'advanced minification' don't work at all. The 'simple minification' works not always. Sometimes. And when I see the dump after loading a code into the board, I don't see scarcely minification: all comments and spaces are not deleted, variables have long names etc.

2000 jsvars is 40k, not 4k. 20 bytes per jsvar. V70 and later uses 16 bytes per jsvar.

On Espruino v69, with the bigram hack on the offical board, you get 2600 jsvars from 64k - 52k for the jsvars, and the rest is used for the stack and JS interpreter.

I'm not sure why your board has 64k of ram, but is only being built with 2000 jsvars - I'd expect you could get 2600 with v69 and 3250 with v70, unless there's something different about that chip.

Is JS interpreter allocated not in flash? What is the Espruino's bin? Is it not a JS interpreter?

Any way the RAM is not enough catastrophically... Seems the way out is to save temporarly unusable variables to SD-card.

Will 'advanced minification' work ever properly?

And when will the v70 be to probe? ;-)

My understanding is that most of the JS interpreter is in the flash, but it still needs that ~12k for itself and the stack.

And yeah, I've been constantly up against the RAM limit on my serious Espruino projects. My desk lamp controller uses over 2100 jsvars, plus for some operations, it needs another 100-200 - and I haven't even started work on integrating the nixie tube display (largely since I haven't finished building it - it's over an hour of soldering per digit)...

You can also minify your code with one of the widely available javascript minifiers - I was doing that before minification was working correctly for me (simple minification is now working 100% for me - though it won't minify anything if there are any syntax errors.)

It seems that the biggest drain on memory is code itself - this is where all my space is going, at least. One of the problems is that the language was designed for general purpose computers with oodles of ram, relative to the size of the code. So there's stuff like String.fromCharCode(), and all the other common builtin functions and libraries have long human readable names. Also, long global variable names (which don't get renamed) can be a killer if you refer to the variable a lot.

At least now (if you do a build yourself, at least) you can get 2800 jsvars and the E.getSizeOf() that's in github right now, which should help. Not sure why you only get 2800 jsvars - but that's what the board file says it gets - i'd think it could get 3250 with 64k ram.

Gordon has talked about offering a higher-memory version of the Espruino (there are pin-compatible chips with more ram), though I think he said he's doing the Espruino mini first (with 64k ram (ie, should be 3250 jsvars), lower price, and not as many IO pins).

I've also considered making modified versions of modules that strip out functionality I don't want. I'd like to make a PHP page to do the snipping automatically, so you could require() the url, passing the module name and functions to exclude, and the page would fetch the module, split it into functions, excise the specified functions, and return the result, which the WebIDE would then send to the Espruino.

You can also use an EEPROM to get more read/writeable memory, though it's certainly less convenient. That was what I had in mind when I wrote the module to interface with them.

@DrAzzy is right about the amount of variables. If you compile your own images it may be possible to increase the amount of available variables.

Previously there were issues with those boards when more variables that were used in them, but that may have been due to problems with the use of FSMC for LCDs. That's all done in software now so it may no longer be an issue.

Will 'advanced minification' work ever properly?

There was another question about it on here as well. It depends if anyone contributes the needed code - the closure compiler needs a description of every function that is provided by Espruino in order for it to reliably 'advanced minify' the code.

Judging by the response in another thread on here it may be that the 'advanced' option disappears, and some offline minifier gets used instead.

And when will the v70 be to probe? ;-)

Next week or possibly the week after. Of course there's always https://espruino-nightly.noda.se/ if you can't wait.

Ok. I'm testing it.
Exception seems to work ok.
Only free jsvars are as well as in 1v69 - only 2800 (seems, they must be 50K/16 aprox.3125)