Memory usage

Hi Gordon,

I just run into "out of memory". I tried to figure out where it happend and what can i optimise to prevent this. I saw that i had around 300 units (Junks of 20 bytes) free before i called a few cascaded functions that converts a string into an array of bytes, calculates checksums length and returns it.

To understand the usage in general i run the following test:

console.log(process.memory());
str='Hi Gordon, a string with 80 characters, that will be converted to an array of B';
console.log(process.memory());

Output:

{"free":1782,"usage":18,"total":1800,"history":3,"stackEndAddress":536909496,"flash_start":134217728,"flash_binary_end":134435552,"flash_code_start":134443008,"flash_length":262144}
{"free":1775,"usage":25,"total":1800,"history":14,"stackEndAddress":536909496,"flash_start":134217728,"flash_binary_end":134435552,"flash_code_start":134443008,"flash_length":262144}

7*20 bytes=140 bytes for a string of 80 chars. Nothing special.

Now i created an array of 80 bytes this way:

console.log(process.memory());
bytes=[];
for (var i = 0; i < 80; ++i) { bytes.push(i); }
console.log(process.memory());

The output:

{"free":1782,"usage":18,"total":1800,"history":3,"stackEndAddress":536909496,"flash_start":134217728,"flash_binary_end":134435552,"flash_code_start":134443008,"flash_length":262144}
{"free":1618,"usage":182,"total":1800,"history":15,"stackEndAddress":536909496,"flash_start":134217728,"flash_binary_end":134435552,"flash_code_start":134443008,"flash_length":262144}

That array needs 164 chunks of 20 bytes = 3280 bytes, nearly 10% of the available memory!?

Is this as expected ?

Thanks

Sacha

Yes. Simple Arrays (the ones you get with var x = [];) are 2 memory units per entry. Storing 1 byte per entry in them gives you ~2.5% memory efficiency.

That's why you should avoid them whenever possible (at least if storing data directly in them), and use typed arrays (like Uint8Array ) or strings instead (and why I've been agitating for a way to make i2c.readFrom() return either a string or a Uint8Array http://forum.espruino.com/conversations/934 ). The downside is that typed arrays are of fixed length - you can't splice and dice them like simple arrays.

I've been trying to do everything with typed arrays, unless the array is either tiny and not worth the size savings, or has to contain objects/arrays/strings/etc that obviously can't be put into a Uint8Array.

In your case, why would you not just leave it as a string, and use charCodeAt() to get the values as you need them?

DrAzzy's spot on. 2 memory units per entry and each memory unit is 20 bytes, so just by allocating an array you're using a lot of memory.

I have an issue open (I have no idea when I'll get around to doing it) that may halve memory usage in these cases, but you'll still run out of memory quickly when working with 'normal' arrays.

There's a bit of info here: http://www.espruino.com/Performance

You can create a Uint8Array and then use the .set function to set its value to that of a string (it's not strictly JS compliant, but it's handy :)

>var a = new Uint8Array(5) 
>a.set("Hello")
>a
=new Uint8Array([72,101,108,108,111])

Thanks DrAzzy, Thanks Gordon,

I didn't found the "Performance" Page before. Very usefull information. I have to reimplement/change most parts, including the xBee module.

Sacha