Understanding Uint8Array Memory Blocks Usage: 3 Bytes Data Consuming 3 Blocks (14 bytes per block)

You are reading a single comment by @user156224 and its replies. Click here to read the full conversation.

•

Gordon
There's a small bit of info on http://www.espruino.com/Performance#array-buffers-are-the-most-efficient-way-to-store-data

But basically Uint8Array/etc store their data as a string, but they have to have one variable allocated which explains what type (Uint8/Int16/etc) they are, with size and offset. They also have to link to an ArrayBuffer which then references the string, which is mainly so you can do:
```
a = new Uint8Array(10)
b = new Uint16Array(a.buffer)
a.buffer==b.buffer
=> true
```
If I just automatically made an ArrayBuffer every time someone referenced .buffer then the two buffers wouldn't be equal.

You'll say why can't the ArrayBuffer store data directly and the reason is that a String isn't just one datatype - we have string_1,_2,etc depending on the amount of characters in that string block - so we'd have to do the same with ArrayBuffer which would then use up more space in our enum than we have.

It can help to use trace(..) sometimes to see what a variable is made of:
```
>a=new Uint8Array([1,2,3])
=new Uint8Array([1, 2, 3])
>E.getSizeOf(a)
=3
>trace(a)
#3677[r1,l1] Uint8Array (offs 0, len 3)  #3676[r1,l0] ArrayBuffer (offs 0, len 3)    #3413[r1,l0] String [1 blocks] "\1\2\3"
```
So it shows Uint8Array->ArrayBuffer->String

@fanoush I'm not sure what happened in your test, but a 3 char string should only be one var:
```
>E.getSizeOf(E.toString(new Uint8Array([1, 2, 3])))
=1
```
So actually the docs are misleading. A String is the most efficient form of storage, but Uint8Array/etc allows you to reference those in a useful way (because normally Strings cannot be modified once created)

•

user156224 in reply to @Gordon

Thank you for the detailed explanation; it all makes sense now! I've included my code below where I store data in the "compressedData = []" array. Is there a way to save this data with fewer storage blocks or space?

var lastTimestamp = 1692048524472;
var lastElevation = 0;
var compressedData = [];


function compressData(sample) {
  // Split sample into timestamp and elevation
  var parts = sample.split("/");
  var timestamp = parts[0];
  var elevation = Math.round(parseFloat(parts[1]) * 100);

  // Calculate deltas
  var deltaTimestamp = (timestamp - lastTimestamp) / 10; // Time difference in tenths of milliseconds
  var deltaElevation = elevation - lastElevation;

  // Mask values
  deltaTimestamp &= 0xff;
  deltaElevation &= 0xffff;

  // Create buffer with 1 byte for timestamp delta and 2 bytes for elevation delta
  var buffer = new Uint8Array(3);
  buffer[0] = deltaTimestamp;
  buffer[1] = (deltaElevation >> 8) & 0xff;
  buffer[2] = deltaElevation & 0xff;

	console.log("size = " + E.getSizeOf(buffer));
	console.log("remaining blocks = " +process.memory().free);
  // Update last values
  lastTimestamp = timestamp;
  lastElevation = elevation;
  compressedData.push.apply(compressedData, buffer);
}

setInterval(function(){
compressData("1692048525472/294.35")
},500);




//the output is this:

size = 3
remaining blocks = 11820
size = 3
remaining blocks = 11817
size = 3
remaining blocks = 11814
size = 3
remaining blocks = 11811
size = 3
remaining blocks = 11808
size = 3
remaining blocks = 11805
size = 3
remaining blocks = 11802

•

fanoush in reply to @user156224

I've included my code below where I store data in the "compressedData = []" array.

so this is not about Uint8Array size since in your code it is only temporary variable. The remaining blocks going down by three is about the ordinary array having three integers added there by push.apply

> var d=[]
>d.push.apply(d,new Uint8Array([1, 2, 3]))
=3
>d
=[ 1, 2, 3 ]
>trace(d)
#30[r1,l1] Array(3) [
  #66[r1,l2] Name Integer 0 = int 1
  #62[r1,l2] Name Integer 1 = int 2
  #70[r1,l2] Name Integer 2 = int 3
]
>E.getSizeOf(d)
=4
>d.push.apply(d,new Uint8Array([1, 2, 3]))
=6
>d
=[ 1, 2, 3, 1, 2, 3 ]
>E.getSizeOf(d)
=7
>d.push.apply(d,new Uint8Array([1, 2, 3]))
=9
>E.getSizeOf(d)
=10
>trace(d)
#30[r1,l1] Array(9) [
  #66[r1,l2] Name Integer 0 = int 1
  #62[r1,l2] Name Integer 1 = int 2
  #70[r1,l2] Name Integer 2 = int 3
  #77[r1,l2] Name Integer 3 = int 1
  #78[r1,l2] Name Integer 4 = int 2
  #79[r1,l2] Name Integer 5 = int 3
  #83[r1,l2] Name Integer 6 = int 1
  #84[r1,l2] Name Integer 7 = int 2
  #82[r1,l2] Name Integer 8 = int 3
]

Understanding Uint8Array Memory Blocks Usage: 3 Bytes Data Consuming 3 Blocks (14 bytes per block)

About

Espruino