Inherit/extend Uint8Array -- how?

Unfortunately, one of the reasons Uint8Array is compact is because it doesn't store class info inside the structure (that might not be 100% JS compliant I'm afraid, but it does save memory). You could just add your functions to Uint8Array.prototype instead?

However, it looks like you're storing 4 x 8 bit numbers for the colour? In which case you could use a simple number instead - which is even more compact - and just shift the colours in/out as needed.

So you could just add to Number.prototype and your colours would be super compact, using just one variable per whole colour. It's actually pretty neat - but obviously you'd want to be careful with your naming so it didn't interfere with other things.