SD Card Directory listing and low memory

Hi all,
I've connected an SD Card via SPI to my Puck.js. I was hoping to get a full list of a directory (40+ files) and be able to iterate through them using a click. Unfortunately I get a low memory error when trying to read a directory. Is there any way to get a full directory listing? I don't mind recompiling firmware to make it happen.

Can you get the directory listing if you have just the code to read a directory in there? Or is it that you have a lot of other code that is using up memory?

It seems that listing 40 files shouldn't usually use up anywhere close to all of the Puck's memory.

If it's the case that you have lots of code there's probably a few things you can do to get memory usage down - http://www.espruino.com/Code+Style#other-suggestions would be a good start

I do have a lot of other code there and I've tried to use memory saving options where possible such as typed arrays. I'll update my functions to use objects also, but they mostly only have one parameter or two at most. Unfortunately the plan was to be able to read 150+ files in a directory at a later stage. Interestingly, saving files to flash and getting a list of them works fine.

I'll try it with an empty puck.js and just the code to read the SD card and compare the memory profile to my full app.

I do have a lot of other code there and I've tried to use memory saving options where possible such as typed arrays.

if you have typed arrays or long strings that are more or less static and read only, you can write it as files to storage. Storage.read returns direct reference to flash memory area so such strings and array constructed from it do not take up RAM.

Still there can be even more files on sd card so some API that would enumerate the list in smaller chunks or with callback would be nice. Maybe fs.readdir could return some iterator object? https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Iterators_and_Generators#iterators

That's a good idea, some are images so they can definitely be stored in flash. Some are constructed from what I read from the SD Card though but they're relatively small. I've also re-written most of the JS to reuse pointers rather than creating new variables.

I like the idea of using an iterator, I was thinking about adding offset parameters to readdir but I think returning and iterator would be better. I'll investigate. Also the ability to get a total count, but that should probably be a separate function call. I like that idea a lot.

Yes, longer term some kind of API with an iterator (or given they're not really used in Espruino, a callback) might be really handy - but right now that's pretty low priority as you're basically the first person to have this problem in 10 years :)

I thought the same as having searched the forums, I couldn't come across something similar. I ended up implementing a function that allowed me to provide an offset and length, allowing me to iterate through the directory structure in chunks. I realise it's very niche which is why I've not raised a PR for it. I appreciate the help and feedback, gave me the direction I needed to go in.