File system API improvement

I'd like to use the espruino for data logging purposes and transmit the log data to a host by WiFi. The file system should be used to buffer data in between connections, but the file system API is only able to work on files as a whole at the moment. So an indexed block read/write improvement would really help, and/or even better a file object (or handle) that was able to keep the file open.

I haven't yet looked into the Espruino source code, but I'd like to help on building the API. What environment/toolchain are you using for building Espruino (I'm sure I've read that somewhere earlier on, but I've forgot where it is).

I use Linux + Codesourcery GCC for building - there's info on the GitHub page.

However @cephdon has been working on exactly what you're after - I don't know what state he's at with it though.

The plan was to implement a stream API - much like Node's. That would mean that you could access files the way you're after, but also that you could simply open them and then pipe the stream to the HTTP response - which would make it really easy to send big files.

... it'd be pretty handy when paired with the Waveform API too - if it allowed you to play something straight off the SD card.

I've found some information about building Espruino in the forums.

I have an alpha version of the api complete and sent it to Gordon. It doesn't support async piping of data yet, but it does do reading and writing on open files in an indexed way. You can specify the number of bytes to read and the position to start reading and writing.

The async pipe code is currently done on my computer, but I need to merge down the latest code and test before I send a pull request.

Sorry guys.. I tried to push my changes in, but philosophical differences got in the way.
To those of you looking for block reads/writes, I am sure they will provide one that implements the node API eventually (well, some of it). Mine was apparently too different for what they wanted. (Basically, I am saying don't wait on my changes, they wont be arriving.)

Sorry about this - by 'philosophical differences', @cephdon means that some of the people hanging out on Espruino's GitHub wanted something node.js-compatible...
however not enough to actually do anything constructive about it.

When I get some time next week I'll try and implement @cephdon's code... Hopefully it'll arrive in 1v62.

I can't even express the level of relief I felt when I found out that they didn't represent Gordon and Espruino officially. Sorry for the late update to my previous statement, at the time, I was under the false impression that they did.

If streams will come, will they be compatible with any socket? As if it will allow stream over TCP socket, that would be "mindblowingly" amazing :)

That is what I was working toward. Right now, all I have is a beginning, and only with files.

Yes, that's the plan... I'm also pretty interested in making it work with Waveforms. For instance being able to stream 60 seconds of audio from SD card would be pretty cool :)

I just want to stand up for the guys on Espruino's GitHub a bit though. From my experience, Open Source software projects generally have the attitude that it's basically up to the contributor to do everything required to make their code just slot in.

At the moment, that's not my attitude with Espruino at all - so few people contribute to Espruino's code that everything is valued, even if I end up having to tweak it a bit. That means that there's a bit of a culture clash when people who're used to the 'committer fixes it' approach on other Open Source projects request changes on pull requests - especially when I've been talking to people like @cephdon and urging him to send me the code so I can take a look at it!

@Gordon,
Re: guys on github,
I think there must be a line drawn somewhere. If people begin demanding what they think is best, without talking to the project owner about it, then the direction of the project gets muddled. You and I have had many conversations about these changes, so I thought my original implementation was slotting in with your code.

The difference came in the way it was handled and the manner of the discussion. I won't be shy about it, though, if they did represent espruino, I was not going to contribute anything more to the project. That's the danger of having people who can influence change in your project without your consent. They have the ability to change the perception, right or wrong, of the project in general.