You are reading a single comment by @Gordon and its replies. Click here to read the full conversation.
  • Thinking back, I wonder whether the 'overwrite on 0' behaviour was actually to avoid problems with file uploads - since these start off just by writing address 0.

    Suppose you upload the file Hello world 123 in two chunks, but now you upload the file Hello world 124 - you only find out the file has changed too late to create a new file, so the upload will fail.

    That's amazingly common when uploading a JS app, when you might change some stuff and re-upload, keeping the file size the same.

    So I guess if writing to address 0 we could read the old file to see if it had already been written at that address, and if so could re-allocate a new file.

    However this seems amazingly hacky to cope with this one edge case, which you could solve just by having a single sacrificial byte at the start of the file

  • I wonder whether the 'overwrite on 0' behaviour was actually to avoid problems with file uploads - since these start off just by writing address 0.

    Why not to just call Storage.erase(file) from webide before calling Storage.write from offset 0 instead, if the intent was to erase the file before upload? Feels cleaner than relying on 'overwrite on 0' magic.

    So I guess if writing to address 0 we could read the old file to see if it had already been written at that address, and if so could re-allocate a new file.

    Don't get it. You need to know when to erase file with some existing data and when not? When starting to write same data from offset 0? Why? Would erasing it as mentioned above solve it or is this another case?

    However this seems amazingly hacky to cope with this one edge case, which you could solve just by having a single sacrificial byte at the start of the file

    Well, without understanding the context of using Storage to (re)upload files from Web IDE the 'amazingly hacky' is the way the Storage API works now regarding offset 0 ;-) Because without explaining it first the suggestion of 'having a single sacrificial byte at the start of the file' as a workaround feels exactly like that ;-)

    Was just advocating for API with least surprises. Having writing to existing file to offset 0 truncate the file and using exactly same call with offset 1 not truncating it is confusing (=the sequence in post #7, lines 13 vs 17). However in reality one could prevent losing data like this by simply not writing to offset 0 after some data is already written before - that is even typical use case how to write to file.

    More annoying on this is the need to write first block in special way - with data and total size together, or call it always like that - with 4 parameters including total size in each write call in a loop. Everything is possible to solve of course if you know that it works like this.

    Becasue the 'naive' way of first preallocating/creating file with no data like write(f,"",0,size) and then writing in a loop starting from offset 0 write(f,data,offset) breaks (as seen in post #5 line 7).

    So there are actually two issues - lost data, lost preallocated file. Both are caused by having offset and size as integer with default being 0 instead of JsVar. Optional offset could fix writing to location 0 (=create new file when not present, keep alone when present, even being zero), optional size would fix truncating size of previously preallocated file when size is not specified again when later writing first data to offset 0 (=check for matching size if present, keep alone if not).

    Also maybe it could make the code cleaner as the semantics of single write of whole file vs repeated write to prealocated file with offset would be easier to distinguish from the way it is called. Lines 626 - 650 look quite complicated due to the need to solve/guess all those combinations
    https://github.com/espruino/Espruino/blo­b/ea8ba1c2ff9c9bfa1d19f5c5911818e0a43c7f­4f/src/jsflash.c#L626

About

Avatar for Gordon @Gordon started