Problem with the new Storage module

Posted on
  • I am trying out the Sorage module. The example in the documentation works fine on my board. However a small change in the code brought me to a problem:
    I would start with an empty file of the required size and then populate the file with content:

    var f = require("Storage");
    f.write("a", "", 0, 20);
    console.log("File-Size: ", f.read("a").length);
    f.write("a","Hello",0);
    f.write("a"," ",5);
    f.write("a","World!!!",6);
    print(f.read("a"));
    

    This results in an error:

    File-Size:  14
    Uncaught Error: Too much data for file size
     at line 1 col 18
    f.write("a"," ",5);
                     ^
    Uncaught Error: Too much data for file size
     at line 1 col 25
    f.write("a","World!!!",6);
                            ^
    Hello
    

    And another question: How to determine the size of a file without reading it? The file might be bigger than the available memory and the "list()" method just lists the file name.

  • It looks like your issue is f.write("a","Hello",0); -Storage assumes that if the offset is 0 you intend to write a new file, so it just creates a new file with 5 characters.

    Instead, just combine the first write and you're sorted:

    f.write("a","Hello",0,20);
    

    How to determine the size of a file without reading it?

    http://www.espruino.com/Reference#l_StorĀ­age_read says:

    This function returns a String that points to the actual memory area in read-only memory, so it won't use up RAM.

    Basically running f.read("a").length is just as efficient as if there were a special-purpose length function, so we just saved some space and left it out.

  • Does the size parameter always determine the total file size even when I use it with a offset > 0? E.g.

    f.write("a","World!!!",6, 14);
    
  • It looks like the size gets ignored once the file itself is created.

  • I found out that it does not get ignored. You can specify always the total file size.

  • Do you have some example code? I did some tests before I wrote that reply and found that it didn't reallocate the file if the size was different.

  • This is my module saver program. It works now and can be used to save modules stored on a webserver onto esp8266 flash memory. Start the program and then key in on the left side:
    s.save('MyModule');

    class StModule {
      constructor () {
        this.st = require('Storage');
        this.wifi = require('Wifi');
        this.http = require('http');
        this.server = null;
      }
      
      init (ssid, pw, server) {
        this.wifi.connect(ssid, {password: pw}, (err) => {
          if (err) {
            console.log('Connection error, cannot connect to ', ssid);
          } else {
            console.log('Connected to ', ssid);
            this.server = server;
          }
        });
      }
      
      save(name) {
        var url = 'http://'+this.server+'/'+name+'.js';
        this.st.erase(name);
        this.http.get(url, (res) => {
          var offset = 0;
          var sz = res.headers['Content-Length'];
          res.on('data', (data) => {
            this.st.write(name, data, offset, sz);
            offset += data.length;
          });
          res.on('close', () => {
            console.log('Module '+name+' saved, size: '+this.st.read(name).length);
            console.log(this.st.list());
          });
        });
      }
      
      show(name) {
        print(this.st.read(name));
      }
    }
    
    exports = StModule;
    
    var s = new StModule();
    s.init('MyWlan', 'mypassword', '192.168.88.117:8081');
    //        SSID        Password       webserver's url
    

    I had to find out the size of the module. It was stored in the header of the received code.
    then it needs to specify the total module size for each chunk to be written to the flash:

    this.st.write(name, data, offset, sz);
    

    Altough some times i get an error which might be from a buffer overrun in the http.get module.

  • I have found the problem of the mentioned error. After calling Storage.compact() the error vanished.

  • Post a reply
    • Bold
    • Italics
    • Link
    • Image
    • List
    • Quote
    • code
    • Preview
About

Problem with the new Storage module

Posted by Avatar for fanThomas @fanThomas

Actions