fs.writeFileSync() not working with serial link

Posted on
  • Thanks to Wilberforce I was able to write a constant string to flash memory. While a constant string is nice, my long term goal is to serve a single page app from flash or a SD card with a file size of 1, 2MB or more. My next baby step was to write a single file of ~800 bytes to flash. Unfortunately after dropping my baud all the way down to 9600 to send a file in binary form to my ESP32, I still lose arbitrary blocks of data. It might be 1 to 20 bytes, but it means I am a long way from sending 200-500 KB file to the my ESP32 flash or SD card. Here's my code(mangled from over a week of hacking) and sample file. The need to write data from a serial link is not only for saving server files, but also for being able to save production data from other devices.

    
    var formatFlashDrive = false;
    var writeFile = true;
    
    Serial2.setup(9600, { tx: D17, rx: D16 });
    var fileWriteStarted = false;
    var fileName = 'index.html';
    var fs = require("fs");
    var intervalID, existingBlinkRate = 0, timerCount = 0;
    Serial2.print("Are you there?");
    
    if(formatFlashDrive){
      console.log('Formatting FS - only need to do once');   
      try {
        console.log("Flash Files: ",fs.readdirSync());
        } catch (e) { 
          console.log('Uncaught Error: Unable to mount media : NO_FILESYSTEM'.e);
           E.flashFatFS({addr : int=0x300000, sectors : int=256, format : bool=false }); 
      }
    }
    if(!writeFile){console.log("Ready to write to fs");}
    if(!writeFile){
      console.log("Reading index.html from flash");
      try{
        console.log(fs.readFileSync("index.html"­)); // prints "Hello World!!!"
        Serial2.write(fs.readFileSync("index.htm­l"));
      }catch(e){
        console.log("File Read Error = ",e); 
        Serial2.write('File Read Error');
      }
      //flashLED(10);
    }
    Serial2.on('data',function(data){
      if(!writeFile)return;
      if(!fileWriteStarted){ 
        setTimeout(function(){
          console.log('Reading ',fileName);
          Serial2.write(fs.readFileSync("index.htm­l"));
        },2000);
        fileWriteStarted = true;
        console.log('Writing data');
        fs.writeFileSync(fileName,data);
        return;
      }
      fs.appendFileSync(fileName, data);
      console.log('Appending data');
    });
    

    1 Attachment

  • Rather than using serial, you can use http to update the flash.

    You can do an http request and then write the output to fs, you could host the file on your local computer, it out the file in github or gist.

  • @Wilberforce
    Thanks for your input. While this might solve my short term problem, it will not solve my long term goal. In my production environment I will need to save, serve and update dozens of non static (10-100KB) files on top of several large static files totaling 1.5-3MB for my SPA . The non static files will need to come in serially, being that one of my connections to the environment will be RS485. I mention RS485 because of the limitation of flow control, if I don't have a way of taking data at at least 19200 baud, hopefully higher, I will need to rethink the Espruino implementation idea.

  • When you do Serial.setup you should be able to specify a cts pin. I'm not 100% sure it works on ESP32, but it should: http://www.espruino.com/Reference#l_Seri­al_setup

    Otherwise, it might make sense to set up some kind of serial protocol with a command and acknowledgement?

  • @Gordon
    Thanks for responding.
    I do not know of a way to implement a cts function with RS485, which is a two wire bi-polar uni-directional interface. I can only guess about interrupts, buffer sizes, buffer overwrites,being a problem. But being that with ESP-32 is new to Espruino and not many others are experiencing a similar problem, I will have to wait or find and pay someone knowledgeable about the ESP-32 and Espruino to fix this.

  • Ahh - with RS485 you'd have trouble with CTS flow control. I'd suggest implementing a command & response sort of thing then... What you're doing won't work that well on any Espruino device: you get data from the serial port a few bytes at a time, and each time you're opening and closing a file... it's just taking too much time.

    If you don't want to do some command/response thing you could use E.openFile to open the file and keep it open, which would help - and could also write in blocks of 512 bytes - which would match the block size in FAT

  • It's the esp32 port so the blocksize is 4096.

  • @Gordon and @Wilberforce thank you for your help.
    My plan for large data captures was to implement request/response protocol, but as initially stated, I cannot capture <800 bytes at 9600 baud reliably. My existing infrastructure requires 19200 baud and I would like to go higher.

  • @Wilberforce
    Have you checked your github account lately.

  • Post a reply
    • Bold
    • Italics
    • Link
    • Image
    • List
    • Quote
    • code
    • Preview
About

fs.writeFileSync() not working with serial link

Posted by Avatar for miCatotonic @miCatotonic

Actions