Avatar for MisterG

MisterG

Member since Oct 2021 • Last active Oct 2021
  • 2 conversations
  • 7 comments

I love Espruino.

Most recent activity

  • in Bangle.js
    Avatar for MisterG

    FWIW I have been successful at over a 30K ~1000+ line file (which would minimize to 9K with comments removed) upload to a Pico. While doing this I kept an eye on available memory.

    This is completely off topic, but on an ESP32 with 4MB of flash space I routinely upload a 180K program file from the Web IDE that gets minified down to 90K or so, and everything works fine.

    I post this just as a reference. The Web IDE certainly works in that case...

  • in ESP32
    Avatar for MisterG

    @fanoush That's super interesting. I had actually often thought that because the Storage module's space is so tiny, that of course I'm going to run into problems quickly. For example -- my logs are now filled up again just after 3 days. What do you mean exactly by "external flash"? How is this differentiated technically from "internal flash"? The PICO I'm using (M5StickC) is just an ESP32 with integrated flash -- I believe -- but I assume it's just wired up via pins like any external flash module.

    Your Magic 3 smartwatch sounds interesting -- 14K vars (woo) -- > competes well with the TTGO watch @ 20K :P I'd love to know more.

    @MaBe That's ALSO super interesting. It could well be related. The last save from storage huh? I'll keep that in mind. I feel like I need to write some test code for this to hammer it out. I actually wrote a small test program to evaluate the effectiveness of your /targets/esp32/main.c modification for the HEAP memory on HTTPS requests.

    There had been some discussion in the GitHub around 45K, 55K, 60K, etc for the heap memory. And so I wrote this code that would run on some ESP32 modules for 24 hours and record success / failures. The technique could be used here (ie: fill up a bunch of Storage space in different ways, then run Storage.compact() and see what happens).

    Tomorrow I will "Storage.compact()" my device -- as it's full -- just 8K left using Storage.getFree(). I'll take a note of some settings and stuff and see what happens, and report back...

    G

  • in ESP32
    Avatar for MisterG

    You might be able to solve this easily enough by setting a flag for when you are doing HTTP access and on the timer side. What seems to happen is that during a page request a timer is called and then you get this error.

    What I found solved it was if you set a flag like:

    function pageRequest(res, req) {
        servingPage=true;
    
        // all your code
    
        servingPage=false;
    
    }
    

    and then after in your setInterval function

    setInterval(function() {
        if (servingPage) return;
    
        // your code here
    
    },milliseconds)
    

    Then this generally resolves the problem.

    For some reason the timers for Espruino in the ESP32 port are very fussy and what I've learned is refactoring your code to use as few as possible really makes your app run more smoothly.

    G

  • in ESP32
    Avatar for MisterG

    What were the results of process.memory() before entering the compact command?
    By chance mixing Storage and StorageFile?

    I had about 1000/4000 JS vars available (ie: around 3000 used). I can check again next time I have to do this which would be in a couple of days. That's when the logs would fill up.

    Yes I'm definately mixing Storage and StorageFile. I use storage for single variables or arrays and StorageFile for logs/files that will be appended to, which from what I understand is how it is supposed to be used I think. Is that supposed to be an issue? As far as I understand StorageFile is just a series of regular Storage files with a backslash and number appended.

    Hi @MisterG while the ESP32 falls under the not 'officially supported' category, have you tried the same on an authentic Espruino or maybe consider supporting the community over Patreon?

    Of course this is always an option, but often the "official" Espruino hardware has been designed with a philosophy of running for two weeks on a CR2032 and BLE. This is great, because PixlJS and BangleJS and PuckJS are really great and super fun. However, the ESP modules are really suited for when you need 20,000 JS vars and some heavy internet lifting like HTTPS. So it's hard to find an overlap sometimes. I think the Patreon route is the best idea in this case.

    I think the Storage module is really fantastic and as I explore it I enjoy it's features more and more. For examples it seems to me you can load code directly into it and Espruino will read that code if you "eval" it, and it won't take up JSVars really. Also just being able to save your state between power cycles / reboots is awesome. However it should be reliable. Also it could be bigger :P

    So what I'm saying is I encourage you to explore and we can continue the discussion. I might try and create a small program to recreate the problem over the weekend. You gave me a good idea: Espruino Linux seems to support the Storage module. I could try and recreate the scenario there and see what happens. This would give me an idea if it's a more general problem or something on the ESP32 port side.

    What is your bizarre conversion issue?

  • in ESP32
    Avatar for MisterG

    Hi everyone. I've been reading the posts on GitHub and here in the forums about making the storage module space bigger. It seems that the limitation is that it is "memory mapped"? But what does that mean "memory mapped"? I couldn't find exactly what that meant?

    My storage area shows about maybe 220K free? I find that small, considering I have 4-16MB of flash space typically.

    Is it possible to expand? I saw someone else ask about this somewhere, but no one responded to them.

    Thanks! :)

  • in ESP32
    Avatar for MisterG

    Hi everyone. You know I love the storage module, but I'm still having issues with it.

    Here's my scenario. I have an app which uses this pattern:

    https://www.espruino.com/Data+Collection­

    and writes a CSV of data in "append" mode (actually to three seperate CSV files).

    So my storage gets full over a couple of days. So it appears that the standard procedure in this case is to download your data and then run storage.compact() to free up space. However this almost always corrupts my storage and I have to reflash everything. This happened constantly on v2.07. So every few days I was reflashing and reconfiguring my modules. I saw there were some changes to fix storage.compact() after in the changelog, so I have upgraded to v2.10.

    So this week my CSVs wrote their data, and the storage got full, so I did a storage.compact().

    But the same problem persists. When I ran storage.compact() I got "Got [ERASED] expected EOF". I used to get that on v2.07, but I could still get my data through the Web IDE. Now for some reason on "reset()" v2.10 ERASES everything, complaining of corruption, which isn't really a great default behavior, because often lots of files are still readable.

    I've included my console output. The sequence of events are:

    1) I get a flash full notice from my app (not shown).
    2) I do storage.compact().
    3) I get a "Got [ERASED] expected EOF" error.
    4) I do a "reset()" and the system erases my data.

    I'm using an ESP32. Any recommendations? Maybe I should "reset()" my module BEFORE doing "storage.compact()", in order to not have open "append" files?

    Anyways here's my output:

    
    >storage.compact()
    =undefined
    Uncaught SyntaxError: Got [ERASED] expected EOF
     at line 1 col 1
    [ERASED]
           ^
    in function called from system
    >c.sf()
    Uncaught SyntaxError: Got [ERASED] expected EOF
     at line 1 col 1
    [ERASED]
           ^
    in function "sf" called from line 1 col 6
    c.sf()
         ^
    >
    Uncaught SyntaxError: Got [ERASED] expected EOF
     at line 1 col 1
    [ERASED]
           ^
    in function called from system
    Uncaught SyntaxError: Got [ERASED] expected EOF
     at line 1 col 1
    [ERASED]
           ^
    in function called from system
    Uncaught SyntaxError: Got [ERASED] expected EOF
     at line 1 col 1
    [ERASED]
           ^
    in function called from system
    Uncaught SyntaxError: Got [ERASED] expected EOF
     at line 1 col 1
    [ERASED]
           ^
    in function called from system
    >reset()
    =undefined
    WARNING: gap set scan error code = 103
    WARNING: set rssi scan not implemeted yet
    WARNING: config adv data failed, error code = 103
    WARNING: check error not implemented yet:103
    Storage is corrupt.
    Erasing Storage Area...
    Erase complete.
     ____                 _
    |  __|___ ___ ___ _ _|_|___ ___
    |  __|_ -| . |  _| | | |   | . |
    |____|___|  _|_| |___|_|_|_|___|
             |_| espruino.com
     2v10.33 (c) 2021 G.Williams
    Espruino is Open Source. Our work is supported
    only by sales of official boards and donations:
    http://espruino.com/Donate
    >
    >
    > 
    
Actions