Cycle race track lap counter

Posted on
Page
of 2
Prev
/ 2
  • @ChristianW,

    ...work keeps me more busy than I'd like right now, because I said I would run some of your code... did not happen yet... :

    BUT: thanks to the error reported in your last post, I found what I wanted to show you in a previous pst: how to queue up runtime statuses without IDE connectivity and no console. But that is not the point. You may already have found this post / conversation about Puck sending same key repeatedly?.

    It is about buttons... even though this is in PuckJS, it generally applicable. I hope this reference is of some help to you.

    A guts analysis - to verify - tells me that too many thing are going on at 'same' time... - too many js execution streaks stacked up and waiting for execution... why? not clear yet. Even it may not be much that is put into the js execution queue, over time it eats up the memory...

    Having studied your code before, I get no (glaring) hint that we deal with a memory leak here... I could though still be an issue. I know that you also throw out unknown connections... but may be to disable conectability altogether could be something to consider (and make it switchabel/togglable by a button on a free GPIO pin (if you still have one free). If not, you may consider to use SWButton (for bundling less used commands/button presses/functions).

  • Sorry for the delay getting back to you on this... I'd be pretty sure the freezing itself stems from the device getting swamped with Bluetooth advertisements.

    What happens is when an advertisment is received, it's poked into a queue in the interrupt, and handled by the event loop (where it is filtered). If the event loop doesn't manage to handle the items fast enough, then you hit FIFO_FULL and a bunch of issues stem from that.

    The event loop should limit the amount of packets it executes such that timers still work when it's swamped with data, but maybe that is failing somehow and it's just sitting there trying to handle BLE packets.

    jsble_exec_pending: Unknown enum type ...

    This shouldn't happen with 2v09 and later firmwares, but it seems like the queue got full and then only part of a Bluetooth message was added to it, which then caused things to get really messed up. How many devices do you have broadcasting? If I can reproduce it here I might be able to fix that particular issue.

    Looking at the code I think there are a few places you could help it out though. Mainly, you want the handler for setScan to be super fast, so you want as few function calls as possible (and to bail out as soon as possible if you don't care about the device). It looks like you have a few different function calls, and you could try something like:

    function scanHandler(d) {
      var v = Velo.list[d.id];
      if (!v || !d.manufacturerData) return;
    
      Display.rxCnt += 1;
    
      v.rssi = d.rssi;
      v.bat = d.manufacturerData[0];
    
      switch (v.state) {
        case 0:
          v.state = 1;
          v.rxCount = 0;
          v.rssiMin = null;
          v.rssiMax = null;
          break;
        case 1:
          if(d.rssi >= CONST.RSSI_THRESHOLD) {
            v.state = 2;
            Led.flash();
          }
          break;
      }
    
      var t = getTime();
      v.tRssiLast = t;
      v.rxCount += 1;
    
      if(v.rssiMax == null || rssi >vthis.rssiMax) {
        v.rssiMax = rssi;
        if(v.state == 2) {
          v.tRssiMax = t;
        }
      }
      if(v.rssiMin == null || rssi < v.rssiMin) {
        v.rssiMin = rssi;
      }
    }
    // ...
    NRF.setScan(
      scanHandler,
      {filters: [{manufacturerData: {0x0590: {}}}]}
    );
    
  • @ChristianW / @Gordon, should the swamping still happen, I suggest 'to turn the world upside down' with the use of an additional 'trigger' / 'solicitor' BLE Advertiser device and make the clients observe rssi and respond with limited advertising.

    The 'trigger' / 'solicitor' BLE Advertiser device works like a constant, continuous beacon and is placed at the same location as the lap counter BLE device. Passing by BLE devices scan and when they notice continuous drop of rssi they advertise for the lap counter as they do now, but only for a limited time. / count (or ack received - assuming (quasi-)simultaneous advertising and scanning is possible).

  • Thanks @Gordon, having the scan handler super short was also something I had in mind.
    Good that you confirm this, I will try to optimize it.

    There are currently 8 pucks available for me, so that is the maximum I should handle - would be nice if there were more.
    However "in the field" not all should be in range at the same time - only when starting the race.

    But it would be good if conditions like these would not break the whole setup.

    My main issue was the memory though I think. Limiting the log to 100 laps made it stable for now.
    I am considering pushing the lap data to an external SD card.
    Is that something you can advise or does it make things slower/worse?

    I would create a new directory for every one (timestamp of start time as directory name to make it simple).
    And then create just one short file for every lap event like (00001.csv) for the first lap, (00002.csv) for the second.
    No longish CSV file, short open and writes.

    But I would like to be able to "serve" this file data via BLE as well so it can be pulled via a Laptop at race time - I would have to be able to scan a directory on request and push out record (file) by record.
    Do you see issues here doing that in parallel?
    I'm about to hook up a SD card to the rig now and see how it works...

  • @allObjects thanks about the idea with the Advertiser.
    However communicating back to the Pucks would mean the Pucks would have to scan as well.
    I was worried about battery consumption here, so that's why I wanted to keep it simple on the transponder side.

  • SD card access can actually block for some time, which would make your problems a lot worse I'm afraid.

    How about just using the on-device storage with require("Storage"). There's about 40kB usually (and I could come up with a cut-down firmware that allowed you around 100k). I'd imagine for lap times that would be more than enough especially if written as binary, and it's nice and fast to write to (and easy!).

  • @ChristianW, power consumption considerations understood.

  • Hmmm. Blocking even if I use the async methods?
    I would keep things small and simple. Just create one directory when starting.
    And write a small file for one lap.
    That would still cause issues you think?

    About require("Storage") I am not sure actually how this could work.
    Maybe create an "empty" file first that fills up all available space.

    >require('Storage').getFree()
    =18440
    

    with my current code.

    And then use it as binary ring buffer writing on pos = i * REC_SIZE or something like that.

    With my current set of datapoints (I ended up at 10 bytes) this would be something like 1800 laps which would give me 225 laps for 8 transponders meaning about 56 km distance on a 250 m track which can easily be reached with a fast velomobile in one hour.
    Tight fit, but an improvement. I'll consider that - thanks.

    What about this cut-down thing? This would mean disabling every compile-time module I don't need for this board I suppose?
    Sounds interesting.

    EDIT:
    At some point I thought since my code is in Flash and RAM is about the same size as Flash there would be more space in RAM instead (and RAM doesn't wear and is easier to access...).
    Was that a wrong assumption?

  • PS:
    I blew up another MDBT42Q because I inserted the battery upside down... Only one spare left.
    The others are at FRAport waiting for customs.
    Hope they arrive this week, since the race is next Sunday.

    Thanks to Brexit shipping times are degraded back to the Napoleonic era...

  • What are the actual memory requirements you feel comfortable with?

    There are fast non-volatile serial - non-flash - memories available up to 2MB with not wait on write and now wear from 500KB 40 MHz SPI up to 2MB 108 Mhz QSPI.

    I've played around with some similar memories 256-Kbit (32 K × 8) Serial (SPI) F-RAM - Ferroelecric RAM - SPI challenges, and cross-developed a memory manager (incl. simple GC) for such memory ( post #14 as part of conversation about Running code off of an EEPROM. Capacities have come.a long way since then...

  • Thanks @allObjects, but since the day of the track race is already next Sunday there is not much time left to try something with external hardware.

    @Gordon I tried to implemented Storage yesterday, but it gave me headaches.

    EDIT: another error found. Don't put something like require('Storage').eraseAll() as first line in your code on upload... oh my.

  • OK - something works. I can somehow create a storage and fill it up with data.
    What does not work is removing it at runtime.

    So how to erase and recreate a Storage file on runtime?

    I can storage.eraseAll() but that would remove my code too.

    I can

    storage.write('name', [data], 0, large_value)
    

    but that gives me

    Uncaught Error: Unable to find or create file
    

    even if I 'overwrite' my current file.

    So I'll try to

    storage.erase('name');
    storage.compact(); // this seems to be necessary to actually free up the space
    storage.write('name', [data], 0, large_value):
    
    

    but run into errors too.

    Is Storage buggy or am I doing something wrong here?

  • What you're doing with storage.erase + .compact sounds fine to me. What errors are you getting?

    I guess one issue is that compaction will take some time so might make you lose Bluetooth packets again - however I feel like you probably don't need to do except when you reset it/go to start recording. Normally I imagine you'll just be adding data.

  • @Gordon I'd like to have a method to do a reset "in the field", meaning starting everything new when I have to.

    What is working is calling my reset() method via commandline.
    What does not work is calling it through a handler after a button was pressed.

    The method does this:

      reset: function() {
        console.log('reset');
        if(!Flash.s.write(Flash.NAME, [CONST.LAP_DATA_SIZE], 0, 1 + CONST.LAP_DATA_SIZE * CONST.LAPS_STORAGE)) {
          throw new Error('could not reserve storage');
        }
        Flash.pos = 0;
      },
    

    manual call:

    >Flash.reset()
    reset
    =undefined
    

    through handler:

    reset
    Uncaught Error: Unable to find or create file
     at line 1 col 114
    ...ATA_SIZE*CONST.LAPS_STORAGE))throw new Error('could not reserve stor...
                                          ^
    in function "reset" called from line 1 col 15
    a&&Flash.reset(),Object.keys(this.macs).forEach(c=>{const d=this.macs[c];...
                               ^
    in function "init" called from line 1 col 122
    ...tons.BTN_MID]&&(Race.init(!0),ErrorLog.log=[]),this.objects.forE...
                                      ^
    in function "wakeup" called from line 1 col 16
    a?Power.wakeup():Power.sleep()
                   ^
    in function "onOff" called from line 1 col 37
    ...N?Buttons.onOff(a,b.state==0):Buttons.event(a,b.state==0)
                                    ^
    in function called from system
    
  • Another issue:
    If I keep my storage file in flash and try to upload new code (minified + direct to flash), it gives me this:

    No errors. Minified 23072 bytes to 14425 bytes.
    >
     ____                 _
    |  __|___ ___ ___ _ _|_|___ ___
    |  __|_ -| . |  _| | | |   | . |
    |____|___|  _|_| |___|_|_|_|___|
             |_| espruino.com
     2v09 (c) 2021 G.Williams
    Espruino is Open Source. Our work is supported
    only by sales of official boards and donations:
    http://espruino.com/Donate
    >fficial boards and donations:
    http://espruino.comds and do$>DgGAMABgAOD4AAAwwEgBgAwAkBhgAAYACAAgAPAIAIAYAAAEGCFBEgkQUIMEAAH/yAJAEAAYADAAYADAAYAAQBIAn/wAAGAMAYADAAYAAAAIAEACABAAgAQAIAAQAEAAAADAKQFICkA+AAD/gIQEICEA8AAAPAIQEICEAkAAAPAIQEICEP+AAAPAKQFICkA0AAAQA/wkASAIAAAAPAISEJCEh/gAD/gIAEACAA+AAAQBPwAAABAAggSfwAA/4AQAYASAQgAAgAf8AAA/AQAIAD4CABAAfAAAPwEACABAAfAAAHgEICEBCAeAAAP+EICEBCAeAAAHgEICEBCA/4AAPwCACABAAQAAAEQFICkBKAiAAAIAfwCEBCABAAAPgAIAEACA/AAAMABgAMAYAwAAAPAAYAYAwAGABgPAAACEAkAMAJAIQAAD5ACQBIAkP8AACEBGAlAUgMQAAAgAQD3iAJAEAAf/AAEASAI94BAAgAAAIAIAEADAAgAQAQAAAFAHwFUCqBBARAAAACAOAAAAQQI/4kASAAAADgAAA4AAAEAAABAAAAQAAEACAH/AgAQAAAFACgH/AoAUAAAEAEAEABAAQAAAGMAYAwBjAAAAwAADEKRDIiiQRIEYAAAIAKAIgAAH4ECCBA/AkQSIIEAACDFChiRSIKEGCAADAAQAAAEAMAAADAAQAwAEAAABADAAQAwAAAAQAcAfAHABAAAAQAIAEACABAAAAQAIAEACABAAgAQAAAgAgAIAIAAACAB4AgAAAPAGADwAAAEQlIKkJKAiAAAIgCgAgAAAeAQgIQDwCkBSAaAAAIQkYKUJSAxAAAYACAQgAOEIAIAYAAAL8AAAeAQgf4EIBIAAATA+gkQSIAEAABBAfAIgEQCIB8BBAAAwAEgBQAeAUASAwAAAffAADCCYhKQjIIYAAEAAABAAAAH4ECCZBSgpQQIH4AAAQBUAqAPAAAAQAUAVAFAEQAAAQAIAEADwAAH4ECC9BU",1024);
    

    The only way to get new code flashed is removing the storage file first it seems.

    >require('Storage').getFree()
    =12116
    >require('Storage').list()
    =[
      "laps",
      ".bootcde"
     ]
    
  • Maybe I'll just switch back to RAM and just use binary, reserved data for now.

    Is there a way to safely acquire memory without crashing the system?
    Something like:

    1. determine how much memory we have left process.memory().free
    2. subtract some margin of memory left unallocated (how much would you suggest?)
    3. calculate how big my ArrayBuffer can get (any clues here?)

    Or I'll just do it via trial + error and put the hardcoded numbers in...

  • I'd like to have a method to do a reset "in the field", meaning starting everything new when I have to.

    Yes, that should be fine...

    Uncaught Error: Unable to find or create file

    Strange... Could you print the arguments you're giving to Flash.s.write? It might help to figure out if anything is 'off'.

    If I keep my storage file in flash and try to upload new code (minified + direct to flash), it gives me this

    Wow, I haven't seen that before. And it's reproducable?

    I guess maybe Espruino ends up compacting, but that compaction takes so long it causes the flow control to time out. Does the upload take a long time at that point? Do you get a FIFO_FULL message? Although I guess you might have got that at some point in the past.

    Is there a way to safely acquire memory without crashing the system?

    You should be able to do something like:

    m=process.memory();myArray = new Uint8Array((m.free-100)*m.blocksize)

    So then you're leaving ~100 memory blocks free for execution. But you may well have to tweak that so you don't get out of memory errors

  • Yes, it's reproducable.
    This is/was the actual line:

    Flash.s.write(Flash.NAME, [CONST.LAP_DATA_SIZE], 0, 1 + CONST.LAP_DATA_SIZE * CONST.LAPS_STORAGE)
    

    Being Flash.NAME = 'laps', LAP_DATA_SIZE=8, LAPS_STORAGE=2000 which would be:

    write('laps', [8], 0, 16001)
    

    which worked perfectly fine in the console, but not when called via a setWatch() callback when a button was pressed.
    I even changed all the this. references to absolute ones in case the context got lost inside the method.

  • Hmm, I just ran this on an MDBT42Q:

    setWatch(function() {
      require("Storage").write('laps', [8], 0, 16001);
    }, BTN, {repeat:true});
    

    and it works absolutely fine, so I feel like there's something else going on... Any chance you could try taking stuff out of your code until you're left with a minimal example that always exhibits the problem? Then I can reproduce here and get a fix in.

  • Thanks @Gordon, there was not much time left before the race and I had to hack everything together to get a semi-working setup.
    It actually worked to a point, but unfortunately the race was called off during warmup because of the wet weather conditions making the track slippery.
    Maybe there is another date in the next weeks, that may give me some time for improvements...

  • Post a reply
    • Bold
    • Italics
    • Link
    • Image
    • List
    • Quote
    • code
    • Preview
About

Cycle race track lap counter

Posted by Avatar for ChristianW @ChristianW

Actions