On B2 - Deleted through the AppLoader.
Checked no remanents of files left through the IDE.
Installed through the App Loader
Started accelog App, pressed start
App flashes for a second and returns to menu.
Am using FW 2.10.27.

Checked console log in Chrome Browser, no obvious errros.
Console log of install attached.

Accellog v0.2 appears to be broken on both b1 and b2 ?

What I see is a quick flash of the overwrite YES/NO dialogue then it returns to the accellog menu. Nothing on the console when connected via the IDE.
Apologies have not tried to fix as I have not got my head around the new Layout module yet.

If you ever find out a way to reproduce this, please let me know!
Sure - its a bit elusive.

. I've just updated the widget!

Fantastic. Will update tonight.

Still puzzling over why the test harness results seem different to what I measure in reality but will investigate with the latest widget first as its the only way I can get the step count out of the firmware.

I installed BeepClock and got an out of memory issue after the screen had timed out a couple of times. Reproduced a second time. I'm using the TapeLauncher not the standard Launcher though.

Execution Interrupted during event processing.
Interpreter error: [
  "LOW_MEMORY",
  "MEMORY"
 ]
New interpreter error: LOW_MEMORY,MEMORY
Execution Interrupted during event processing.
Interpreter error: [
  "LOW_MEMORY",
  "MEMORY"
 ]
New interpreter error: LOW_MEMORY,MEMORY
ERROR: Ctrl-C while processing interval - removing it.
Execution Interrupted during event processing.
Interpreter error: [
  "CALLBACK",
  "LOW_MEMORY",
  "MEMORY"
 ]

Not sure about the code below at the end of Beebclock.js, multiple calls to Bangle.loadWidgets() a good thing ?

Bangle.on('lcdPower', (on) => {
  if (on) {
    Bangle.loadWidgets();
    Bangle.drawWidgets();
    drawAll();
  } else {
    clearTimeout();
  }
});

Set the default clock to something other than beebclock, then delete beebclock using the AppLoader, then restart the Bangle with a BTN3, see if problem persists.

Also, not sure what everyone thinks, but how do you feel about Storage Corrupt, erasing...?

Storage Corrupt - Yes. Not so sure about the erasing but once the corruption is deteced its needs to be pointed out regularly so that its not missed. Maybe some sort of crash dump analysis file could be generated that could be retained, uploaded, then manually deleted. This could record the answers to common questions like - what apps were installed, what was the config, mem size at the time. I'm not talking a full system image but some useful debug info that might help nail the problem down / eliminate possibilities.

@Gordon - interested to look at examples of apps that do the myapp.boot.js trick.

Storage should have been pretty empty. Only App running was Accellog, widpedom and a couple of other widgets like widbt etc. Would have expected used RAM to be less than 70% used. Kitchen Combo would have been unloaded from ram, that can go up to 80% ram. Watch was just logging me sat at the desk working. Just noticed that it had frozen. Being fairly religeous now about deleting any old accellog files in case they push things over the edge. Its not happenned since and have done a couple of other 1 hour logs since then.

Results of testing javascript version 3.13 will test the Firmware v2.10.27 once its clear how widpedom is handling the passes through the state machine.

I am wondering about widpedom though and the step event.

Have now flashed my bangle 1 and 2 with 2v10.27 so should be able to do some real world tests against the actual firmware now rather than the javascript version.

Does Widpedom get 6 step events when the firmware passes through the state machine to reach 6 steps ?

  Bangle.on('step', (up) => {
    let date = new Date();
    if (lastUpdate.getDate() == date.getDate()){
      stp_today ++;     // only gets one step when 6 would be returned by the state machine
    } else {
      // TODO: could save this to PEDOMFILE for lastUpdate's day?
      stp_today = 1;
    }
    ....

During the day there can be 200 passes through the state machine that would result in 6x200 steps. 1200 steps. If the firmware only returns 1 step for these events then at the end of the day widpedom will be 1000 steps short.

Have just verified experimentally that widpedom is only counting one step for the first 6 steps of the state machine. I think either a) Bangle.on('step') needs to provide a step size value of 1 or 6. OR b) the firmware will need to send 6 step events when it moves from S_STEP_22N to S_STEPPING state OR c) need a Bangle.getSteps() which will return the number of steps since the last reboot or something like that.

Thanks for that. The test harness is basically using the same code that the step counter is using ? So I was scratching my head and realised that my javascript version does

function onAccel(a) {
  var m = a.mag;

Hence wondering if a.mag comes from the accelerometer chip or its calculated in the firmware ?

but the IDE should do this too

It works if the code is loaded into the right hand side window of the IDE and upload to RAM. I'm uploading the js file through the IDE by clicking the STORAGE icon and selecting Upload File into the storage on the device. Then I got an run the modified App from the launcher. I guess this must bypass the way modules are designed to work.

M value? Acceleration magnitude?

Yes Magnitude.

We should really be doing the same stuff in the test harness.

Yes - need to be the same so we can do predicatble modelling that will match the real world measures when actually walking with the device etc.

I was not sure if the accelerometer chip does the calculation for magnitude OR the firmware does it. In the test harness I can see the In32_sqrt() function. Is this the same in the Bangle firmware. Was not sure where to look in the code.

Found this in ./libs/banglejs/jswrap_bangle.c

JsVar *jswrap_banglejs_getAccel() {
  JsVar *o = jsvNewObject();
  if (o) {
    jsvObjectSetChildAndUnLock(o, "x", jsvNewFromFloat(acc.x/8192.0));
    jsvObjectSetChildAndUnLock(o, "y", jsvNewFromFloat(acc.y/8192.0));
    jsvObjectSetChildAndUnLock(o, "z", jsvNewFromFloat(acc.z/8192.0));
    jsvObjectSetChildAndUnLock(o, "mag", jsvNewFromFloat(sqrt(accMagSquared)/8192.0));
    jsvObjectSetChildAndUnLock(o, "diff", jsvNewFromFloat(sqrt(accdiff)/8192.0));
  }
  return o;
}

Found this in
./targetlibs/nrf5x_15/external/micro-ecc/curve-specific.inc

static void mod_sqrt_default(uECC_word_t *a, uECC_Curve curve) {
  ...
}

Not really sure how this compares to:
Espruino/libs/misc/stepcount.c

// quick integer square root
// https://stackoverflow.com/questions/31117497/fastest-integer-square-root-in-the-least-amount-of-instructions
unsigned short int int_sqrt32(unsigned int x) {
  unsigned short int res=0;
  unsigned short int add= 0x8000;
  int i;
  for(i=0;i<16;i++) {
    unsigned short int temp=res | add;
    unsigned int g2=temp*temp;
    if (x>=g2)
      res=temp;
    add>>=1;
  }
  return res;
}

Apologies I think * is confusing the code formatter

With FM 2.10.09 (might have been 2.10.15).

Was attempting tog capture a 1 hour log of accelerometer data and had a freeze on Bangle 1 after 1 hour.
The App just frooze, nothing was updating on the screen.
On BTN3 reset came up with 'no clock found'
When connected through IDE - found a corrupt file with a filename of digits.
Deleted it and re-installed Bangle via AppLoader / About / Install My Favourites.
And recovered watch Apps / Configuration.

I only had a few Apps in the Storage and no other data so should have been enought storage for 2 hours of accellog data ?

Here's what I did.

Checkout the latest code from
https://github.com/espruino/BangleApps/apps/s7clk/app.js
Upload app.js through the IDE , click on the storage icon, navigate to file, change name to 7clk.app.js.

Restart the s7clk App and it will fail ?

// this works on the Right hand side of the IDE , if you send it to RAM
// but wont work in the actual app.js file
require("Font7x11Numeric7Seg");

The App loader does something secret in the background to add the font module into the source file that is mimised - maybe ?

I have also uploaded the Font7x11Numeric7Seg.js file onto the bangle and this makes no difference. The only way that the App will work is if its installed through the App Loader or Run in RAM. If you edit the code and upload to storage it stops working and gives this exception.

I did some actual realworld tests and came to the conclusion that the test harness results are not matching what you would get in a real world physical test. However the situation is complicated. In order to investigate will need to record the steps according to the AmizFit and also the current version of the step counter running on the bangle. Then compare with what the test harness gives.

My final conclusions about what is working are based on real world tests rather than the test harness results. Our users will only every test this way.

8 steps and 17 as the Thresold turned out to be too hard = it reduced the impact on DRIVING but at the expense of the accuracy of real world walking.

• 15min Driving: Amix 19, Bangle 9 (test harness predicted 80 if I recall).
  
• Walking Amiz 6605, Bangle 3.12 5261, 79% - bad result.
  
  

Tried v3.13: 6 Steps / Threshold of 14.

SLEEP Amiz 0, 3.13 65
DRIVE Amiz 60 3.13 90 -- most of this was the first 400m. ** (see note below)
WALKING Amiz 3191 v3.13 3158 -- 98.8%
DRIVING Amiz 3, v3.13 = 78

More real world test cases required BUT Here is my justification for the pay offs between
X in X and the RAW threshold:

This is what we ideally want in terms of behaviour from our pedometer:
SLEEP: Less than 100 steps - this loses 1% of in 10K steps - ACHEIVED
SOFA/DESK work - 0 steps or very low. - ACHEIVED
DRIVING - Ideally no more that 100 steps per hour - Not really acheived
WALKING - We want 98-100% accuracy - ACHIEVED - need more results

Lets consider a typical day to consist of:
SLEEP, DRIVE, WORK, WALKING - if the driving is a commute it may be 100-200s per leg of journey. With a return journey you could rack up 400 steps DRIVING in the day. But not
everyone drives everyday.

** HOWEVER - I have observed that DRIVING is very non deterministic even with the AmixFit. The
same drive one day will record 0 steps, the next day it will record 60 steps. The different
bumps in the road, traffic conditions, etc produce a unique journey as far as the Accelerometer
data is concerned.

My final push on the firmware repro will be to set X=6 and RAW_THRESHOLD=14.
I have these set on v3.13 on my bangles and will continue some tests.

Future Work
-Gather a good solid test of test results. EG 5 measures of the SLEEP, DRIVE, WORK, WALK so that we can be confident we dont just have one off results.
-Investigate / Meaure how accurate the test harness is based measurements in the real world
-Gather more / longer accellogs of different mixed activies (but with timeslines of activities).
-Ask others to repeat some tests aginst FitBits and other smart watches if they have them.

Thanks for the links. Still not sure what is going on here. Once minute the app worked. The next after editing, it failed and then when replaced with the unedited version it fails again. There is no Font7x11­Numeric7Seg.js file on the device as far as I can tell and I cant see anything that would cause it to be installled. It feels like the App Loader does something behind the scenes to make the module somehow available that is not being done when you edit the s7clk.app.js file and upload a change. Following the fonts links - it say all you need to do is type:

require("Font7x11Numeric7Seg").add(Graphics);

But this fails in the IDE and in the App.
Had a look in the Fonts tutorial and the example embeds code in the app for the font.
But in the case of s7clk there is no code for the font.

Was hacking about with s7clk.app.js
I commented out the date and seconds display code and uploaded as s7clk.app.js
Now when I try and run the app I get an exception.
I reuploaded the original app.js back to s7clk.app.js but still get the same exception.

Where do these fonts live on the watch ?

Also wondering how the 7*6 works in the code.
It looks like you are scaling the x,y to work across bangle 1 and 2 but not sure what is going on here.

  var size = Math.floor(g.getWidth()/(7*6));
  var x = (g.getWidth()/2) - size*6;
  var y = (g.getHeight()/2) - size*7;

Ok - so I think in practice the Test Harness results are a bit on the optimistic side.
Just added a couple of recordings and logged what the Javascript version recorded.

Does the M value come from the chip or is it calculated by the firmware ?

gcc -std=c99 main.c -o main
github.com/gfwilliams/step-count
X_STEPS = 8, RAW_THRESHOLD = 17
File, Expected, Simulated, Diff, (Orignal)
HughB-walk-1834.csv, 1834, 1754, -80, (1093)
HughB-walk-2331.csv, 2331, 2143, -188, (1874)
HughB-walking-2350.csv, 2350, 2198, -152, (1042)
HughB-walk-6605.csv, 6605, 6012, -593, (3223) // actual 3.12 => 5261 ie -1344 under
HughB-driving-36min.csv, 0, 13, 13, (1199)
HughB-driving-29min.csv, 0, 46, 46, (1153)
HughB-driving-18.csv, 18, 78, 60, (580) // actual 3.12 => 9
HughB-working-1h.csv, 0, 0, 0, (760)
Hughb-work-66.csv, 66, 56, -10, (980)
HughB-mixed-390.csv, 390, 356, -34, (1871)
TOTAL DIFFERENCE 1455

Added 2 more logs, updated harness, done pull request.

I'm not entirely sure you got my point

Yes apologies, I realised a few hours later this is what you mean't.

Good point, probably wont matter as we have the state machine to get through, but that assumes the state machine has timed out. But I agree its one less false PEAK and logically better.

so long-term it feels safer to use than than to maintain two different step counters

Yes I agree. The javascript stepcounter app was only to allow quick prototype of ideas etc.
Its close to having served its purpose now.

Uploading a log of 1 hour working at a desk. 0 Steps recorded on the AmizFit GTS2.

Have just cloned the step counter test harness repository and ran it successfully.

Done a lot of experiments and arrived at 8 steps in 8, gated entry to the state machine based on a raw threshold of 17 and no threshold at all on the filter output. This works really well. I did a test with the gating removed first and the difference on counting non steps was in the 1000s.
I then wondered what the impact of having no threshold on the filter would be and the impact was positive.

Latest test run below.
github.com/gfwilliams/step-count

File, Expected, Simulated, Diff, (Orignal)
HughB-walk-1834.csv, 1834, 1754, -80, (1093)
HughB-walk-2331.csv, 2331, 2143, -188, (1874) I actually got 2195 on the JS version (3.12)
HughB-driving-36min.csv, 0, 13, 13, (1199) - lowest I have got it with no impact on walking
HughB-driving-29min.csv, 0, 46, 46, (1153) - lowest I have got it with no impact on walking
HughB-working-1h.csv, 0, 0, 0, (760)
TOTAL DIFFERENCE 469

Have done a couple of pull requests to the step-count test harness AND the Espruino repositories.

I think we now have a pedometer that works !
Will do some more logs soon.

but again, this is why it's really good to be able to run the code in a test harness

Totally agree. Once we have all the necessary logs and agreed algorithm, identified parameters that we tune against the data - this will be the only practical way to run the 100+ tests that might be required to optimise the 2-4 tuning parameters. I have installed Node.js / npm but I have no Node.js experience - so its going to take a while for me to convert my code to a test harness. Going to focus on getting good recordings for a bit longer. Plus track the firmware updates you are doing.

SLEEP TEST RESULTS
Amiz GTS 2 - 10
2.10.13 - 191 ** this is the extra steps coming in due to noise on the input
3.10 App - 169

As is, when gate_open becomes true, accFiltered is now suddely>0 and all it takes is
for the next sample after that to be lower and you've now got a step recorded.

No this is not the case.
The raw threshold works like this.

Count up to 3 consecutive ABS(raw) values that are over threshold - then open the gate
Now we have readings that are not noise and represent actual movement
The gate only turns off if active_sample_count-- counts down to 0 and it will take 3 consecutive out of threshold values for it to count down. This avoids the potential for counting steps when at rest.

The gate opens up quickly when there is real movement, stays open and closes down quickly when the movement stops.

The M of the accelerometer when the watch is at rest/stationary is biased in the 3-8 range.
As per the screenshot above. The noise is not balanced across the zero line which is what I
would have expected. I have thought of building that into the threshold code. Too early to tell.

personally I feel like it would be a lot cleaner to do the threshold after the filter.

Ideally I would agree but the amplication of the filter is not linear and very dependant on what gets through. With the 2.9.90 filter any input got heavily amplified. Also with a 12.5Hz sampling frequency we can only have a filter that works up to 6.25Hz - this introduces distortations and
errors etc.

The only way we could prove lineararity would be through mass tests and to try and get a set of real world recordings that had a range of increasing raw values in the output. The driving recordings are the most interesting for this. Using the theoretical response is not going to help when in the real world the signal has got low level noise and bumps in the round etc.

Why there need to be a threshold on the raw M input

Place the watch on a table and you will still see some raw m signal from the accelerometer:

523593,6,0
523673,4,0
523754,4,0
523833,6,0
523913,6,0
523993,8,0
524073,9,0 PEAK
524153,6,0
524233,5,0
524357,5,0
524393,4,0
524473,5,0
524553,6,0
524633,7,0
524713,7,0
524793,8,0 PEAK
524873,7,0
524953,6,0
525033,8,0 PEAK
525113,5,0
525193,2,0

This signal has PEAKs. When using PEAK detection it means that the step state machine will get called from very small movements - EG when sat on a sofa.

I have flashed 2.10.13 and am testing against 3.11.
15 mins sat on the sofa the FW has done 13 steps and 3.11 has done 6 steps.
The difference is that the FW has no minimum raw threshold that must be reached before
we take the filter output.

3.11 has this bit of code :

  if (raw_clipped > RAW_THRESHOLD || raw_clipped < -1*RAW_THRESHOLD) {
    if (active_sample_count < N_ACTIVE_SAMPLES)
      active_sample_count++;
    if (active_sample_count == N_ACTIVE_SAMPLES)
      gate_open = true;
  } else {
    if (active_sample_count > 0)
      active_sample_count--;
    if (active_sample_count == 0)
      gate_open = false;
  }

  if (!gate_open)
    accFiltered = 0;

What this code is doing is saying is the raw m value bigger than room noise for more than 3 samples.
The raw threshold is set to +-10. And to take the filter output the ABS(input) must be greater than the threshold (10) for about 1/4 of a second. This is necessary otherwise PEAKs in the chip noise or noise from the ground get passed into the state machine. We want to definitely moving before we start calling the state machine.

I'm convinced this concept is needed AND it can also be one of the parameters we use to TUNE the step counter against the accellog recordings.

Assume line 1 - is original code / filter.
As this is actual walking data - I would not expect the state machine to have to do much work or show much difference for that dataset. When I collected this data I tend to walk fairly quickly the 1 mile walk or so. I think there will be big difference when you run against the non walking recordings.

1) driving log 0 steps in 36 minutes
2) driving log 0 steps in 29 minutes

Its the driving that is the weakest part now. I suspect tweaking things to get these to 0 will massively reduce the accuracy of when counting actual steps.

Here are the things I think can be tweaked.
1) X steps in X seconds could be 5 or 6 or 7 - after 7 I think it starts to work against accuracy
2) The Raw threshold to gate the output of the filter, Currently +-10. Can go higher.
3) The time we want the raw signal to be over or under threshold before we open / close the gate.

I have a linux setup on my chromebook so I could setup the C version. I will do that once I feel I have got to the end of testing / loggng. I will also setup the Node.js as well.