Assume you are referring to the min time threshold of 333ms for a step in the state machine. Good question. Yes its needed. In theory it should not be needed but we dont have a perfect filter and a perfect sampling rate. I have seen the lower threshold reject false steps when driving and other occasions. I have done a lot of watching of the debug output I print to the console so have a good feel for what is going on now. We could take it out and do test runs through the recorded data to see the impact. At the moment I dont think it is causing any harm or a major source of error.

One thing I am wondering if we need a different format for the recorder. The recorder logs the x,y,z componants - but in the code we use the m value which is calculated by the firmware. To eliminate another potential difference / source of error - I think we should record time and m only.

Hi @Gordon - I recorded a 6hr accelog recording. When I attempted to download I got the following being dumped to the IDE console and the watch hung and stopped updating the watch app. Had to do a reset.

ODg0LDQxOAoxMjc3OTQ2MCwyMDUwLC03OTAzLDM4MwoxMjc3OTU0MCwyMDQxLC03OTcyLDM3NQoxMjc3OTYyMCwyMDAzLC03OTc5LDM5NwoxMjc3OTcwMCwxOTU5LC03OTM5LDM5MAoxMjc3OTc4MCwxOTU1LC03OTYzLDM4NwoxMjc3OTg2MCwxOTU0LC04MDAwLDM5NAoxMjc3OTk0MCwxOTYyLC04MDQzLDM4OQoxMjc4MDAyMCwxOTgyLC04MDQ2LDM2OQoxMjc4MDEwMCwxOTc1LC04MDMwLDM2OQoxMjc4MDE4MCwxOTU0LC04MDQ5LDM2MQoxMjc4MDI2MCwxOTg4LC04MDQ5LDM3MwoxMjc4MDM0MCwyMDM3LC03OTk4LDQyMgoxMjc4MDQyMCwyMDE4LC03OTQ4LDQxOQoxMjc4MDUwMCwxOTkyLC03ODgyLDQzNQoxMjc4MDU4MCwyMDE0LC03ODYwLDQ1MQoxMjc4MDY2MCwyMDQxLC03
OTE4LDQ1MAoxMjc4MDc0MCwyMDcxLC03OTQ1LDQ0OQoxMjc4MDgyMCwyMDc2LC03OTkyLDM4MQoxMjc4MDkwMCwyMDYxLC03OTc5LDM2NgoxMjc4MDk4MCwxOTk1LC03OTMxLDQxMAoxMjc4MTA2MCwxOTU1LC03OTQzLDQ5MAoxMjc4MTE0MCwyMDA1LC03OTQ4LDUxOQoxMjc4MTIyMCwyMDExLC03OTQyLDQ1NgoxMjc4MTMwMCwxOTcyLC03OTM5LDQxOAoxMjc4MTM4MCwyMDEzLC03OTIxLDQzNQoxMjc4MTQ2MCwyMDQwLC03OTAzLDQ2OAoxMjc4MTU0MCwyMDE5LC03OTU1LDQ1OAoxMjc4MTYyMCwyMDI3LC04MDA1LDQyNwoxMjc4MTcwMCwyMDA0LC04MDAzLDQyMQoxMjc4MTc4MCwyMDIwLC04MDEwLDQxMAoxMjc4MTg2MCwyMDUyLC04MDA1LDM3MQoxMjc4MTk0MCwyMDA3LC03
OTc3LDQxMgoxMjc4MjAyMCwxOTk1LC03OTgxLDQ4MwoxMjc4MjEwMCwyMDA3LC03OTc0LDQ5OQoxMjc4MjE4MCwxOTY5LC03OTcxLDQ1NQoxMjc4MjI2MCwxOTYwLC03OTk1LDQyNgoxMjc4MjM0MCwxOTgyLC03OTk4LDQ1NQoxMjc4MjQyMCwxOTkyLC03OTgyLDQ2MAoxMjc4MjUwMCwxOTc5LC03OTQxLDQzNwoxMjc4MjU4MCwxOTc0LC03OTQ3LDQxOAoxMjc4MjY2MCwxOTYyLC03OTU4LDM5MQoxMjc4Mjc0MCwxOTY2LC03OTM4LDQwNQoxMjc4MjgyMCwxOTQyLC03OTM3LDQ4MAoxMjc4MjkwMCwxOTM5LC03OTIwLDUzMQoxMjc4Mjk4MCwxOTk4LC03OTM3LDU0NwoxMjc4MzA2MCwxOTk5LC03OTgwLDU0NgoxMjc4MzE0MCwyMDA5LC03OTc0LDQ5NAoxMjc4MzIyMCwyMDMwLC03
OTcxLDQ1MgoxMjc4MzMwMCwyMDE0LC03OTYxLDQwOAoxMjc4MzM4MCwxOTkwLC03OTU2LDM2MQoxMjc4MzQ2MCwxOTk2LC03OTgwLDM5MQoxMjc4MzU0MCwyMDA3LC03OTc0LDQxMwoxMjc4MzYyMCwyMDM4LC03OTYzLDQwNwoxMjc4MzcwMCwyMDY5LC04MDAwLDQwMwoxMjc4Mzc4MCwyMDc1LC04MDE3LDQ0NQoxMjc4Mzg2MCwyMDQ0LC03OTc0LDQ5MAoxMjc4Mzk0MCwxOTc4LC03OTU0LDQ2OAoxMjc4NDAyMCwxOTczLC03OTYyLDQwNgoxMjc4NDEwMCwyMDA1LC03OTU3LDM4NgoxMjc4NDE4MCwyMDI5LC03OTcyLDM4NQoxMjc4NDI2MCwxOTk3LC03OTYxLDQyOAoxMjc4NDM0MCwxOTgyLC03OTY1LDQyMgoxMjc4NDQxOSwyMDA5LC03OTg0LDQxOQoxMjc4NDQ5OSwyMDIyLC03
OTI1LDQ0NAoxMjc4NDU3OSwyMDExLC03ODk3LDQyMQoxMjc4NDY1OSwyMDA2LC03OTEwLDQyNwoxMjc4NDczOSwxOTk1LC03OTMzLDQzMAoxMjc4NDgxOSwyMDA3LC04MDAzLDQ0MAoxMjc4NDg5OSwyMDI5LC04MDI3LDQ0NAoxMjc4NDk3OSwxOTk3LC04MDA1LDQzOAoxMjc4NTA1OSwxOTY1LC03OTg1LDQyNwoxMjc4NTEzOSwxOTU4LC03OTc2LDM3OQoxMjc4NTIxOSwxOTU1LC04MDA2LDM2NQoxMjc4NTI5OSwxOTg4LC04MDAzLDQxOAoxMjc4NTM3OSwyMDIxLC04MDAyLDM5MgoxMjc4NTQ1OSwxOTg0LC04MDM1LDMyMAoxMjc4NTUzOSwxOTg1LC03OTkxLDI4NgoxMjc4NTYxOSwyMDI4LC03OTU0LDM1NgoxMjc4NTY5OSwxOTk2LC03OTcwLDQ4MgoxMjc4NTc3OSwxOTgyLC03
OTc4LDQ1NgoxMjc4NTg1OSwyMDUyLC03OTU2LDQxOAoxMjc4NTkzOSwyMDY3LC03OTM3LDQ3NwoxMjc4NjAxOSwyMDE0LC03OTYyLDUwNQoxMjc4NjA5OSwxOTY0LC04MDAwLDQ2NwoxMjc4NjE3OSwxOTQxLC04MDAwLDQzMQoxMjc4NjI1OSwxOTU2LC03OTg0LDQxNQoxMjc4NjMzOSwxOTYyLC03OTkzLDQwNwoxMjc4NjQxOSwxOTQ5LC03OTgwLDQyOQoxMjc4NjQ5OSwxOTY4LC03OTU3LDQzMQoxMjc4NjU3OSwxOTc3LC03OTQ2LDQ0MAoxMjc4NjY1OSwxOTk4LC03OTY3LDQzNgoxMjc4NjczOSwxOTkxLC03OTkzLDQwOQoxMjc4NjgxOSwxOTM2LC03OTgwLDQ3MwoxMjc4Njg5OSwxOTM3LC03OTQ4LDUxMQoxMjc4Njk3OSwxOTU0LC03OTE3LDQ1MgoxMjc4NzA1OSwxOTUzLC03
OTI3LDQyNQoxMjc4NzEzOSwxOTcwLC03OTgzLDQwMwoxMjc4NzIxOSwyMDEyLC04MDE4LDQxOQoxMjc4NzI5OSwyMDA2LC04MDE3LDQwOQoxMjc4NzM3OSwxOTk1LC04MDA5LDM4OAoxMjc4NzQ1OSwyMDA3LC04MDEzLDQyMAoxMjc4NzUzOSwyMDIxLC03OTY0LDQ0NAoxMjc4NzY5OSwyMDYxLC03OTQxLDQ0MgoxMjc4Nzg1OSwxOTg4LC04MDA3LDM5NAoxMjc4NzkzOSwxOTk1LC03OTUzLDQ1NwoxMjc4ODAxOSwxOTk4LC03OTM2LDQ4NwoxMjc4ODA5OSwyMDE3LC03OTE5LDQ2NgoxMjc4ODE3OSwyMDA4LC03OTQ1LDQwNgoxMjc4ODI1OSwyMDA0LC04MDAwLDM4NgoxMjc4ODMzOSwyMDExLC04MDE4LDM1OQoxMjc4ODQxOSwyMDE0LC04MDE3LDMzMAoxMjc4ODQ5OSwyMDMzLC04

new recording. Sitting for 1hr with 2-3 getting up and moving to a different room. The AmizFit recorded 58 steps for this.

ideally I'd like to be able to run it over some data and say for sure that yes, it does do a better job.

Can I beg you to implemnt the state machine as well in the firmware. It really is the only part of the algorithm that is accepting and rejecting real and false step sequences on every step at the moment. The current F/W implemenation does not do this well at all in my view. Its an essential part of the algorithm. If you look at the results I am getting with the javascript version - the improvement for non step counting is entirely down to the state machine. It may seem a bit more complex BUT I think the results prove its worth. It would potentailly mean you would have to change the STEP event to return X or 1 steps depending on if there had been a full pass through the state machine at the start of a sequence. But its a small price to pay in my opinion for a vastly improved step counter.

There is still a lot more tweaking and testing work to be done. But if you move to the same approach I am using in 3.10 we will be aligned. I am considering building a test rig so I can feed recordings into my javascript version so I can do quick tests of different values of X in X and the raw threshold etc.

Moved my latest experimental code to:
https://github.com/hughbarney/bangle_bits/blob/master/step_counters/steps_310_oxford.js

Here's 10 steps with the oxford filter. In my 3.10 version.
Obviously 2 traces is not enough - but seeing how the different filters change the outout characteristic I realised the only way to normalise the code between different filters was to use PEAK detection.

here's why we need PEAK detection not threshold crossing. Look at the steps after 6000ms. They are not crossing the orange threshold line. This was why when I took the filter taps into my App the state machine was not being called.

last record in above log is: 938968,-1468,217,-8059

Working on 2 sets of test results. 3.10 and 3.11 as same code except 3.10 uses the oxford filter. I'm leaning towards that one as it is looking better.

Here is an accellog for 1834 steps walk (as per AmizFit GTS2).

New firmware is better but still way out for none walking activities.

With regards to threshold - I will post some charts I did that compare the filter outputs when walking. The old filter had more pronounced amploification and also tended to centre the output wave around zero better. The new filter can have a full since wave above the zero line. This is why I have now coded a PEAK detection method. To do that I store the last 3 consecutive filter readings (Left Middle, Right), If the middle one is greater than the left and right reading then a PEAK has been reached. This seems to be much more adapatable, self tuning and reliable. You can see my code in the github link above.

What did you think about using the simpler oxford filter ? I'm testing the firmware one and the oxford one at the moment. Looks like the oxford one is even better.

summary of results so far.

I attempted to build a javascript version (3.91) using the new filter. But this failed. I noticed steps were not being detected as the 1000 threshold was not being crossed. This is a difference in the way the filter amplifies the input. Rather than tweak threholds I decided to try PEAK detection. This has the advantage of being self adjusting. However it does need a simple gating mechanism otherwise you start finding peeks in the noise signal when the bangle is at rest. When combined with the state machine it works really well. I have tested against the oxford filter but will go back and repeat against the bangle v2.10.9 filter taps and see which one looks best.

The C code of the oxford step counter has a much smalled set of filter taps and implements two filter ranges. See code.

https://github.com/Oxford-step-counter/C-Step-Counter/blob/master/src/filterStage.c

This has:

static magnitude_t filter_taps[FILTER_TAP_NUM] = {
    -2696,
    -3734,
    11354,
    17457,
    11354,
    -3734,
    -2696};

I figured I could scale the vales down so divided by 256 and tried this as a filter as well.

// a scalled down oxford LPF
var filter_taps = new Int8Array([-11, -15, 44, 68, 44, -15, -11]);

My latest javascript code is at:
https://github.com/hughbarney/bangle_bits/blob/master/step_counters/steps_310.js

Next step for me is to retest another version (3.11) of 3.10 with the same filter taps in the 2.10.9 Bangle Firmware.

The key problem is when driving. Even Fitbit had a lot of trouble with driving counting steps for a while. There is a support thread on the Fitbit site with users complaining about counting 1000s of steps which driving. I think it depends on the device and formware as I suspect different devices had different teams write the firmware. My original Charge HR seemed to do ok when driving. AmazFit have the driving problem cracked - so it is doable.

I'm happy to try the 4/8 hour recordings but think it would be better just to record actual activities like WALKING, WORKING at a DESK, SLEEPING. My worry would be if you just adjust thresholds to fit an end of day value that you will loose accuracy when counting walking steps.

The key problem to be solved I think is screening out counting steps while driving. I did some recordings of my own today, just printed the raw/filtered output to a file in the IDE when I was driving for a few minutes. I could clearly see the impact of going over speed bumps and little bumps in the road. I am wondreing if it will be possible to get an RMS value for driving vs walking etc.

@valvin - new cutting edge firmware out. V2.10.9 . New Pedometer. Worth testing.

4 steps in ~8 seconds. I would argue this is too long a period for a single step. It would allow steps at the rate of 2 seconds for a single step. Which in reality is not walking. Its slow motion.

1 step per second is about as low as we want to go.
So I would suggest 4 steps in 4 seconds or 5 steps in 5 seconds would be better.
This might improve the driving inaccuracy, and reject non stepping activity.

SLEEP test on v2.10.9 firmware - 149 steps.

This is a really good result. v2.09.90 was about 1200 steps.
AmizFit would be around 20.

100 steps is 1% of a 10,000 step goal so anything over 100 steps needs to be looked at.

The accuracy goal for me is 97% accuracy - thats 300 steps off in a 10K step day.

For a step counter to be considered good it needs to be 98% of a calibrated mechanical step counter. That was quoted in one of the papers I looked at.

Doing an overall test for today, which will be housework, going to the tip, walking the dog, buzzing about etc. Will update at the end of the day.

3Hrs GENERAL Activities

AmizFit GTS2 .... 1984 steps Bangle (2.10.9)....... 6323 steps, massively over.

This was 3hrs of washing up, clearing the garage, driving to a chartity shop, unloading, driving to the local tip, unloading, driving home.

I am happy to record a 4 hour segment of mywork day with a bit of walking around the house etc.

Same again but for the new low pass filter.

A general observation is that I can definitely observe fast movements of the hand and arm are not resulting in any amplication or much of a filtered output. This is loads more effective than the previous filter.

Looks like the filter is much smaller AND massively improved - no ringing that I can see.
I copied the filter taps from the code - so I could investigate the filter in javascript.
Hopefully not mis-copied.

// (3) Low Pass filter 
var filter_taps = new Int8Array([5, 6, -4, -17, -15, 4, 11, -10, -27, 8, 80, 118, 80, 8, -27, -10, 11, 4, -15, -17, -4, 6, 5]);

Here is the ringing on the V2.09.90 filter in response to a single arm sideways open and close.

Early results on 2v10.9 Pedometer

.................................... Amiz GTS2 ............ Bangle
DRIVING (15mins) .......................9...................549 Bad
WALKING (1.7mile)................3617................. 3570 Expected good accuracy ~98.7%
DRIVING (15mins) .....................19....................653 Bad

I will do a SLEEP test tonight.

I'll flash the 2v10.9 firmware. I'll do 4 basic tests.

WALKING - 1m, 2m walks - looking for around 2000 steps counted per mile. I suspect this will work. Counting steps when actually walking is a lot easier than not counting them when doing othre things. The previous version of the firmware with the old filter was quite good at this.

SLEEP - how many steps recorded while sleeping - needs to be less than 100.
DRIVING - needs to be low as possible.
WORKING - needs to be minimal while sat at a Desk for 3-4 hours typing.

That will give an initial set of results that can be compared with the other results I published to this thread. We can then look at doing the recordings. Will there be enough space on the bangle to record 8 hours of accelerometer data ?

I dont think there is anything new to test yet. But it would be good to gear up for testing something new as real world tests take up a lot of time and require focus to record the results and repeat the same test again etc.

The steps_mc35.js on page 3 of this thread is my attempt to improve on the current firmware, on the basis that experimenting in Javascript is a lot easier than compiling the C and flashing a new version of firmware onto your watch. The key issue with the current step counting algorithm is that it counts steps when you are not walking eg when sat at a desk typing, when driving, when sleeping. Its no good waking up with 1200 steps on your step counter when all you have been doing is sleeping in bed for 8 hours. This is due to a couple of problems. 1) the low pass filters keeps oscillating for 5-7 'rings' after the movement has stopped 2) the firmware only enforces 6 steps in 6 seconds but does not enforce the period between each and every step. So example sssss_s (where s is a step and _ is a longish gap) is accepted as long as it took 6 seconds. What is needs is that the gap between each step is checked against the timestamp of the last so that it falls within 0.3s to 1s.

steps_mc35.js adds a gating meachanism to shut output of the filter off if the movement has stopped and secondly adds a state machine to check the gap berween each step. You would be more than welcome to test that so that you get a feel for how I was experimenting. Just load the file onto the watch through the IDE and execute it through the FileManager.

When testing I normally wear the bangle and another smartwatch like a FitBit or AmizFit Bip on the same wrist. I record step counts at start and end of the test period and the type of activity and compare results. As well as walking 1,2 or 3 miles I check counts when sat at a desk working for a few hours, when driving, when sleeping.

The conclusion at the moment is that we need to try a different low pass filter that has less ring. If we can get close to 1 ring when the movement stop that can be compensated for in the state machine.

3Hz was quoted a lot in the papers I read. I constructed the table below when thinking about this. 2.5 steps / s is fast marching pace where its easier to run that walk. 3Hz would allow for some slow running to have a chance of working. Suggest trying a simple cut off at 3Hz.

The 'windowing' concept in the oxford step counter is effectievely the state machine approach which ensures step events arrive within the expected time range for every step.

@Gordon - glad you were able to see the double redraw and I was not completely off the mark.

Anymore thoughts on this @Gordon. I have tried a couple of other iterations but the payoff is between accuracy and not recording steps when not walking. The only major thing I could suggest would be a simpler filter that has much less ring. After that you would have to go to more complex approaches like the muti-threaded oxford counter - but that will use up more precious RAM etc. I think I have gone as far as I can with this.

If you are getting E-CALIB it means your compass has not been calibrated. I would not expect this to cause any memory issues though. I use ActivePedom as well. I dont use GadgetBridge. When developing Kitchen Combo my experience was that when memory gets to around 90% you start to see LOW_MEM issues. There might be ways that the Kitchen combo code could be optimised better, I'm open to any quick wins that anyone can see. The key issue was that in order to share a common object across watch faces I had to define the object in main app scope and not in the instance of the watch face.

@Stunts - with the setup you show I would expect apps on lines 9,10,12,13,14 and maybe 17 would be active in memory. The rest will be in storage if not being used.

Hi there. HughB here. So @Gordon is right I am not using GadgetBridge. In Kitchen Combo I am pushing the limits of memory as far as they will go to squeeze in the functionality that I wanted to put together. My setup runs at about 82% of RAM usage most of the time. I did run into LOW_MEMORY issues a lot when I was using buffered writing to the display. I since rewrote the code and accepted screens that have a small amount of flicker. The thing with multiclock , it is only doing clock type functions which are not very complex, fairly simple logic. In Kicthen combo the GPS face shares an object with the WayPointer face so that there is a seamless switch between the GPS data, ie no waiting for a fix etc. Each face can call gpsObject.getFix() and do what it needs with it. Recent changes in the way the firmware works might make that redundant now. But basically there is no central control of the GPS functionality. You can't poll the GPS using something like Bange.getGPSfix(), you have to setup a listenner for every app etc. One of the ways of reducing the footprint of Kitchen combo would be to delete the watch faces you dont want to use, that will give a small amount of RAM back but most of the memory is taken up by the common code in kitchen.app.js. I have also produced single App versions of the watch faces in Kitchen Combo, look for Apps of the same name that are quoted in the README file.