Better Pedometer - HELP NEEDED!

The 97% while walking was really good, but the almost 10 time difference between the Fitbit and the Bangle when sitting still at the computer wasn't as good... Although way too small a result to say much.

I'll try some other situations in the coming days.

I think recording the steps (not watching the counter) and counting can be good. We don't know which way other devices are skewed, so if you can walk normally and count then the data is useful. I find I can count accurately for 1000 steps, and then I get bored and my mind drifts, but for those 1000 steps I am pretty certain of my counting, and I think only for the first few steps I am paying more attention to the steps than I usually would (and possibly walking unnaturally).

Yeah, I just walk normally and count to 10 over and over and use the fingers on my right hand to count the 10s up to 100 and then I use my left hand to keep track of the 100s. That way I can very accurately count much higher than 1000 without much effort.

Hah, my technique is similar but I only use a hand for the hundreds, not the tens.

I find it a lot easier to keep up with steps when only counting to ten...

Its a known issue and there is plenty of test data already to show the over counting when not stepping. Its not going to get any better without a different approach.

Was threshold 15 really the best approach then, and not a slightly higher one?

Thanks for that. We really need measurements against another device like a 'fibit' or 'amazfit bip'.

I have a Mi Band 5, I can make a comparison with that.

The threshold is finely balanced. Its purpose is to say, if we dont have activity above this threshold then dont look for steps. Setting too low you overcount, too high and you undercount. Setting above 15 reduces 10K step accuracy down to 95% or less which for me is too low. For a step counter to be consider good it should be 98% - I have that in a paper somewhere. 95% accuracy looks like 9500 steps counted when the fitbit would count 10,000. To me thats too inaccurate.

Was threshold 15 really the best approach then, and not a slightly higher one?

The new DC filter means we have to gather more data. What the DC filter does is correctly compensate for positive bias in the signal (ie it would look like the average eas not 0 but say 5). Now that we are compensating for bias then we need to gather more data. My gut feel for it (and supported by test harness runs) is that somewhere between 14 and 15 is the right value.

BUT it is really hard to know what is going on out there. Not everyone with a B2 reads or posts to this forum and some may just not be interested in steps at all. We really need some sort of polling system in the forum 'is the step counter good enough ? ' 0 for poor, 10 for excellent.

Its purpose is to say, if we dont have activity above this threshold then dont look for steps. Setting too low you overcount, too high and you undercount

Is there a way to change the threshold from the Bangle? Or a way to temporarily disable the steps counter? E.g. before lunch I swept the garden sidewalk and the Bangle (but Mi Band also) counted over 100 steps but I was "standing" in place.

No. Counting steps when making a rythmic forwards and backwards motion is a standard way to fool all step counters. I can make any step counter count steps by holding the watch in my hand and flicking it backwards and forwards about once per second. The amount of gravitation force when moving , arms swinging and head bobbing up and down will be very similar to a sweeping or mowing motion. This is why for really accurate distance measurements the GPS is used in most smart watches for sporting activities etc.

Counting steps when making a rythmic forwards and backwards motion is a standard way to fool all step counters.

I know it, which is why sometimes I would like to (temporarily) stop the step counter. I'd like press a "pause counter" button, swept the garden sidewalk and then press a "unpause counter" button :-D

Is it technically possible to develop a js app that does this?

Today I wore both a Bangle2 and a Mi Band 5 on the same arm.

Before the treadmill: Bangle2 1683 vs Mi Band 1046 steps.

After 2 Km on the treadmill: 3557 vs 3480 so the Bangle counted 1874 vs 2434 steps.

(I read that small and short activities are ignored by Mi Band by design, e.g. I walk from study to kitchen, pause, and then from kitchen to study: Bangle counted 10+11 steps vs 0 by Mi Band).

I attached the log of the treadmill walk, hope it helps!

1 Attachment

• 20220202-2km-treadmill.csv

We know extraction of steps from acc is difficult and best way to improve is test against a set of known results - as you did with stepcount.c
We also know that ground truth data is needed for that but difficult to come by (counting steps is cumbersome and other devices might not be reliable).

So what about the following idea (new or already discussed?): strap another bangle1/2 or puck at ankle.
That gives us
a) a file from wich step count should be extractable much more reliable since foot has much stronger and clearer pattern and much less distractions (other movements done with arm)
There could be a special ankle algorithm, but using the same one should work also I think especially if bangle2 for both would be used.
b) real ground truth, in that we know both recordings do have exactly the same step count (even if we do not know the count, we do have an error measure to minimize from the difference)

So far I have recorded two such pairs. Both times bangle2 (b2) at wrist and bangle1 (b1) over ankle. Additionaly I had a Mi3 band for comparison.
I have not done much analysis yet due to time limitations, but thought I post it here anyway in case anybody is interested to follow the idea or look into the files.

4 Attachments

• 210202a-Mi-walk1-b2-accellog2-Mi3=1609.csv
• 210202b-Mi-walk2-b2-accellog5-Mi3=5593.csv
• 210202a-Mi-walk1-b1-accellog2-Mi3=1609.csv
• 210202b-Mi-walk2-b1-accellog5-Mi3=5593.csv

By the way: doing that experiment I found

• acc recorder menu is not working well on bangle1. On bangle2 one typically navigates the menu with touch and then stops with button. On bangle1 one typically navigates the menu with button and very often this start is immediatly recognised as stop.
  
• since download takes a while I didn't wait and later on mixed up the files first. So in addition to go from file number to date/time as filename as already discussed in some thread (for recorder app though, but thats equally valid for all recorders) one can think of inlcuding the 4hex name of the device also in the filename. That would make very clear what recording that is also after download.
  

This is great. Do you have the step counts recorded by the Bangle as well? This helps sanity check differences with the harness and actual steps counted on the Bangle.

Have you uploaded 4 separate logs, looks like 2 logs.

Have you uploaded 4 separate logs, looks like 2 logs.

yes, should be 4 different logs: 2 walks with two devices each.
I see 4 attachments here, what do you see?

Do you have the step counts recorded by the Bangle as well?

No unfortunatly not. Will do this next time.
But the numbers in harness look reasonalby:
X_STEPS = 6, RAW_THRESHOLD = 14
File, Expected, Simulated, Diff, %, (Original)
210202a-Mi-walk1-b1-accellog2-Mi3=1609.csv, 1609, 1664, 55, 103.42 %, (1113)
210202a-Mi-walk1-b2-accellog2-Mi3=1609.csv, 1609, 1473, -136, 91.55 %, (1590)
210202b-Mi-walk2-b1-accellog5-Mi3=5593.csv, 5593, 5854, 261, 104.67 %, (2892)
210202b-Mi-walk2-b2-accellog5-Mi3=5593.csv, 5593, 5450, -143, 97.44 %, (5665)
TOTAL DIFFERENCE 331

X_STEPS = 6, RAW_THRESHOLD = 16
File, Expected, Simulated, Diff, %, (Original)
210202a-Mi-walk1-b1-accellog2-Mi3=1609.csv, 1609, 1645, 36, 102.24 %, (1113)
210202a-Mi-walk1-b2-accellog2-Mi3=1609.csv, 1609, 1423, -186, 88.44 %, (1590)
210202b-Mi-walk2-b1-accellog5-Mi3=5593.csv, 5593, 5835, 242, 104.33 %, (2892)
210202b-Mi-walk2-b2-accellog5-Mi3=5593.csv, 5593, 5383, -210, 96.25 %, (5665)
TOTAL DIFFERENCE 372

Just a quick manual test. Havent understood the auto search yet.

You see that bangle1 (ankle) is closer to Mi3 than bangle2 (wrist) - but its not as clear and stable as I would have expected

(counting steps is cumbersome and other devices might not be reliable).

I give any device I am going to compare against a test on a circuit that I know is 1.03m.
If it comes out around 2000 steps I know its accurate enough to use as a source of ground truth.
I usually do 2 or 3 tests on the same cicuit to establish reliability. I trust FitBit HR and Amizfit Bip to be accurate enough for what we need. I imagine other smart watches out there will be reasonably good as well, but you have to calibrate your measure first before using it as a yardstick.

Might be worth doing a 1 mile walk with the Run app v0.05 and checking how many steps are counted and distance is reported by the Mi5 band to see how accurate it is.

The idea of using another bangel running 'in development' software does not sound great as both measurements will be subject to version control and known issues. I think we either have to manually count the steps or use a known quantity (another sports watch that we have checked).

b) real ground truth,

Hmm, I think the only truth would be that it was a bangle nearer the ground. Thats does not mean it will reliably count the steps any better. I would kind of expect the results of the B1 at ankle level to be better as the accelerometer swing will be much greater. However since the watch is going to be worn on the wrist tests have to be done with the measurement watch on the same wrist and accelerometer logs need to be done on the wrist as well otherwise they introduce another variable into the tests.

Is it technically possible to develop a js app that does this?

Yes , you could build a clock where pressing BTN1 would start and stop counting steps if you wanted to. The Run App works like this already. You could copy that as a starting point. Or you could count steps using the Bangle.on('step',x) event BUT only count steps when you had not put your app into pause mode.

(I read that small and short activities are ignored by Mi Band by design, e.g. I walk from study to
kitchen, pause, and then from kitchen to study: Bangle counted 10+11 steps vs 0 by Mi Band).

Each watch does this slightly differently. Fitbit HR requires 5 steps before it will count, AmizFit Bip requires 8, Bangle requires 6. Mi may require a lot more, you can establish how many through multiple experiments. If anyone is watching when you do this you look quite odd. I once did this in a farmers field :)

How many actual steps is it from your Study to Kitchen ?

b) real ground truth,

Hmm, I think the only truth would be that it was a bangle nearer the ground.

No, that is the idea of a) with the thesis that ankle is easier than wrist -which of cours is to bee seen and needs to be testet. I hope to see a much more robust outcome when varying parameters.

b) contains a bit of real ground truth! We have two acc input data streams a,b for which we know for sure that steps(a)-steps(b) =! 0. So even if we do not know the number of steps, we know that the bigger that difference is, the worse the algorithm "steps()" performs. No approximation here.
It is just a necessary, not a sufficient condition thought, so can not be used alone. E.g steps(x)=0 would be considered perfect but is completely useless obviously.
Still I think its a powerfull error measure, available with limited effort. If degenerate cases are excluded (eg. by demanding step count within +-20% of Mi/Fitbit or so) I would expect that to be a powerfull objective measure for guiding search.

Good to know, thank you! I'm putting this on my to-do list ;-)

How many actual steps is it from your Study to Kitchen ?

I counted 12 steps so the Bangle is quite correct.

If anyone is watching when you do this you look quite odd. I once did this in a farmers field :)

Haha! :-D

No, that is the idea of a) with the thesis that ankle is easier than wrist

Sure -if you want to walk around with a bangle watch strapped to your ankle. The watch is for wearing on the wrist not the ankle. Most people are not going to strap it to their leg etc. The accelerometer traces and thresholds will be completely different for a step counter designed work off a foot to one worn on a wrist. Its completely different approach. We are trying to optimise for the step counter to work when the watch is being worn - just like any other fitness band. Adding accelerometer logs recorded at ground level is introducing an uncalibrated meaure thats going to confuse matters.

(eg. by demanding step count within +-20% of Mi/Fitbit or so)

We are looking for 95-98% of a good step counter. 20% is far too inaccurate.

The fundemental problem isn't counting steps accurately when walking - its how NOT to count steps when NOT walking but still count them accurately when walking. This is where the bulk of our thinking needs to go.

We are looking for 95-98% of a good step counter. 20% is far too inaccurate

Yes of course. That interval was meant to exclude degenerate cases. Just means something like "we can assume Fitbit/Mi not worse than 20%".

This is where the bulk of our thinking needs to go.

Yes, I am very aware of that.
I also did record some of those already (eg see attached one that I do think will be pretty challenging from a cahier. Here we do have regular patterns that might look very much like steps when the hand moves goods over the scanner)
But thats exactly the point: to seperate those phases with and without steps in real life situations. That is almost impossible if alternating relative rapidly (as opposed to car driving /sleeping). With the development tool of ankle measure, those could be determined I hope (eg. for cashier I wouldn't expect any correlation when moving goods, only when walking)
What I am up to is to generate a more relieable step signal for training purposes (each step, not only sum number).

I seem not to be able to express myself clearly - I am very sorry for that.
Lets continue that discussion after I can present some code here. That may clearify (or invalidate) my idea.
And please don't get me wrong: I am very impressed what you achieved already!
For my personal uscase its already more than good enough. I play with that not out of need but scientific curiosity. (I did work with acc in the past on quite challenging problems in my job).
On the other hand: when haveing more time I might concentrate on heart rate, that is more in need right now I think. Have no prior experince with PPG signals though.

1 Attachment

• 220203b_cashier_accellog_gt=0.csv