-
The LPF we used in the existing algorithm is actually quite good. A previous filter suffered from lots of ringing. So one step would generate 4-5 echo steps.
The existing algorithm uses the Magnitude of the accelerometer signal. This SQRT(x*x + y*y + z*z). Many designs use the Magnitude method as it means you dont have to worry about the orientation of the device - eg held flat.
One article I read showed a process of dealing with the x,y,z signals seperately, then passing them through an LPF and them making a decision what was going on. For example when driving one of the directions (y) will show acceleration and deceleration due to the car moving forward, another z will be impacted by bumps in the road at random points, x will be impacted when moving the steering wheel. I am not sure how the accelerometer chip knows its orientation (unless it has built in gyros). So for example what if you wore the watch strap around you knuckles. Does that mean that x,y,z swap round.
BTW - what device are you using to experiment with algorithms - is it a mobile phone ? Be aware that sampleing frequency of the Bangle accelerometer is about 12.5Hz. I read a paper that suggest you should use 100hz and that accuracy at 20Hz was poor. I am told by Gordon that 100Hz would plave too much load on the Bangle though.
-
The existing algorithm uses the Magnitude of the accelerometer signal. This SQRT(x*x + y*y + z*z). Many designs use the Magnitude method as it means you dont have to worry about the orientation of the device - eg held flat.
My current best versions use only the component which has changed most since the previous sample.
I am not sure how the accelerometer chip knows its orientation
I don't believe the accelerometer does know its orientation, and the watch will be getting turned this way and that while walking and especially running. Other algorithms use a separate gyroscope to determine orientation and then use just the Z component, which best corresponds to a running motion. I thought that the Bangle had a gyroscope but looking through the reference I can't find it, so I may have misremembered (possibly because I've also been reading about the Puck?).
Edit: It says here it has "device orientation algorithms" https://www.kionix.com/product/KX022-1020
BTW - what device are you using to experiment with algorithms
Actually, I'm just using the data in the two test harness git repos. I didn't check all of them, but I assume they're all as 12.5 Hz. I haven't noticed any papers using as high as 100 Hz, but I did see quite a few use up to 50 Hz. I could use a phone for testing I suppose, but from what I've read the difference between a phone in a pocket and a wrist-based device are large.
-
I'm just using the data in the two test harness git repos.
Great. If you are experimenting in C and using the test harness then you are up and running.
I recorded a lot of those samples on a Bangle 2 using the accellog app. Most of the ones I recorded are called hughb-something.
Its good to have fresh minds on this.
-
Everything I've done can be implemented in C, though it has been a good decade since I wrote any.
If you have written lots of C in the past (before modern tools etc), then you are probably steeped in it to the point that it will all come back very quickly and you will know how to double check a ton of stuff as you write it otherwise you spent the rest of night looking at a core dump.
I found the process of translating my javascript code back into changes to the C part fairly simple. One of the nice things about javascripts is that you can write as if it were C, ie the basic block structure. You will see the state machine I wrote was almost a cut and past job for Gordon to slot it into the C code.
My only worry would be if you write another test harness in Python - just because you need to prove the test harness produces good results that match the real world. Both of the test harnesses work reasonably well.
-
My only worry would be if you write another test harness in Python - just because you need to prove the test harness produces good results that match the real world. Both of the test harnesses work reasonably well.
That is fair enough. My main reason for using Python is that I can most quickly prototype in Python and determine what is a route worth going down. The true measure of success will be the official test harness and, ultimately, real world tests.
Oh, forgot to say, thanks @Gordon - I can compile the harness now and it runs fine too.
-
@HughB Although I may come back to this at some point, so far your current approach keeps winning out. It is easy to beat it for one or two of the sample files, but it is always at the great expense of another. My best experiments have all come back to a variation on the same theme as your state machine, and none have beat it.
-
After @Gordon changed the threshold for the step counting I'm spending today testing this.
Using the same setup as described earlier, with both an old Fitbit Charge 2 and a Bangle.js 2 on the same arm, and after the first few hours it's looking pretty good. Have had the pedometers working for 4 hours, while I've mainly been going about the house and the majority of the time I've been working at the computer . Current step count is 834 vs 839...
I'm gonna do some more housework during the day and maybe take a short walk in the afternoon, so it's going to be interesting to see how it develops.
-
Update... The time is about 22:00 (CET) and I've taken the watches off.
On the 14 threshold the result was, after wearing both watches from 07:00 to 22:00 (mixed activities):
Fitbit - 8765
Bangle - 10188Today with the 18 threshold, after wearing both watches from 07:00 to 22:00 (mixed activities, but not as active as last time), the result was:
Fitbit - 6404
Bangle - 5830Would have liked to see about the same number of steps, but there's definitely a change.
Threshold 14 meant that the Bangle would count 16% more steps than the Fitbit, while threshold 18 meant the Bangle counted 9% fewer steps than the Fitbit. While monitoring the count throughout the day it seemed to me that the count while being still and doing housework was more accurate this time around. The big discrepancy didn't really start until I went for a short walk in the afternoon and then it stayed at about 400-600 steps difference the rest of the day.
The above observations seem to correlate somewhat with the reasoning made by @Gordon here: http://forum.espruino.com/comments/16360753/. Walking became more inaccurate while other activities became more accurate.
More testing needed though...
-
The question is, what is better. Obviously in the case that led to the threshold change, the phantom steps were unacceptable, but missing real steps for real walks is also not great.
Several of the articles I read accept that "steps" is really a proxy for physical activity, and in my view housework counts somewhat in that regard too. If I am using the step counter because I'm on a hike or a run, I want accuracy for actual steps. If I'm using it as a monitor of my general lifestyle, I probably want other exercise-like behaviour included, or at least don't mind about it.
-
Appreciate this is just a first round of testing, a just a rought feedback and more is needed.
Mixed activities are not great, they dont make good comparisons. We have to be quite rigid in the testing regime and a range of activties measured seperately AND the activity set needs to be as close a possible the same as the last test.
I came up with this list of tests:
1) Walking a 1 mile circuit
2) Sleeping overnight.
3) Driving 15 minutes or 30 minutes
4) Sitting at a Desk for 4 hours typing
5) General housework.Even if you walk exactly the same walk, same route, same distance 10 times it will comes out different each time with the same device because your path will always be slighly different, unless you had painted footprints in the path and walked exactly the same footprints etc.
You have to do the same tests multiple times and take the average, one result is not enough.Whats is already known:
on Threshold 14 Bangle will over count on none stepping acctivties like driving, housework but be within 98% of a control device like a fitbit or Amazfit Bip.
On Threshold 18 - I would expect the Bangle to under count by maybe 2000 steps over a 10K steps walk. I did the tests and have the result somewhere.
When I set the threhold at 14 it was because over 10K steps of actual walking I got the most accurate results. I dont want to go on a 10 mile walk and the step counter tell me I have only walked 14000 steps instead of the 20,0000+ it should be saying- as its a pretty worthless step counter in that situation.
On the days I am not walking I am less bothered about accuracy. If I know I have been in the house all day apart from taking the dog round the block, done a bit of housework - I dont really care if the counter reads 1000 steps or 1500 steps - I know I simply did not get enough exercise that day and I'm not going to fool myself just because a step counter says I did 200-300 steps more than I actually did.
-
Appreciate this is just a first round of testing, a just a rought feedback and more is needed.
Yes, hence "more testing needed though". So far I've not made any attempts at any kind of controlled tests... Currently don't have much time for anything else than the mixed activities rough testing. Will hopefully be able to provide more controlled results in a near future.
-
Mixed activities are not great, they dont make good comparisons. We have to be quite rigid in the testing regime and a range of activties measured seperately AND the activity set needs to be as close a possible the same as the last test.
Actually, I think if we can get the timestamps of when the activity changed this could be very useful as an additional comparison.
1) Walking a 1 mile circuit
2) Sleeping overnight.
3) Driving 15 minutes or 30 minutes
4) Sitting at a Desk for 4 hours typing
5) General housework.Running would be another good one. I think you said somewhere in the thread that runners will be more interested in other things, but step detection is important for some of the stats runners are interested in. If we can get the step count right, we can even put in a rougher version of distance if GPS is turned off.
I also liked (or hated) your train journey sample @HughB - I take the train far more often than I drive. I suspect that sample would look very, very different with a higher sampling rate. I'd be interested in more train journey samples if anyone happens to be commuting that way at the moment.
-
I thought I would explain how the current step counter works, where it is weak and where it could be improved.
I refer to the diagram below to the various stages.
TH: - the raw signal is taken scaled and a gating signal derived from the input.
I will describe why the gating process is needed shortly.
This has a lot of weaknesses and assumptions but I was unable to come up with
some simple code to do what was needed. Others may be able to help in this area.F : The raw signal is passed onto the LOW PASS FILTER with a cut off frequency of 3hz if I remember. This is to take out high frequencies that will not be walking. Steps take beween 300-1300ms. I did a spreadsheet of ranges that is published in this thread. However the sampling frequency is only 12.5Hz and this in itself results in a filter that could be improved and is unable to
detect a signal greater than 6.25hz (Nyquest Sampling rule). The current filter is a version of the Oxford step counter filter.The filter has an amplification effect on any signal. A small amount of noise will come out of the filter amplified with peaks and troughs that could look like steps - this has been observed with 2 people who have watches that phantomly count steps.
P : Peak detection. The undulating signal its checked for a Peak. The primary reason for this is that the input to the filter can not be guaranteed to be a blanced zero average signal. In many cases it is offset in the positive direction. This means that some low points will be above the zero line and will not cross a negative threshold value. Using peak detection avoids this issue. The peak detection is crude BUT it works. It was well tested with recordings and I sometimes walked around with a debug version of the step counter connected to the IDE running on my phone. It maybe does not look elegant or good code but it works in practice. I would not say this is the weakest link.
The primary weakness is the implementation of the Threshold Gating signal - I will cover the reasons for saying after describing the G: and S: phases.
G: The gate. The gate_open boolen value set by the TH code is used to decide do we send the detected PEAK onto the statemachine or send throw the PEAK away. The idea is that we are not just sending amplified noise to the state machine. It is also necessary because I swapped to PEAK detection shortly after switching to the Oxford filter. The previous filter would result in a more balanced sinewave like signal that would cross the zero line. However the previous BAND-PASS filter also head a 'Ringing' issue that mean't when you stopped moving the signal would echo a further 5 or more cycles that would be counted as steps. Having swapped to PEAK detection it was important to ensure we were not just sending a peak in the noise signal onto the state machine.
So returning to the TH Threshold detection.
The idea was to say 'are we generating a strong enough signal to warrant checking if we are walking.
There are various problems.There is not a lot of differentiation between gentle walking and noise.
The changes in acceleration are actually quite small unless you are a STOMPER of a walker like the incredible hulk.The raw signal can be unbalanced. In that you would expect the sine wave to cross the zero line but it is often offset by a positive value.
The code assumes that the raw signal will not be offset and only checks that the raw signal is greater that +TH or less that -TH where TH is the threshold value.
What is really needed is a way to detect the average and to offset the raw signal so that its undulations rise and fall around a zero line. Then what is being fed into the LOW PASS filter will be balanced and just be amplified.
OR there needs to be two thresolds one for the position range and one for the negative range. EG 14-18 positive but -5 for negative.
The gate needs to open and close quickly, we allow 3 samples in a row that are above or below the threshold to set the gate and for this to be reset after 3 samples below the threshold. If we extend the time take to decide if the signal is some movement or noise is too long then we could start miss counting more steps during that period. 3 samples is less than 1 second at a 12.5hz sample frequency.
if (v > RAW_THRESHOLD || v < -1*RAW_THRESHOLD) { if (active_sample_count < N_ACTIVE_SAMPLES) active_sample_count++; if (active_sample_count == N_ACTIVE_SAMPLES) gate_open = true; } else { if (active_sample_count > 0) active_sample_count--; if (active_sample_count == 0) gate_open = false; }One approach to improving this would be have a POS_THRESHOLD and NEG_THRESHOLD and to run tests against the test harness to establish the best pair of values for both.
But we also need a lot more controlled samples from different people and different watches.
@johan_m_o - would be good if you could record some samples of different distinct activies against your fitbit. 1 mile walking. driving etc.
1 Attachment
-
@HughB Thanks for the explanation, it confirms my understanding and adds a couple of details I missed.
Your spreadsheet of step speeds is at http://forum.espruino.com/comments/16122158/ - I have just realised that the range I was looking for is slightly extended (allows for faster running), so when I get back to this I am going to pay more attention to the effect of that. My own running pace, which I didn't think was that fast, falls outside of your highlighted range.
-
Its a bit of a compromise with running and a question of how far you go. If you go less that 300ms for a step then its possble that a rythmic vibration will look like you are stepping.
For running I think we should rely on the RUN app / GPS for accuracy. I am about to checkin a fix for the RUN app that corrects the distance calculation from the GPS.
I think if we improve the more common 'walking' and non walking cases first we can then explore the limits (eg moderate to faster running) and see how it performs.
-
Real world Test results v2.11.30 Step Implementation - INDICATIVE only.
I have only two two tests so these are indicative.
@Gordon I was wrong about the threshold - combine it wit X_STEPS = 8 and THRESHOLD = 17 and you start to drop 1000s of steps. Its in the checkin comments of stepcount.c back in september.Method: I wore 3 watches on my left wrist. AmizFit BIP, Bangle 2 with 2.11.27 and Bangle 2 with 2.11.30. I used the RUN app with GPS and HEART turned off but it makes it much easier to count steps for a test.
AMIZ: 5438 10k (scalled up to 10K = 1.84 factor)
2.11.27 5455 10037 (scalled up by 1.84)
2.11.30 5166 9505 (scalled up by 1.84)I have scalled these to 10k to compare what the step count might be at 10K steps. I have used the scaling factor of the AMIZ - this is just for comparison purposes. Really I would always like to test a 10K step stretch but it is not always possible.
This shows that 2.11.27 is 100.37% of the AMIZ which is a great result.
2.11.30 would be 95% of the AMIZ which is the trade off you get. Target is really 98%HOUSEWORK - I did a test doing 1h44 housework BUT the 2.11.27 watch got poked somehow and rebooted so I lost the result as the steps got reset. This is the hazard you get when you try and wear 3 watches on the same wrist :)
AMIZ 524
2.11.30 482
The 2.11.27 was always higher.
What this shows is that maybe the threshold should be 15, 16 or 17 as 18 is making it too insensitive.More tests to do.
HughB
myownself
johan_m_o
-
Post a reply
About
Better Pedometer - HELP NEEDED!
Posted by
@Gordon
No problem. I might be done with this for the day, but if I get to it before Gordon responds I know from your message that I can probably just check out an older commit.
I've played with an algorithm today that depending on parameters does great on actual steps but not on e.g. your train journey, or does very well on the train journey (not perfect, but not bad) but misses too many real steps. I am hoping a low pass filter will improve matters.