Better Pedometer - HELP NEEDED!

Regarding ringing, yes we switched to a low pass filter. Its the same one you use in the c code with the coefficients scaled down.

Hi @user107850,

Thanks for your work on this! I did pull in and try your changes (and they were in 'cutting edge' Bangle.js firmwares for a while), but actually I think the most recent changes came after I'd moved to a new algorithm myself, so I didn't end up trying those I'm afraid.

I felt like the problem initially was that the original, super-basic step counter actually did a half decent job of step counting for normal walking (it was within 10%). The issue was for everything else.

The Oxford step counter was undoubtedly better for walking, but it actually (at least initially) seemed even worse for non-walking activities, and I had the same issue as @HughB understanding what was really going on so I couldn't easily make useful changes (although looking back, your 'pipeline of stages' explanation suddenly makes things look a lot clearer!).

Given the reasonably large memory overhead and complexity, I figured I was better off starting from scratch with something I understood as it seemed like the overall algorithm should have been really simple.

I did however shamelessly steal a few of the Oxford Step counter ideas though - developing on desktop first with a data-driven approach (https://github.com/gfwilliams/step-count) and using that great filter generating website.

As @HughB mentioned, initially I thought that a bandpass filter would help me filter out low frequency elements of the accelerometer data as well as high, but while on the whole it worked well, a big spike of acceleration would create ringing that looked like a bunch of steps. We moved to a low pass filter (then finally to the exact Oxford step counter one!), and @HughB added peak detection code (which then didn't require high pass filter) and a state machine to count steps.

I feel like it's actually working pretty well now (although I still wonder whether filtering on X/Y/Z separately and then getting magnitude would help with false steps).

However, there is always room for improvement, and getting students at Malmö involved would be amazing if there's interest. I'm happy to do a competition, but last time that was mentioned, there wasn't too much interest :(

Apart from step counting there's heart rate as you say, which is potentially very interesting. It seems the manufacturer supplied algorithm (which is a binary blob that we don't really have access to the code for) uses accelerometer data (probably to discount readings) but is able to work off very low res data (like maybe 4 samples/sec) which means vastly improved energy usage since we currently run at 50 sps I believe.

There's also a whole area of 'exposure adjustment' where the LED power is scaled in order to get the best readings. We're actually using the manufacturer-provided version which we reverse engineered at the moment on Bangle.js 2, but it just takes forever.

There's also sleep tracking which would be really cool (and maybe other options around that). I imagine that's something that could definitely be handled in a data-driven way if we had a few captures of accelerometer data over the course of the night.

Hello @Gordon , thanks for the reply.

I understand that it makes sense to try with a more ad-hoc peak detector, especially if you know that the hardware is always the same. Our algorithm can be optimised a lot, starting from removing the buffers, but we didn't have enough time for that. As for the heart rate, I would probably trust the proprietary blob better than anything else. With time, if we manage to get a reliable algorithm, we would ideally get rid of it.

Back to the competition idea: I have created a repository where we can test algorithms, see here.

I have already populated it with the data I have collected myself plus some of the data collected by @HughB. I have not included those files where the two measurements of steps are too different, because I don't trust either, and for those that I have included, I have computed the average between the two given step counts.

@HughB would you be able to send me a pull request with your algorithm? Please check the structure inside the dummy algo. If that doesn't fit with yours, I am happy to change it, just le me know (or submit an issue on Github).

With time, we can add more algorithms, or simply improve ours. It's going to be fun (at least for me😀).

I would like to keep this repository also for testing the algorithms for heart rate or sleep detection when we get to it, and I will probably share it with students if I manage to involve some.

I feel like it's actually working pretty well now

Its pretty good, but not perfect. I'd like to get the housework log down a bit.

When I have a day at home sat at a desk I expect to see about 600-800 steps for the day, and often see 1500-2000. But where as before it was really obvious where the inaccuracies were I think it might be a while before users start raising issues. Once the new daily logging is in place then I thnk it will be more obvious for people to spot the over counting.

filtering on X/Y/Z separately and then getting magnitude would help with false steps.

We'd have to do the experiments. Maybe an experiment for the future using the javascript version.
I think looking at plots of the x,y,z when of the driving accelerometer logs would be intereseting to do.

I will have a look at how easy it is to attach the current step counting algorithm into your framework.

Note its already in the current firmware at:
https://github.com/espruino/Espruino/blob/master/libs/misc/stepcount.c

I'm concerned that the test harness does not have enough non-stepping data and long enough logs.

We must have good tests cases for when not walking - otherwise we are just prone to the same mistakes and errors as before. We can easily be fooled in thinking a code tweak has improved things when it fact it has not. A bad selection of test data will waste our time. Any step counter must be measured against stepping and non-stepping scenarios in equal measures.

The Accelerometer logs below are all ZERO steps and should be part of the controlled test data in my view. I was not walking when they were logged, I was sitting or driving or typing at a Desk. These and similar logs must form at least 50% of the test data in any test harness.

HughB-drive-36min-0.csv
HughB-drive-29min-0.csv
HughB-drive-a3-b136.csv
HughB-work-66.csv

It would be useful to keep the same file names so that provenance of the logs can be tracked. Or we should keep a proper register of every log detailed where it came from and and any other measurements that were done against it at the time.

I am also concerned that the current 0 step samples (0.csv 1.csv, 2,csv) are only 30 seconds long. This is not long enough to flush out problems. For example the drive-a3-b136 log was a 15 minute log and the AmizFit Bip registered 3 steps where as the Bangle registered 136. I suspect it would be possible to select many different 30 second segments from that log and have a target step counter register 0 steps - thus fooling ourselves.

I'm glad you have included HughB-walk-a10021-b10248.csv as 10134.csv as its a really good log. It was recorded recently. I wore both watches on my left wrist whilst walking across the yorkshire dales :)

@user107850 - I ran into build issues. I had to create the build directory, but there is also a clash with time_t.

I am on a Linux Debian environment

$ uname -a
Linux penguin 5.4.131-16049-gc688e114d34e #1 SMP PREEMPT Sun Sep 19 21:16:20 PDT 2021 x86_64 GNU/Linux

$ 
$ mkdir build
$ pwd
/home/hughbarney/src/banglejs-algos-tester
$ 
$ ll
total 12
drwxr-xr-x 1 hughbarney hughbarney    0 Oct  4 21:13 build
-rw-r--r-- 1 hughbarney hughbarney  608 Oct  4 21:04 CMakeLists.txt
drwxr-xr-x 1 hughbarney hughbarney   10 Oct  4 21:04 data
-rw-r--r-- 1 hughbarney hughbarney 1068 Oct  4 21:04 LICENSE
-rw-r--r-- 1 hughbarney hughbarney 1703 Oct  4 21:04 README.md
drwxr-xr-x 1 hughbarney hughbarney   50 Oct  4 21:04 src
$ 
$ 
$ cd build
$ cmake ../
-- The C compiler identification is GNU 8.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/hughbarney/src/banglejs-algos-tester/build
$ 
$ cmake --build .
Scanning dependencies of target banglejs-algos-tester
[  8%] Building C object CMakeFiles/banglejs-algos-tester.dir/src/main.c.o
In file included from /home/hughbarney/src/banglejs-algos-tester/src/stepcounters/stepcounters.h:4,
                 from /home/hughbarney/src/banglejs-algos-tester/src/main.c:6:
/home/hughbarney/src/banglejs-algos-tester/src/stepcounters/../types.h:8:18: error: conflicting types for ‘time_t’
 typedef uint32_t time_t;
                  ^~~~~~
In file included from /usr/include/x86_64-linux-gnu/sys/types.h:129,
                 from /usr/include/stdlib.h:394,
                 from /home/hughbarney/src/banglejs-algos-tester/src/main.c:4:
/usr/include/x86_64-linux-gnu/bits/types/time_t.h:7:18: note: previous declaration of ‘time_t’ was here
 typedef __time_t time_t;
                  ^~~~~~
make[2]: *** [CMakeFiles/banglejs-algos-tester.dir/build.make:63: CMakeFiles/banglejs-algos-tester.dir/src/main.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:73: CMakeFiles/banglejs-algos-tester.dir/all] Error 2
make: *** [Makefile:106: all] Error 2
$ 
$ pwd
/home/hughbarney/src/banglejs-algos-tester/build

@user107850 - I managed to get it to compile with this hack:

I dont think you need to worry about rolling over time_t in 1 year of milli seconds for a test harness where the logs are a few hours at most.

$ git diff
diff --git a/src/types.h b/src/types.h
index 6e74ab1..39a972e 100644
--- a/src/types.h
+++ b/src/types.h
@@ -1,11 +1,12 @@
 [#include](https://forum.espruino.com/search/?q=%23include) <inttypes.h>
+#include <time.h>
 
 // type for the accelerometer samples, this depends on your hardware
 typedef int16_t accel_t;
 
 // type used for time: warning the algorithm is not robust to roll-over of this variable
 // example: a year worth of ms needs 35 bits, 32 bits allows you to store about 50 days of ms
-typedef uint32_t time_t;
+//typedef uint32_t time_t;
 
 // steps count type
 // the size of this depends on what is the maximum number of steps you are willing to show and store
$ 

But when I ran it I didn't match the results you showed in the README file.

$ ./banglejs-algos-tester |grep oxf
steps-controlled, oxford, 0.csv, 0, 0
steps-controlled, oxford, 0_1.csv, 0, 0
steps-controlled, oxford, 0_2.csv, 0, 0
steps-controlled, oxford, 0_3.csv, 0, 0
steps-controlled, oxford, 100.csv, 77, 100
steps-controlled, oxford, 100_1.csv, 79, 100
steps-controlled, oxford, 100_2.csv, 80, 100
steps-controlled, oxford, 100_3.csv, 170, 100
steps-controlled, oxford, 100_4.csv, 268, 100
steps-controlled, oxford, 100_5.csv, 270, 100
steps-controlled, oxford, 100_6.csv, 270, 100
steps-controlled, oxford, 100_7.csv, 270, 100
steps-controlled, oxford, 150.csv, 326, 150
steps-controlled, oxford, 150_1.csv, 345, 150
steps-controlled, oxford, 150_2.csv, 564, 150
steps-controlled, oxford, 150_3.csv, 727, 150
steps-controlled, oxford, 150_4.csv, 914, 150
steps-uncontrolled, oxford, 0_HughB-static.csv, 0, 0
steps-uncontrolled, oxford, 10134.csv, 17314, 10134
steps-uncontrolled, oxford, 3058.csv, 20694, 3058
steps-uncontrolled, oxford, 0_HughB-work-0.csv, 22287, 0
steps-uncontrolled, oxford, 0_HughB-work-66.csv, 24182, 0
steps-uncontrolled, oxford, 0_HughB-drive-29min-0.csv, 26783, 0
steps-uncontrolled, oxford, 0_HughB-drive-36min-0.csv, 29674, 0
steps-uncontrolled, oxford, 0_HughB-drive-a3-b136.csv, 29678, 0
$ 

I noticed that the step count was just incrementing file on file.
So I looked at the reset code and put a reset in after printing the results from a single test.
See diff below.

diff --git a/src/main.c b/src/main.c
index 0dc2d1d..a4838cb 100644
--- a/src/main.c
+++ b/src/main.c
@@ -69,6 +69,7 @@ void becnhmarkFolder(char *prefix, char *dirname)
 
                 fclose(pFile);
             }
+           resetStepCounters(); // reset after every run
         }
     }
     closedir(folder);
@@ -89,4 +90,4 @@ int main()
     resetStepCounters();
 
     return 0;
-}
\ No newline at end of file
+}

Now the results look more realistic and the 3058.csv result for the Oxford of 3301 steps EXACTLY matches your results.

$ ./banglejs-algos-tester |grep oxf
steps-controlled, oxford, 0.csv, 0, 0
steps-controlled, oxford, 0_1.csv, 0, 0
steps-controlled, oxford, 0_2.csv, 0, 0
steps-controlled, oxford, 0_3.csv, 0, 0
steps-controlled, oxford, 100.csv, 113, 100
steps-controlled, oxford, 100_1.csv, 101, 100
steps-controlled, oxford, 100_2.csv, 102, 100
steps-controlled, oxford, 100_3.csv, 91, 100
steps-controlled, oxford, 100_4.csv, 77, 100
steps-controlled, oxford, 100_5.csv, 84, 100
steps-controlled, oxford, 100_6.csv, 80, 100
steps-controlled, oxford, 100_7.csv, 80, 100
steps-controlled, oxford, 150.csv, 138, 150
steps-controlled, oxford, 150_1.csv, 150, 150
steps-controlled, oxford, 150_2.csv, 192, 150
steps-controlled, oxford, 150_3.csv, 131, 150
steps-controlled, oxford, 150_4.csv, 151, 150
steps-uncontrolled, oxford, 0_HughB-static.csv, 0, 0
steps-uncontrolled, oxford, 10134.csv, 17314, 10134
steps-uncontrolled, oxford, 3058.csv, 3301, 3058
steps-uncontrolled, oxford, 0_HughB-work-0.csv, 1347, 0
steps-uncontrolled, oxford, 0_HughB-work-66.csv, 1639, 0
steps-uncontrolled, oxford, 0_HughB-drive-29min-0.csv, 2372, 0
steps-uncontrolled, oxford, 0_HughB-drive-36min-0.csv, 2651, 0
steps-uncontrolled, oxford, 0_HughB-drive-a3-b136.csv, 1093, 0

Can you check the 2 changes I have made to the test framework code are correct.

I will do a pull request for the test files.

Hi,
sorry there were a few bugs, I should have fixed them now, please pull. I have also run some more benchmarks with your new data and tuned the movement detection stage a bit.

It would be useful to keep the same file names so that provenance of the logs can be tracked.

Yes, good idea! The only requirement is that the first characters correspond to what we consider the "ground truth".

The Accelerometer logs below are all ZERO steps and should be part of the controlled test data in my view.

Yes, if you are sure about the fact that you did not do any step.

These and similar logs must form at least 50% of the test data in any test harness.

The more data we have the better. Actually we should also include running and walking in different conditions, like on soft floor, hard floor, with and without shoes etc.

No algorithm is going to be perfect, some may be more precise when walking, others when not. In the end it depends on the use case: runners may prefer better accuracy when running, others may prefer better accuracy when walking etc.

As a general aim I think we should try to have something as good as an average fitness tracker.

Hello, I have added the espruino step counter as well (with minor changes in names). Looks like it has better accuracy than the Oxford one, especially in absence of movement. Well done!

We could use this repository to try to improve the algos with further tweaking. Pull requests are very welcome.

Got the same build error again. Did you resolve the time_t definition clash ?

I recloned the repository and started from scratch. Still get the build error.

Scanning dependencies of target banglejs-algos-tester
[  7%] Building C object CMakeFiles/banglejs-algos-tester.dir/src/main.c.o
In file included from /home/hughbarney/src/banglejs-algos-tester/src/stepcounters/stepcounters.h:4,
                 from /home/hughbarney/src/banglejs-algos-tester/src/main.c:6:
/home/hughbarney/src/banglejs-algos-tester/src/stepcounters/../types.h:11:18: error: conflicting types for ‘time_t’
 typedef uint32_t time_t;
                  ^~~~~~
In file included from /usr/include/x86_64-linux-gnu/sys/types.h:129,
                 from /usr/include/stdlib.h:394,
                 from /home/hughbarney/src/banglejs-algos-tester/src/main.c:4:
/usr/include/x86_64-linux-gnu/bits/types/time_t.h:7:18: note: previous declaration of ‘time_t’ was here
 typedef __time_t time_t;
                  ^~~~~~
make[2]: *** [CMakeFiles/banglejs-algos-tester.dir/build.make:63: CMakeFiles/banglejs-algos-tester.dir/src/main.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:73: CMakeFiles/banglejs-algos-tester.dir/all] Error 2
make: *** [Makefile:106: all] Error 2

The easiest way to fix this is to comment out the time_t definition in types.h and add include at the top of types.h

Looks like the test harnesses are in agreement in that the results we get from the Bangle harness are the exactly the same counts as your harness.

Brilliant to see these comparisons.

Looks like it has better accuracy than the Oxford one, especially in absence of movement. Well >done!

Thank you. Its good to see the confirmation of the approach and also the approach to testing.

$ ./banglejs-algos-tester |grep -v dummy|grep uncontrolled
steps-uncontrolled, oxford, 0_HughB-drive-29min-0.csv, 469, 0
steps-uncontrolled, espruino, 0_HughB-drive-29min-0.csv, 192, 0
steps-uncontrolled, oxford, 0_HughB-drive-36min-0.csv, 512, 0
steps-uncontrolled, espruino, 0_HughB-drive-36min-0.csv, 160, 0
steps-uncontrolled, oxford, 0_HughB-drive-a3-b136.csv, 321, 0
steps-uncontrolled, espruino, 0_HughB-drive-a3-b136.csv, 124, 0
steps-uncontrolled, oxford, 0_HughB-static.csv, 0, 0
steps-uncontrolled, espruino, 0_HughB-static.csv, 0, 0
steps-uncontrolled, oxford, 0_HughB-work-0.csv, 259, 0
steps-uncontrolled, espruino, 0_HughB-work-0.csv, 20, 0
steps-uncontrolled, oxford, 0_HughB-work-66.csv, 414, 0
steps-uncontrolled, espruino, 0_HughB-work-66.csv, 97, 0
steps-uncontrolled, oxford, 10134.csv, 14824, 10134
steps-uncontrolled, espruino, 10134.csv, 10253, 10134
steps-uncontrolled, oxford, 3058.csv, 3135, 3058
steps-uncontrolled, espruino, 3058.csv, 3013, 3058

The more data we have the better.

Totally agree. I was attemoting to collect up to 4 10K+ step logs when I ran into a problem with the logger. I will get back to collecting logs again soon. Need some 1 hour logs for sleep, driving, working, sitting watching TV - all ZERO step logs.

I forget how little portable C is!
OK, I have renamed time_t to sc_time_t so it doesn't conflict with linux time_t. Try pulling.

Need some 1 hour logs for sleep, driving, working, sitting watching TV - all ZERO step logs.

Yes, but let's not forget collecting steps data under different circumstances (slow walk, fast walk, running, etc).

It would be nice if we could involve other users, just to confirm that there are no major differences across devices.

As soon as I have a working watch (I've lost my charging cable for my Bangle.js v1) I can probably start contributing some data.

Thanks @johan_m_o, do you have another device to calculate the step count? We need logs that are a known step count. I use an Amazfit bip or GTs on the same wrist when making recordings. This allows us to check the simulated counted steps against the step count recorded by the other device. We need logs with 1000s of steps, its not really practical to manually count steps.

@HughB I've got an old Fitbit that I can use as a control.

@johan_m_o - thats brilliant. I've found them to be quite accurate for walking and non walking
activities. EG - my Fitbit Charge HR records about 20 steps overnight during a sleep period, non while driving and about 2000 steps per mile which is spot on for my stride length. So I trust them as something to measure against. At this stage a few steps over/under on the control is neither here nore there in terms of testing. We are sill a long way off that kind of accuracy.

The method I use is:

1. note START step count on control device.
   
2. not START step count on Bangle
   
3. start the logging
   
4. start walking - I usually walk the dog for a couple of miles
   
5. stop walking, stop the logger
   
6. when stopped not END counts of bangle / control device
   
   

When you upload the file looks like we have agree'd a filename format like:

1234_Name_Activity_bYYYY.csv
Where 1234 is the control step count.
bYYYY is the Bangle step count (2.10.27 firmware or greater)
Name - your name
Activity - walking, sitting, sleeping, driving, tv-sitting, typing

I install the active Pedometer Widget and set the display to large digits so I can see what the step count is.

I'd be happy to checkin the logs into the various repo's if you are not used to git etc.

@user107850

OK, I have renamed time_t to sc_time_t so it doesn't conflict with linux time_t. Try pulling.

Thanks - can confirm compiles without error now.

Thanks to work on the harness - its great to have 2 algorithms to compare.
Are you going to write a brief for your students ?
Pointers I would give them would be:

1. Use 1 .c file and one .h file and a simple make file.
   
2. Do not over engineer the stages using fancy queues, buffers, message passing etc
   
3. Use the int_sqrt() function as that is what the embedded devices will mostly have to use.
   
4. avoid large arrays.
   
5. Add lots of comments to explain how the code works.
   
6. If you use some statistical symbol / concept make sure you reference to a site that explains it clearly - ie don't just say /* using black's algorithm */
   
7. Avoid fancy Computer Science terminology like 'single directed graph - instead use plain english like - tree structure'
   
8. Testing should prove good accuracy when walking for long periods (not just short bursts) and non stepping use cases such as sleeping. driving, sitting at a desk, watching TV - for an hour of such activities we want sub 10 steps to be logged.
   
   

As a general aim I think we should try to have something as good as an average fitness tracker.

Agree. I would like to get to a consistant 2% within control on walking and 200 steps across the day on non stepping activities. 200/10000 steps is 2%. Still a long way from the later.

I have a 1 hour train journey tomorrow so going to try an capture a log just sitting on a train - 0 steps.

In the end it depends on the use case.

I've had a few fitness bands / watches over the years. The most common use case for me and most people I know who use them is just monitoring that you are getting your 10,000 steps per day in. For training / running and anything that needs split second and distance accuracy you have to use GPS - I would see the Bangle Run app fulfilling that niche - if its made to work well.

3hz is within slow running pace, approx 12 mins per mile.
See the table I produced for this. http://forum.espruino.com/conversations/359542/?offset=75

There is a fundemental limit on what we can do with the Bangle with a 12.5hz sample time and 3hz low pass filter. The effectiveness of the LPF is constrained by the sampling rate.

@HughB You guys still working on this? The other day I was looking through a drawer and found that old Fitbit I mentioned earlier in this thread (a Charge 2). Made me remember the work you guys were doing here...

Today I wore it right next to my Bangle.js 2 with the Pedometer widget running. Wore them from 07:00 to 22:00 and here are the results:
Fitbit - 8765
Bangle - 10188

This was a mixed day of sitting at the computer (a lot), doing housework, walking to the store, playing with the little one, etc.

The discrepancy is quite big, so if you're still working on optimising the algorithms I'd be happy to help any way I can.

Hi there @johan_m_o, I have gone as far as I can with the current algorithm. In the early days it was a aweful lot of testing by actually walking. The current approach has weaknesses but I am not sure how to proceed to fix them. The challenge is not so much counting steps but not counting them when you are not walking. Driving and Housework are the area's where the current algorithm over counts. The good news is that we have a test harness for testing new approaches that is fairly accurate in predicting real word results. We can run a number of accelerometer logs of known step counts against the test harness. I was very pleased to see that we beat the oxford step counter both on walking and non walking recordings. I have tried a couple of tweaks but when tested they did not improve things and in some cases made them worse.

In comparison to earlier implementations of the steps counter; on Bangle 2 I have hardly seen anyone complain about the step counter, its been several months since Bangle 2 got out there and your post is the first I have seen saying you think its not as accurate as it might be. There have been a couple of posts by people who appear to have bad accelerometers but not other mention.

The other thing I would say is that there can be a wide variation of results across step counters even when doing a controlled test like a 30 minute walk around the same route or a 20 minute drive, there can be 1-5 % variation on the same device etc. FitBit in the early days was notorious for counting steps when driving.

We have a new generation of Bangle JS programmers now so I am hoping someone will be interested enough to take it to the next level - I hope they can come up with something that performs better than where I was able to get it to AND the test harness results prove the improvements work.

We have a new generation of Bangle JS programmers now so I am hoping someone will be interested enough to take it to the next level

I just spent the last hour since @johan_m_o posted reading all about step counting algorithms. When I'm actually on the computer with access to a compiler I might try a few things out just to get a grasp of how the test harness works (the link is quite buried, so for people who haven't already read this whole thread twice: https://github.com/dariosalvi78/banglejs-algos-tester.).

I was hoping to have a look at this paper, which seemed interesting: https://ieeexplore.ieee.org/abstract/document/9050443 but sadly it is paywalled. Some of the other papers I have looked at use gyroscope readings instead of or in addition to accelerometer, which is something I don't think the data in the test harness has (right?).

I read a variety of papers, most of them did not demonstrate test results for non stepping activities; such as driving, sleeping, sitting at a desk. From experiments I came to the conclusion that anyone can think they have built a step counter that counts steps well when walking but 95% of the problem is coming up with one that does not count when driving, sitting, moving a mouse etc etc. Any paper that does not cover how to differentiate between activities or tests for non step activities probably is suspect. The Oxford step counter looked good on paper, the technical paper looked impressive but the results were actually pretty bad under a balanced test regime.

The easiest way to start is to do the work in Javascript, then you can quickly prototype ideas and dont have to compile and flash firmware. If you read this whole thread you will see some sample code I wrote in javascript which Gordon converted to C. I then tweaked the C and tested against various accelerometer logs that I recorded of known step counts. The link you posted above came back 404. The test harness Gordon developed for Bangle is at:
https://github.com/gfwilliams/step-count

The link you posted above came back 404. The test harness Gordon developed for Bangle is at:
https://github.com/gfwilliams/step-count­

I had added an extra dot to the URL by mistake, but I was apparrently confused about which of the test harnesses was the test harness (there are two discussed in the forum, and working backwards through the posts the one I came to was the one I found first).

The easiest way to start is to do the work in Javascript

Maybe, but if I'm going to prototype I may as well prototype in any language. As far as I know the test harness(es) only exist in C, so it is write in C or create my own test environment for another language (and if I'm doing that, it may as well be Python).

Someone has kindly sent me the paper I linked to, so I will read that later and see what it brings.

but if I'm going to prototype I may as well prototype in any language.

Sure but the point about prototyping in Javascripts is that you can actually run it on the watch as an app and test it out in the real world that your idea's work. Many ideas look like they work on paper but in reality behave less when on the watch.

I implemented the current algorithm (or changes to the current algorithm) in javascript and was able to compare against the existing firmware at the same time.

2 test harnesses.

One is a generic harness that works in C that the guy who managed the oxford step counter built. He added the the Bangle code into it. That would be a good place to start BUT you need to test this on much longer walking samples - 10K steps, not just a burst of 150 steps, an approach I feel is flawed as it hides the errors that will build up over a longer period.

The 2nd Test Harness is is the Espruino codebase and you can fiddle with the step code in C and rerun the harness against all the samples.

If you are really serious about this I could spend sometime writing up how to run the test harnesses but the information is actuall all there. If you get it up and running make some improvements or come up with better code / options that beat the current results, especially in terms of the weak area's like housework, driving, and sleeping which should be lower AND without reducing accuracy on the counting when walking - I will send you my spare Bangle 2.

I have played around for a few hours this evening. I didn't have any success compiling Gordon's test arness, even though the instructions were straight forward. Is there anything obvious to you?

$ make
gcc -std=c99 main.c -o main
In file included from main.c:55:
../Espruino/libs/misc/stepcount.c:19:10: fatal error: jsutils.h: No such file or directory
   19 | [#include](https://forum.espruino.com/search/?q=%23include) "jsutils.h"
      |          ^~~~~~~~~~~
compilation terminated.
make: *** [Makefile:2: all] Error 1

I after updating my Espruino and step-count repos, I get the same error.
I noticed that @Gordon added jsutils.h on 14 Jan to stepcount.c
Probably quicker for Gordon to comment.
I'm not up for debugging this one right now.

$ pwd
/home/hughbarney/src/step-count
$ make
gcc -std=c99 main.c -o main
In file included from main.c:55:
../Espruino/libs/misc/stepcount.c:19:10: fatal error: jsutils.h: No such file or directory
 [#include](https://forum.espruino.com/search/?q=%23include) "jsutils.h"
          ^~~~~~~~~~~
compilation terminated.
make: *** [Makefile:2: all] Error 1
$