Search results for:

Hi Gordon,

I have some news. The students have completed their work on development and bencharming of the algorithms. They had time to work only on the step counting, we have left the heart rate measurement for later. We decided to create our own repository for this, as it was more practical. It doesn't follow the code conventions of Espruino completely, but it should be relatively simple to adapt the code in case you want to try it within the firmware. The repository is here: https://github.com/VirginiaSek/Algo

But I guess that you would be more interested in the results from the tests the students have done. I am sharing their thesis here in attachment.

If you want to discuss these results, I am happy to do it here or by email or a call. As for the data they collected and used, we would need to take it privately.

Hope this helps!

thanks for the availability and the suggestions! You will be receiving pull requests soon 😃. I'll keep the signature as it is for now, and only change it if it is required.

One little note about the code, I see that you mostly use generic types like short, int, float etc. We will try to keep that style too, although I wonder if it wouldn't make more sense to be more specific and indicate also the n. of bits? What's the philosophy here?

Re. activity detection: it is not planned at the moment, but it's something I'd like to work on at some point.

hello again, some news about the new algorithms benchmark.

Data: we have prepared a data collection protocol, we aim at 20 people, doing different activities, both on treadmill and outdoor walking. The protocol will get an ethical approval from Politecnico di Milano. Right now we don't have the permission to publish the data, but we are planning to obtain it. I understand that the data is what you would mostly appreciate, so I am keen to publish those files, but it may take some time. We are planning to acquire the data at its default sampling frequency for both acceleration and PPG. Would that be OK for you, or would it make more sense to acquire at higher frequencies, in case you want to test out different frequencies?

Tests: the repository for the tests that you have may need some adaptation though. We are starting with the step counter and we can already see some changes to make. Concretely:

1. the main.c needs to be adapted to be able to run different algorithms, not just 2 (espruino and "original"), we are going to change it and make it more general. I hope that it's OK?
   
2. in terms of output, I would suggest to print out just the totals, comma separated, then that could be copied into a spreadsheet where all sorts of statistics can be done. Or we add stats to main.c, but maybe after the totals.
   
3. algorithms would go into a dedicated folder, that includes moving the "original" algorithm into its own .h/.c files instead of being mixed within main.c. We can keep the espruino one linked from the Espruino code so it's always upated to the last version.
   
   

If you are OK with these, we will be sending pull requests, as we advance with the task. You can of course review those and suggest changes if there's anything you don't like.

Another issue: from what I understand the current API for the step counter (stepcount_new(int accMagSquared)) return a 1 if a step is detected or a 0 otherwise (and in some cases it tries to make up for lost steps too? haven't studied this well yet). I would suggest to change the signature to something that is more generalisable for other algorithms, for example instead of returning the "new" steps, it would return the total step count, since the algorithm was inited (or reset) and instead of accepting the magnitude as argument, it could accept the raw x, y and z. I know that this has repercussions in many places within the Esrpuino code, so it's a lot to ask, but it would make the signature compatible with algorithms that are based on different approaches. Let me know how doable this would be. We can of course decide later, if and when we have some good alternative candidate algorithm.

Thanks!

thanks for the answers!

That would allow to do circular buffer in the file.

A circular buffer is roughly the idea I am thinking about: as new data comes in it gets appended. When the file is full, old data gets overwritten. This should be possible pre-allocating enough space using also the size parameter in the write() method so that I can re-write on the same file with offset in the write() method to append. However, the documentation says that I can't rewrite the same position on the same file which brings two problems:

Problem 1: I cannot rewrite older data. Workaround is to pre-allocate enough space and not overwrite. If space ends, I either create a larger file (and copy the existing content), or I give up and warn the user. I can live with the second, though not great.

Problem 2: I need to keep track of what is the last byte that was written on the file, otherwise when I read back the file I will get all the "empty" data, the 0xFF bytes. Solutions are that I either impose that 0xFF cannot be used or that I keep track of the actual size of the storage file in a separate permanent memory, but Storage (and StorageFiles) would not work for this because I cannot rewrite them.

Do you wise people have a better idea?
Gordon, maybe you can extend the current API to support overwriting already written positions?

Hello,

we are building a sort of fitness tracker type of app. The idea is to store data at a high(ish) freuency (minute by minute or so) which does not fit with the already existing health database that you have implemented.

In order to use space efficiently, we would like to save the data in a binary format, AND we need to append data to the file. Basically the files are written in append mode on sensors reading, and read week or so for synchronization with a smartphone.

Which option do you suggest using?

1. Storage.write() and Storage.read(). The problem with these is that there is no append mode. Maybe pre-allocating the file size and then writing with an offset would work? If yes, is there any efficient way to read bytes instead of strings (ideally without preloading everyting into RAM)?

2. StorageFiles offer a handy append mode, but, AFAIK, one can only write strings.

3. Flash library, would do to access the underlying flash memory, but I checked how much memory was available and got only 250kB, which makes me think that it points at the 1024kB on-chip flash instead of the exteranl 8MB flash? There is no way to access that memory directly (and safely)?

any other idea?

Thanks.

what do you use for capturing the reference data?

HR: Polar HR sensor with BLE (based on ECG)
Steps: we put an IMU on a shoe. With some simple thresholding, the step count is quite reliable.

@Gordon I am working with 2 students on the algorithms for HR and steps. We would like to integrate with your work in the most efficient way so that, in case they come up with something promising, you can integrate their code easily into the firmware.

What we are going to do:

• collect raw data with reference, both step count and heart rate
  
• test out a few alternative algos and compare them in terms of accuracy / resources utilization
  
  

I am aware of these two repos that you have created:
https://github.com/gfwilliams/EspruinoHRMTestHarness
and
https://github.com/gfwilliams/step-count

We can simply push code to those 2, or, maybe more efficiently, we could merge them into one?
Some guidance would be greatly appreciated.

Thanks...

My wishlist:

1. OLED or (even cooler) e-ink
   
2. more buttons (useful but not strictly needed)
   
3. accelerometer with integrated step counter (Bosch has some really good ones)
   
4. PPG sensor with well documented HW and SW
   
5. multi color PPG (for example green and infrared) for oxygen saturation
   
   

I think that there is demand for really programmable smartwatches. So far the only ones I am aware of are yours and PineTime.

Hello all,

very interesting developments here!
I have been trying the new algorithms myself too. I had the Heart Rate Monitor app on the Bangls JS (version 2 bought from Kickstarter campaign, FW 2v19), and another smartwatch plus a PPG from the samrtphone on the other arm. I have noticed some differences, especially at lower HRs, but couldn't do this very systematically. I am planning to do some more tests with a bluetooth Polar ECG belt in the coming days and will report back.

I have read in this thread that the manufacturer's library also supports SPO2 and Blood Pressure. It would be extremely interesting to receive these 2 as well. I do not trust BP at all, but SPO2 is doable and has been proven reliable on other devices (for example Withings). Is there any plan to support these? And /or to allow collecting PPG from the 2 LEDs (my understanding is that there is a green LED and an Infrared LED)?

Thanks!

For developing a good algorithm we would need the following:

1. raw PPG signal (as it comes from the driver)
   
2. acceleration signal (useful for trying out motion artefacts cancellation)
   
3. reference HR
   
   

looks like we can get all of them with the apps we have?

Hi all,

linking to this other thread, I had setup a repo for testing algorithms on Bangle JS. We currently have algos for step counting but it would make sense to add HR as well.

I have some student working on HR from PPG on mobile phones, but, if time allows, would like also to contribute to BangleJS.

The first thing to do would be to collect raw PPG data, as it comes to the algorithm, with a reference HR from another, trusted, sensor.

Once we are happy with the HR, it would be also cool to see if we can extract other parameters, such as respiration rate, but that would probably come later...

thanks for the explanation, now it's more or less clear.

As for testing, I have created a repo for testing the algos of BangleJS. Currently it contains only the step counter, but we could happily add also the heart rate. The idea is to upload some test data with reference (maybe coming from another commercial sensor) and benchmark different algorithms.

Hi,

I have some students developing an algorithm to detect HR from raw PPG. We are trying to make the algorithm somewhat hardware indipendent, so the outcome could be applicable to Bangle as well (if good enough!). But first I wanted to understand how the current implementation works.

I know you used to compute autocorrelation at some point, but the current algorithm seems more like a peak detector? As the code is not greatly commented, can anybody (Gordon?) explain a little what is the idea behind?

Thanks a lot,
Dario

I forget how little portable C is!
OK, I have renamed time_t to sc_time_t so it doesn't conflict with linux time_t. Try pulling.

Need some 1 hour logs for sleep, driving, working, sitting watching TV - all ZERO step logs.

Yes, but let's not forget collecting steps data under different circumstances (slow walk, fast walk, running, etc).

It would be nice if we could involve other users, just to confirm that there are no major differences across devices.

Hi @fanoush, I have followed your work closely in the past. I can see that this is a collaboration among the best experts in reverse engineering watches there are around 😀!

As far as I understand what you are saying, the driver is just an implementation of the I2C protocol, and you have reverse-engineered it. Quite an achievement without a datasheet.

As for the higher level library, the reason why you do not want to use it is because it's not open sourced?

In any case, the fact that it requires acceleration is an indication that it makes use of it to probably discard data that is corrupted by movement. We could probably record some raw signals, including acceleration and PPG and put them in a repository for benchmarking different versions of algorithms. We would also need a reference heart rate for that, which we can obtain from another device, like a Polar or similar. Then we could invite people to test their best algorithms in some sort of competition...

I am working on an HR detection algorithm with students, but using the PPG from a smartphone. As the problem is similar, I may try to include data from the BangleJS as well. I'll keep you informed.

Hello, I have added the espruino step counter as well (with minor changes in names). Looks like it has better accuracy than the Oxford one, especially in absence of movement. Well done!

We could use this repository to try to improve the algos with further tweaking. Pull requests are very welcome.

Hi,
sorry there were a few bugs, I should have fixed them now, please pull. I have also run some more benchmarks with your new data and tuned the movement detection stage a bit.

It would be useful to keep the same file names so that provenance of the logs can be tracked.

Yes, good idea! The only requirement is that the first characters correspond to what we consider the "ground truth".

The Accelerometer logs below are all ZERO steps and should be part of the controlled test data in my view.

Yes, if you are sure about the fact that you did not do any step.

These and similar logs must form at least 50% of the test data in any test harness.

The more data we have the better. Actually we should also include running and walking in different conditions, like on soft floor, hard floor, with and without shoes etc.

No algorithm is going to be perfect, some may be more precise when walking, others when not. In the end it depends on the use case: runners may prefer better accuracy when running, others may prefer better accuracy when walking etc.

As a general aim I think we should try to have something as good as an average fitness tracker.

Hello @Gordon , thanks for the reply.

I understand that it makes sense to try with a more ad-hoc peak detector, especially if you know that the hardware is always the same. Our algorithm can be optimised a lot, starting from removing the buffers, but we didn't have enough time for that. As for the heart rate, I would probably trust the proprietary blob better than anything else. With time, if we manage to get a reliable algorithm, we would ideally get rid of it.

Back to the competition idea: I have created a repository where we can test algorithms, see here.

I have already populated it with the data I have collected myself plus some of the data collected by @HughB. I have not included those files where the two measurements of steps are too different, because I don't trust either, and for those that I have included, I have computed the average between the two given step counts.

@HughB would you be able to send me a pull request with your algorithm? Please check the structure inside the dummy algo. If that doesn't fit with yours, I am happy to change it, just le me know (or submit an issue on Github).

With time, we can add more algorithms, or simply improve ours. It's going to be fun (at least for me😀).

I would like to keep this repository also for testing the algorithms for heart rate or sleep detection when we get to it, and I will probably share it with students if I manage to involve some.

OK, good to know!
I guess that the vcare library is not open source?
Does the library also compute HR?

Hello,
one note about your "ringing": it is maybe because of the pass-band filter. Have you tried with a low pass instead? Or can you just remove the filter altogether?

It would be interesting to see what it looked like without the buffering and pipeline and was just sequential.

Yes, I asked students to do that but they didn't have enough time. I agree that the buffers are not needed.

The approach the Bangle currently takes is very similar

Peak detectors are somewhat similar. The main issue is adapting to a signal where amplitude can change quickly. There are also other approaches than peak detection, like analysis in frequency, wavelets, or even machine learning, but I don't think they are suitable for an embedded device like a smartwatch.

If you could build a framework around those test cases above then I think we would have what we need.

Yes, I'll set it up and post the link here. I'll try to do it next week.

Hi, yes I was involved, I proposed the idea as a BsC project and I supervised students both at Oxford and at Malmo who worked on that.

The approach is described in two papers: here (behind paywall) and here. The basic idea is that the signal is converted to magnitude, then peaks are somewhat amplified, identified and selected.

You can split the algorithm into a pipeline of stages, with buffers between each stage. Not all stages are mandatory, for example I observed that interpolation and linear filtering do not bring any significative improvement and can be skipped.

As far as I remember, Gordon embedded the algorithm into the Espruino firmware at some point, but he complained about the fact that it took too much memory. After that I spent some time trying to remove redundancies and optimising memory a little bit, but I don't know if Gordon included those changes or if he decided to adopt another approach in the end.

Re. non-walking activities: the algorithm was meant to be used during walk and was not optimised to distinguish between genuine walk and other activities. To address this, I later added a basic movement detection step which improved things a bit on my short tests, but I have never tested it "in the wild" for a long time.

As for the competition, we could separate the algorithm from the Espruino code, prepare the testing dataset and add some wrapping code that runs the algorithm and compares it with the reference. I have something similar already I used myself to test my algorithm: I would happily polish it a little and share it if there's interest.

thanks. So it looks like they do not come with embedded algorithms. Developing some reliable step counting and heart rate measurement is going to be a challenge for us!

Hi all, excited to see how things are progressing here. I was curious to know how much of the original windowed peak detection is left in the current implementation?

The last version of the Oxford step counter in C was optimised for Bangle actually, and I even added a detection stage to filter out non-movement based on a minimum amplitude. I collected some data myself, 17 files with manually counted reference, and the accuracy was >90%.

Gordon, maybe having some sort of competition would be indeed a good idea, but we need to set it up well, which means good data and a simple way to run the code. I could also propose it to students here at the Uni.

A bit unrelated, but we could do the same for heart rate.

Gordon this is absolutely fantastic!

As my application area is health and fitness, I wonder what can you tell us about the step counter and the heart rate sensor. Chips used? Does the accelerometer include its HW step counter or are you relying on the same algorithm used on Bangle 1? What about the heart rate sensor?

I am very excited about this project, thanks very much for working on it.

thank you all! you can follow the development at https://github.com/espruino/Espruino/issues/1846