Avatar for PaulC

PaulC

Member since Apr 2020 • Last active Jun 2020
  • 2 conversations
  • 14 comments

Computerologist

Most recent activity

  • in Bangle.js
    Avatar for PaulC

    @JumJum Fantastic work! I have updated the versions on my repo and it still work (surprising as its a major version bump)

    I have added a download button which will bring down the model on the prediction page.

    I ran through the conversion steps you posted, and they do produce a TFLite model (your Google-Fu is strong!)

    The problem I see now is that the TFLite model it produces is ~14Kb, and around 19Kb when converted to base64. I'm not sure if this is too large. I see the python script does quantization to optimize for size.

    I went through the steps @Gordon linked to, and the emulator runs out of memory, and I checked the model on the gesture app its around 5K. So maybe we can do some optimization during the conversion process?

  • in Bangle.js
    Avatar for PaulC

    Thanks @ConorONeill ! It will be interesting to see how this project evolves.

    I look forward to see what you build @JumJum, it's all very exciting.

    Just a quick note on all this: I have looked high and low for scripts that convert TFJS generated models to other model types (especially TFLite as that's what we require for the Bangle), and I don't see any, I also came across a post from April this year (by a TF contributor I believe) that said you can only convert models to TFJS not the other way around, and they don't plan on introducing that feature.

    I just don't want people to invest to much time only to find out there is a massive dead-end with model conversion.

    Hopefully I am wrong, and someone elses Google-Fu is better than mine and comes across a solution.

  • in Bangle.js
    Avatar for PaulC

    @JumJum Glad it worked for you!
    @Gordon Cool, yeah I think the gesture detection stuff opens up many doors for cool projects.

    Well as per the apps, this belongs to the community, so if you missed the repo link in the site, here it is: https://github.com/paulcockrell/bangle-t­ensorflow-game

    (The 'game' in the repo name is there as I had a PhaserIO game hooked in but this was too much of a distraction to the point of the repo, so I removed it)

    Please feel free to fork the repo and work on it, I don't know how you will go about coordinating if its not in the main Bangle repo (for issues and feature requests).

    I cannot guarantee my availability for chipping in on this so please take over and run with it... I'll join in where I can :)

    Cheers.

    Paul C

    Side note: It looks like you would have to have an endpoint on a server to convert the generated tfjs model to any other model type, doesn't appear to be a part of TensorflowJS Api unfortunately.

    The model this generates is in local storage (devtools->application->local storage)

  • in Bangle.js
    Avatar for PaulC

    Yeah its all in the browser!

    Yes you will get an unhelpfull 'training failed' message if you record only one gesture..

    The UI is a bit non-intuitive, but these are the steps to do a successful train:

    1. Enter gesture name + select sample size (5 is fine if the gesture movements are very different)
    2. Click "Record gesture" .. the page will scroll down to the feed back section, wait for the feed back bar to turn red (recording mode) and the message to say its ready.
    3. Perform your gestures (first always fails as it needs the raw data to be cleaned..)
    4. Once the feedback bar goes green, scroll back up to the top of the page, and enter a NEW gesture name, keep the sample value the same as before.
    5. Perform steps 2-3 again
    6. Now you will have recorded TWO gestures (you can confirm this in the bangle gesture readings section at the bottom of page)
    7. Click train, and let the magic happen.

    I can certainly look at having it generate a TFLite model. I wanted to know if this was something of use to the community before investing much time!

    Let me know if you are successful.

  • in Bangle.js
    Avatar for PaulC

    Hi, as this is not an app, for the app store, I thought I'd post it here and see what people made of it.

    I have put together a purely online gesture learning AI for the BangleJS: https://paulcockrell.github.io/banglejs-­tensorflow-example/

    This is not the same as the one that generates a model in Python, for you to load into the Bangle. But rather this simply lets your bangle send raw gesture data to the browser, and it will use the Tensorflow model (stored in local storage in the browser) to predict your gestures.

    Since the Espruino stuff is all about the Javascript, I thought it would be nice to have a pure JS implementation for gesture learning. Its all in the browser, so no server setup.

    I don't know exactly how this would be applied to something (maybe browser card games, or menus..) but thought it would be fun.

    Props to the following that helped alot with this project:

    1. https://medium.com/@devdevcharlie/play-s­treet-fighter-with-body-movements-using-­arduino-and-tensorflow-js-6b0e4734e118
    2. https://www.nearform.com/blog/running-te­nsorflow-lite-on-nodewatch-bangle-js/


  • in Bangle.js
    Avatar for PaulC

    Hi @Robin, Yes I guess that might be a solution going forward, as we all hope the BangleCraze will take off soon, but that will lead to a flood of app submissions and as you point out that may bring a ton of issues.

    @Gordon has really hit the nail on the head with the Bangle App store. I hope schools pick up on it, as I think this would get kids buzzing.. no soldering, electronics theory or easy to swallow components, just a chunky watch, a bit of JavaScript, and imagination. All other IOT projects can only really end up as lengthy blogs, and private implementations.

    Back to the point tho, You raise good issues in your can-of-worms thread. I can't remember but did Pebble have a community app store, can we get hold of any ex-engineers to see what they did from a legal point of view etc?

    This really isn't my area of expertise so I can't chip in.

    • 7 comments
    • 351 views
  • in Bangle.js
    Avatar for PaulC

    Hi @Gordon

    I let the battery drain and installed the firmware again. No luck in afraid. I tried the commands you suggested and not much changed. Any idea on what I could try next?

    Cheers

  • in Bangle.js
    Avatar for PaulC

    I wonder if this needs a little more thought. Usually the creator will always posses and manage the original source in their own repo, no matter how its licensed. But here it is relinquished to the community immediately, where the original vision can change at whim through the community (through no bad intentions, but everyone likes something slightly different).

    I think this ultimately would be a bad thing for the app-store, where over time apps become more platypus and less the original duck.

    For example: I wrote the Mario Clock watch face. It is designed to mimic the classic Gameboy design (low res graphics and grey-scale colours). Someone could come along with very good intentions (and probably better skill) and make changes to make the graphics: high-res, 3D, and full colour, like on Mario Kart or something. This would ruin the watch face for anyone that loves the intended retro Gameboy Mario look, but make a different set of people happy. Thus conflict.

    I wouldn't know how you'd manage that problem.

    Just my ten-cents, my fear is more to do with success of the app-store and its apps.

Actions