How to use quantized TensorFlow Lite files on Bangle?

Posted on
Page
of 2
/ 2
Next
  • Am trying to wrap my head around TensorFlow Lite on Bangle. Have successfully managed to collect gesture data (twitch hand left and right) from the watch and also trained a neural network in Edge Impulse.
    From Edge Impulse I can then download float32 and int8 quantized files as well as the full SavedModel files (see their forum: https://forum.edgeimpulse.com/t/using-ed­ge-impulse-with-other-boards/88/9).
    I'm attaching the int8 and the SavedModel files for reference.

    My question is how I can deploy these files to Bangle and use them there?

    Have backed Bangle v2 and am hoping I could include it in my master thesis research. The thesis project is to conduct a AI-course at university level (during spring 2022), and as main platform I'm planning to use Edge Impulse, am now "hoarding" different edge devices like Bangle, OpenMV Cam, HiMax etc. to use for collecting different type of data. Bangle v2 seems like an optimal fit for accelerometer type of data as it can be transferred wirelessly. If I first get something working on Bangle v1 and later on v2, I'll see if my university would be willing to buy 10 to 20 watches to use by the students.


    2 Attachments

  • Fri 2021.09.24

    Hi Thomas @ThomasVikström while I have no experience with TensorFlow, I was able to locate a sample code snippet.

    Tflite input data type (int8) conversion in javascript

    Until others with more experience respond, there are other samples to follow:

    Google:    tensorflow lite   site:espruino.com

  • Thx @Robin!
    I had indeed before searched the Espruino forum and internet, but for some reason I did not find the post you are linking to.
    In this forum and on the net I've found different bits and pieces, but not the whole picture, or at least I cannot understand it.

    What I'm not understanding is how to use the tf-lite model files, can/should I save them to Bangle first and invoke them? How?
    Perhaps @Gordon can provide me some pointers

  • Probably below code snippet is part of the solution, but how do I copy the content from the .lite file to …atob("....")… ? The .lite file seems to be a binary file, should I convert it first?

    require("Storage").write(".tfmodel",atob­("....."));
    require("Storage").write(".tfnames","lef­t,right");
    
    
  • Try using the File Converter to convert it to Base 64.

  • Hi! That's great news about the export from Edge Impulse, I didn't realise you could do that!

    I think the best solution is actually to upload via the IDE. So go to the IDE, click the Storage icon (4 discs) in the middle of the screen, then 'Upload a file', choose the file and name it '.tfmodel'.

    There should really be a proper tutorial (and if Edge Impulse's export really works then I think that'd be a great thing for me to add) but right now the main examples are at:

    https://github.com/espruino/Espruino/blo­b/master/libs/tensorflow/README.md#actua­lly-using-it

    and

    https://nodewatch.dev/software#google-co­lab-from-workshop (which includes a Google Colab for gesture training, as well as the code to export the .lite file to base64 if you wanted to go that route)

  • @NebbishHacker & @Gordon
    Thx a lot both of you!

    With your guidance I've now been able to try out my very simple model and it works technically, albeit it almost always shows "right" even if it should have been "left". This might very well be because I only had 10 samples of each which probably is far too small dataset, I'll look into this.

    I had actually tried the Google Colab workshop but the quantized exporting always fails with "ValueError: the operator SPACE_TO_BATCH_ND is not supported by TFLite Micro", and as I for now prefer Edge Impulse for its simplicity, I did not dig deep into possible causes of the error message.

    @Gordon
    Yes, a tutorial would be nice! When I myself understand the process and get it streamlined I'll write the steps down as I'll anyhow need them later in my research. I'll provide them to you in case you might benefit from them.

  • Sat 2021.09.25

    How about directly to the 'Tutorials' forum heading:

    Home >> Tutorials
    http://forum.espruino.com/microcosms/130­/

  • That could be an option, especially if it would be possible to paste images (= screen captures) from Espruino Web IDE, Edge Impulse Studio etc.

  • 'paste images (= screen captures) from Espruino Web IDE'

    Use the Image button above (between Link and Quote) this Post a reply edit window

  • Well, yes... but then I'd need to store the images at some website and paste the URL into the link box. I can't just paste the image from the clipboard like in e.g. Word.

  • You can attach images to the posts... To be honest even if you just emailed me it'd be a great help and I could convert them. It'd just be nice to have a decent tutorial on it

  • Sure, I'll write down the steps, take screenshots and provide them one way or another to you.

    After some struggles, I'm new to JavaScript, I've since an hour ago finally been able to collect gesture data, train it in Edge Impulse and upload the model to Bangle. Now it works fine, and I have e.g. been able to control a PowerPoint presentation just by twitching my hand. That comes in handy as I'm a teacher :-)

    When using CSV-files with Edge Impulse, the reguirement is one sample per file, format is like this:

    timestamp, x, y, z
    0,  7, -5, -65
    3,  7, -13, -61
    6,  5, -1, -48
    9,  4, 41, -24
    ...
    

    With Bangle JS I've so far created one file per sample, but it gets quickly tedious when I need to download a lot of CSV-files from Bangle to my computer. Instead I'd like to create one big file per event (e.g. one file for samples of left twitches, one for right, one for up ...). But how to split these files into separate CSV-files on my computer? I'll later search for some Python-code or similar, but if anyone has something ready made I'm grateful.

  • Ok, to split files with Python, this can be used

    import re
    PATENTS = 'C:\\temp\left.txt'
    
    def split_file(filename):
        # Open file to read
        with open(filename, "r") as r:
    
            # Counter
            n=0
    
            # Start reading file line by line
            for i, line in enumerate(r):
    
                # If line match with teplate -- <?xml --increase counter n
                if re.match(r'timestamp, x, y, z', line):
                    n+=1
    
                    # This "if" can be deleted, without it will start naming from 1
                    # or you can keep it. It depends where is "re" will find at
                    # first time the template. In my case it was first line
                    if i == 0:
                        n = 0               
    
                # Write lines to file    
                with open("{}-{}.csv".format(PATENTS, n), "a") as f:
                    f.write(line)
    
    split_file(PATENTS)
    
    

    Found it at Stackoverflow.
    Need to modify the JS-code slightly to create just one file per event type, but that's not a biggie. After that I might have most other things in place for a write-up.

  • @Gordon, I have now started with the tutorial, I hope it's ok that I use my own website for it? This way I also learn how to add posts there using Wordpress.

  • Great! Well, it's totally up to you. I think it'd be nice to have something on the Espruino website but I guess I can always have a page that just links to yours.

  • Well, I'd also prefer it to be on the Espruino website, I don't have any specific needs to have it on my own website. Do note that I'll also cover the Edge Impulse part in the tutorial, so it will not only be about Bangle.

    Seems like your website is pretty much done with Github. I seem to have a Github account from before, so I'll setup a private project and try to create the tutorial there instead. Then I guess I can some way or another share or send the content to you? Don't know yet though how the image embedding/linking works, I'll need images mainly for the Edge Impulse part, so in the end perhaps not too many.

  • Ok, great! Did you see this already? http://www.espruino.com/Writing+Tutorial­s

    There's a note in there about how to handle images too

  • Thx for the link, hadn't seen it! Had already started adding a few lines of text and images that I can copy over once I fork the EspruinoDocs-repository. Seems to be pretty straightforward to use.

  • @Gordon, have now created the tutorial using GitHub and made a pull request. Whenever you (or others) have time, take a look at it and try it out, and shoot me some comments!

  • @ThomasVikström That looks really good! My Bangle.js v1 is currently dead because I've lost my charging cable, otherwise I would get on testing this straight away.

    Been wanting to dabble with this kind of stuff since I got the watch, but never had the time to figure it out. That tutorial of yours is gonna be a godsend once I have a watch that's running again.

  • Sat 2021.10.02

    reply post #20

    Tutorial at GitHub Direct link

    Bangle.js and Edge Impulse for machine learning #610

  • Thank you - this looks brilliant! I'm a bit busy right now with the KickStarter but I'll try and give this a try when things calm down. It looks so much easier/quicker than the Python alternative.

    Published version on the website is here: http://www.espruino.com/Bangle.js+EdgeIm­pulse

  • Great, thx!
    I'll also post at link to the tutorial on Edge Impulse forum.

  • Post a reply
    • Bold
    • Italics
    • Link
    • Image
    • List
    • Quote
    • code
    • Preview
About

How to use quantized TensorFlow Lite files on Bangle?

Posted by Avatar for ThomasVikström @ThomasVikström

Actions