Tensorflow on MDBT42Q ?

Posted on
  • My knowledge about TensorFlow (see Bangle.js) is close to 0. Before I invest time in that, I would like to get an idea from the expert whether my idea would be possible at all.
    Could we implement tensor flow for the MDBT42Q board to get speech recognition?
    For the beginning 18 words from standard (18w) would be fine. If we could have or train an additional start word (like Gordon or Espruino for example), life could be wonderful.

  • :) Yes, that's actually totally possible. Compiling Tensorflow for MDBT42Q is as simple as adding TENSORFLOW to the board.py file and maybe taking out some other stuff (NET?) so it fits.

    Tensorflow Lite for Microcontrollers has an example of speech recognition too. The only gotcha is that the speech recognition needs to have the input sound 'windowed' and an FFT performed on each window. There's code for doing that inside TF but it's not built into Espruino - so you'd have to add that to the build and then modify the existing code so you could call it.

    But it's totally possible - and something I would have built in from the start if Bangle.js had a microphone :)

  • From time to time I spent an hour or so for speech recognition.
    There is an example https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/micro/examples/micro_speech for tensor flow lite.
    One of my open questions is microphone.

    • simple microphone would expect fast A/D conversion.
    • I2S microphone (like INMP441) needs an I2S interface.
      At the end, both solutions need lot of memory, which will not be available on all boards.
      Last not least, see problems with adding tensorflow on MDBT42Q board in other chat.
      Any nice idea, how it could go on ?
  • I think you could just use Waveform - that can handle 8kHz which should be enough. But then you need FFT/etc. There's a lot of processing done which turns the sound data into something much lower bitrate.

    To be honest it could do with a bit of extra support for it in the Espruino firmware.

    Any nice idea, how it could go on ?

    It should if you remove some of the extra functionality (like networking!) :) However I found that actually the 'hello world' example I based Espruino on contains everything. You can delete 3/4 of the ops available and it'll be fine - I'll be merging tensorflow2.2 soon and I'll do that then, so it should fit much more easily

  • Post a reply
    • Bold
    • Italics
    • Link
    • Image
    • List
    • Quote
    • code
    • Preview
About

Tensorflow on MDBT42Q ?

Posted by Avatar for JumJum @JumJum

Actions