Recording and analyzing sound

Posted on
  • Hi Guys,

    I have a sound which is between 150Hz-250Hz. I need two things:

    1) The average frequency of the sound for the last 5 seconds. After a discussion with @Gordon it appeared that 1Khz sampling rate will be enough and this could be done on the Puck with FTT.
    2) Get the digitalized sound from the microphone for the last 5 seconds.

    The microphone is connected on D2(for analog), D1 (for power) and GND.

    Thank you.

  • Something like this seems to work:

    var w = new Waveform(512,{doubleBuffer:true,bits:16}­);
    var a2 = new Uint16Array(512);
    var a = new Int16Array(a2.buffer);
    w.on("buffer", function(buf) {
      a2.set(buf);
      E.FFT(a2);
      var m=0,n=-1,b;
      for (var i=150;i<250;i++){b=Math.abs(a[i]);if(b>n­){n=b;m=i;}}
      console.log(m.toFixed(0)+"Hz @ "+n);
    });
    w.startInput(D2,512,{repeat:true});
    

    It's sampling at 512Hz, which seems to be good enough for what you want.

    Ideally the input voltage range would be higher and you could go back to 8 bits though - I had to use 16 because my microphone isn't very good.

    Also, there appears to be some bug in the FFT where it's returning signed values when it shouldn't be - I'll look into that, but for now the attached code seems to work ok.

  • Ok, FFT is now fixed, and will be in the 1v92 release.

    When 1v92 is released you'll be able to use this (slightly cleaner & faster) code:

    var w = new Waveform(512,{doubleBuffer:true,bits:16}­);
    var a = new Uint16Array(512);
    w.on("buffer", function(buf) {
      a.set(buf);
      E.FFT(a);
      var m=0,n=-1;
      for (var i=150;i<250;i++)if(a[i]>n)n=a[m=i];
      console.log(m.toFixed(0)+"Hz @ "+n);
    });
    w.startInput(D2,512,{repeat:true});
    

    The 'for' loop is intentionally quite tight - Espruino's execution speed isn't that fast so I'm trying to make it as quick as possible.

    The @ xxx part that's printed is the volume of the most prominent frequency. If no frequency is playing then you'll get a random frequency shown, but with a lower volume.

    Also, just a note: don't expect great battery life when using this. Realistically you'll need to use a bigger power source if you want to get more than a day's life out of it.

  • Hi Gordon,

    Really appreciate your feedback and quick reaction! I will require an average frequency for 5 seconds every hour, as we discussed it could double decrease the battery life, but the Puck open architecture give possibility for additional power supply, everything is a matter of evaluation.

    Thank you again.

  • Ahh, forgot about that - yes, 5 seconds an hour will provide much better battery life.

    It should be easy enough for you to tweak the code above to do a 5 second average (probably just keep a sum of the FFTs, and do the loop over them at the end).

  • Hi @Gordon,

    Based on the code below:

    var clipSeconds = 3, currentSecond = 0;
    var w = new Waveform(512,{doubleBuffer:true,bits:16}­);
    var a = new Uint16Array(512 * clipSeconds);
    w.on("buffer", function(buf) {
    
      a.set(buf, currentSecond * 512);
      currentSecond++;
    
      if (currentSecond >= clipSeconds) {
        w.stop();
    
        E.FFT(a);
        var m=0,n=-1;
        for (var i=150;i<250;i++)if(a[i]>n)n=a[m=i];
        console.log(m.toFixed(0)+"Hz @ "+n);
      }
    });
    w.startInput(D2,512,{repeat:true});
    

    My understanding is that all the sound as buffer will be stored in "a" from where once all the data is aggregated we could define the frequency.

    When this code is deployed the flow hangs on "E.FFT(a);" and I cannot connect to the Puck anymore. Please advice.

    Thank you.

  • Are you using the up to date (1v92) firmware?

    If it still fails I'd try and simplify things a bit... if you have a massive buffer already, why not just write into it directly - for example:

    var clipSeconds = 3;
    var w = new Waveform(512*clipSeconds,{bits:16});
    w.on("finish", function(buf) {
      var a = buf;
      E.FFT(a);
      var m=0,n=-1;
      for (var i=150;i<250;i++)if(a[i]>n)n=a[m=i];
      console.log(m.toFixed(0)+"Hz @ "+n);
    });
    w.startInput(D2,512);
    

    It is possible that the buffer size is too large for E.FFT, but there are checks in place to avoid that kind of problem and to throw an error if there isn't enough memory available.

    Failing that, is there a reason you don't want to perform the FFT in smaller chunks and average the result?

  • Hi @Gordon,

    I have following use-cases.

    • Client requests an average frequency for the last N seconds.

    var clipSeconds=5, currentSecond=0, sBuff=new Uint8Array(clipSeconds);
    var w = new Waveform(512,{doubleBuffer:true,bits:16}­);
    var a = new Uint16Array(512);
    w.on("buffer", function(buf) {
      a.set(buf);
      E.FFT(a);
      var m=0,n=-1;
      for (var i=150;i<250;i++)if(a[i]>n)n=a[m=i];
      sBuff[currentSecond]=m.toFixed(0);
      console.log(sBuff[currentSecond]+"Hz @ "+n);
      currentSecond++;
      if (currentSecond >= clipSeconds) {
        w.stop();
        var avg = E.sum(sBuff)/clipSeconds;
        console.log("Average " + avg.toFixed(0) + " Hz");
      }
    });
    w.startInput(D2,512,{repeat:true});
    

    This code is tested and working on firmware (1v92)

    • Client requests an audio for the last N seconds.

    var clipSeconds = 5;
    var w = new Waveform(512*clipSeconds,{bits:16});
    w.on("finish", function(buf) {
      console.log(buf.length);
    });
    w.startInput(D2,512);
    

    For ought I see the "buf" contains the audio which should be stored and send to the client ?

    I really appreciate your feedback!

    Thank you.

  • Yes, that should be correct - you'll just have to be careful with how you send it to the client, since over Bluetooth LE it could take a while!

    You could also consider just recording it in 8 bits?

  • Hi @Gordon,

    Yes recording in 8 bits is reasonable.

    I would like to send the clip to the client. I have following questions based on the code:

    var soundBuff;
    NRF.setServices({
      0xBCDE : {
        0xABCD : {
          value : "N/A",
          readable : true,   // optional, default is false
          writable : true,   // optional, default is false
          notify : true,   // optional, default is false
          onWrite : function(evt) { // optional
            var clipSeconds = evt.data[0];
            var w = new Waveform(512*clipSeconds,{bits:8});
            w.on("finish", function(buf) {
              soundBuff=buf;
              NRF.updateServices({
                0xBCDE : {
                  0xABCD : {
                    value : buf,
                    notify: true
                  }
                }
              });
            });
            w.startInput(D2,512);
          }
        }
      }
    });
    
    1. Is this the right way of doing this ?
    2. How can I free memory once the data is sent to the client ?
    3. How can I view the result with "nRF Connect"

    Thank you.

  • Hi,

    Yes, that looks fine - however there are limits on how much you can send by Bluetooth LE in one go - I believe 20 bytes at the moment - so you may need to send the data in several chunks, by just calling NRF.updateService from a setInterval.

    The only thing causing your data not to be freed is soundBuff=buf; - if you didn't have that I believe everything would be freed automatically.

    Actually seeing your data in nRF Connect is unlikely to be possible - while you can read the value of a characteristic, it's not set up to read repeated notifications - which is what you'd have to do to send the amount of data you want to send. You'd need your own app.

    The other option is actually to use the Bluetooth UART that's on by default. For instance you could write:

    function getData(clipSeconds) {
            var w = new Waveform(512*clipSeconds,{bits:8});
            w.on("finish", function(buf) {
              Bluetooth.write(buf);
            });
            w.startInput(D2,512);
          }
    

    And then writing "getData(2)\n" to the UART's RX characteristic would cause the TX characteristic to output all the data. You could view that on the Nordic UART app.

  • Hi @Gordon,

    Thank you for the advice! I installed "nRF UART 2.0" and tried

    var v;
    function g(s) {
      v=s;
      Bluetooth.write(s);
    }
    

    If you write without a semicolumn once you connected to the Puck it says "unexpected EOF". With a semicolumn I succeeded to light the LED on from here https://www.espruino.com/Puck.js+BLE+UAR­T

    If you write the function(with or without) a semicolumn in "nRF UART 2.0" or "nRF Connect" with notifications enabled I receive what I wrote, for example in "nRF UART 2.0" I have:

    TX:g(5);
    RX:g(5);

    TX:g(9)\n
    RX:g(9)\n

    Do you think this could be Android specific issue as appeared in the Security thread ?

    Thank you.

  • No, that's expected - it's because you're interacting with the JS interpreter, which echoes back what it receives. To disable that you can also send echo(0)\n.

    The easiest way is to send "\x10g(5)\n" - sending character code 16 on a blank line will stop Espruino from echoing just for that line. Obviously it's harder to do that (if not impossible) via nRF UART though!

    If you don't send a newline, the command won't be executed. Then when you do send a newline you'll get g(5);g(9) executed.

  • Hi @Gordon,

    You are right. The main issue was how to enter new line character, but that can be done typing "g(5)" followed by "shift + enter" for new line character and then press "send" button and the characteristics return the expected result.

    Thank you.

  • Post a reply
    • Bold
    • Italics
    • Link
    • Image
    • List
    • Quote
    • code
    • Preview
About

Recording and analyzing sound

Posted by Avatar for user73560 @user73560

Actions