Programming Espruino with Audio

Hi - this is something I showed off at FullStack and EclipseCon, but sadly the videos aren't online yet, so I did some more :)

Background

What if you want to control some hardware from a computer, phone or tablet? That's almost certainly going to involve developing an application per platform right now - and that's a lot of platforms: iOS, Android, Mac, Windows, Linux, Chromebook, and so on...

What if you could just use webpage instead? Web pages are much easier to make, and they run on almost everything...

Well, how could you communicate from a website? The obvious answer is via the internet, but WiFi on a microcontroller draws quite a lot of power, and is often quite expensive...

But if not that, what else? Not all devices have Bluetooth (and it can't be accessed from a website anyway), and while most devices have USB they all have different type sockets.

In fact the one socket that pretty much every computer has is a headphone jack - and it's easily accessible from a website without the need for drivers or special permissions.

What's it like:

https://www.youtube.com/watch?v=BQDZNFXygrM

https://www.youtube.com/watch?v=IP7e5KJGVnM

Note that there was no extra software on the Espruino Boards. The data coming out of the cable is interpreted directly by the USART as JavaScript code which is then executed.

Those videos show transmitting, but receive works too - to the point where there's a full terminal window just like the left-hand side of the Web IDE:

https://www.youtube.com/watch?v=3gWgrlbitiY

There's also integration with Eclipse Orion. See here for more details.

How?

It turns out it's relatively easy. To get a '1' on one of Espruino's inputs, you need at least 1.546v, and to get a '0', you need below 1.166v. You just need to shift the 'resting' voltage level coming out of the headphone jack to something that just about reads as a '1' (so more than 1.546v). This is done with nothing more than a capacitor and two resistors (one of which is built in to Espruino) - see the diagram at the bottom of this post.

As the headphone jack can output -1v to +1v (and all you need is 0.4v), you've got plenty of voltage swing available to make reliable communications. It's then just a matter of creating the right waveform.

As above, you can do similar things for receiving data too - everything you need is here: https://github.com/espruino/EspruinoOrion

How's it used?

To get one-way comms, just include the file:

<script src="https://espruino.github.io/EspruinoOrion/serial_to_audio.js"></script>

And then to send and execute code, do something like the following:

audio_serial_write("digitalWrite(LED1,1);\n"); 

Two directional communications is harder as the code for it is part of and depends on the EspruinoTools library on GitHub - so would be a little harder to extract and use on its own.

What's it useful for?

There are so many uses for this, for instance:

• Teachers can use low-cost Android tablets to teach programming and robotics - there's no software to install, no wireless pairing and no fragile connector - just sturdy a headphone jack.
  
• Maybe you want to read data back from a data logger at a remote location? It's easy to make a headphone plug waterproof too.
  
• Maybe you want a web-controlled hexapod that walks around holding your phone? Who doesn't? This would be perfect for that.
  
• You've might have made something and you just want a nice easy touchscreen UI to set it up - but don't want all that extra mass and cost connected to it all the time.
  
  

Will it work on non-Espruino devices?

Maybe, but not without having to write more software to run on the device you're connecting to. With Espruino you can just get it out of the packet, add the 2 components and headphone wires, and you're sorted!

If you've come to this site through a referral, please check out the KickStarter that I'm currently running for the tiny board that you see in the videos.

---

1 Attachment

• 

That is slick!

Can you expand on how the data is sent from the audio jack in the tablet? How do you generate the pho-audio that looks like UART to the Espruino from the text that's being sent?

@DrAzzy - this probably explains it better than I could: https://github.com/espruino/EspruinoOrion/blob/gh-pages/serial_to_audio.js

There's another one (just posted the video above) where I can do 2-way. The code for that is similar, but a bit more complex (and requires permission to use the mic).

Wow. That is really cool.

Very cool, well explained too

Massively cool!

Why is the light changing so terribly slow?

@Tobbe - it's because the two one-way examples are only sending data when required...

Basically, when nothing is playing, the voltage on the headphone jack is 0v - but we want it to be 1v, so we can go down to -1v and get a 2v swing. You can't just change it to 1v and start sending data, so it has to slowly ramp it up to 1v and wait for the capacitor in the circuit to charge. It could probably be done faster, but 1 second seemed like a nice round number.

If you wanted to speed it up, could you constantly play a 1V tone, so you could skip the ramping up before sending the actual data?

And why a 2V swing? I thought you only needed a .4V swing. Is it just to get it as reliable as possible?

@Tobbe - that's exactly what I do with the two-directional comms - hence why the characters appear pretty quickly. I left it out of the one-directional examples to make the code simpler.

And yes, just for reliability. 1V would probably be ok, but then a lot of devices turn the volume down when you plug headphones in, so getting the maximum output possible does help with that.

Also, as you send data, the capacitor discharges. If you transmitted 0V to -1V and were sending a 50% duty cycle wave then the capacitor would charge such that -0.5V went to the potential divider's voltage. That means you've then only got 0.5v left, which is too close to 0.4v for comfort.

Just to add - I put in a How's it used? section above, showing you how you could include it in your own pages.

Is there also an audio_serial_read() function? I didn't find one in the EspruinoTools lib...

I'm afraid not. You've basically got two methods of using audio:

One-way

Using audio_serial_write() - there's a bit of a delay when sending a command as it's got to wait for the capacitor to charge up, but you don't need any special permissions and it only uses CPU on your PC when sending.

Two-way

This obviously needs access to the mic, and so needs you to click 'ok' when the web browser prompts you - and it uses CPU on your PC all the time, because it's constantly analysing the waveform.

It uses this code - however it depends on stuff in 'EspruinoTools'. It could easily be extracted though, so I'd be interested in a pull request if anyone wanted to pull it out into a self-contained file...

Is the resistor on-board a pull-up to 1V? The circuit is a little misleading -- it looks like you are capacitively coupling the signal to the board, and then... how do you set the bias to 1V?

This is very cool. Did you consider doing something similar with a photo sensitive diode held up to an area of the screen? If that was built onto the pico it would require even less hardware and you would have your device independence. You would have to have a good "key" pattern to enable transfer.

<script src="https://espruino.github.io/Espruino­Orion/serial_to_audio.js"></script>

Something is very strange about that link. If you copy-paste it (at least in chrome), the URL gets changed to https://espruino.github.io/Espruino­-Orion/serial_to_audio.js when you paste it into the address bar (of course, that address doesn't work). But if you paste it into anything else, it comes out right! I don't understand why this is. Does anyone else?

@user49986 there's a 40k pull-up (to 3.3v) inside the chip (which is enabled by default). With the external 47k pull-down it manages to bias the voltage to 1.8v (ish).

It has the nice effect of pulling the signal to 3.3v (the 'idle' state for the UART) when the cable is not connected. If you were using some other non-espruino device that didn't have switchable internal resistors then you'd have to add an external pullup.

@user50001 I'd wondered about that. The problem is that the screen only really refreshes at max 60Hz, so it would be very slow to transfer data. This solution actually works without needing any special software on the host you're communicating with. As long as the UART is set to 9600 baud it'll work.

@DrAzzy I'm not sure - I don't seem to have problems here.

"The problem is that the screen only really refreshes at max 60Hz, so it would be very slow to transfer data."

I can see that :)
I totally accepted that for a day. Wouldn't be the end of the world just tape the pico to the screen provided it was reliable - but another thought hit me too (ignoring the fact that the board is incredibly tightly packed already and may have all available i/o pins utilized). It is just such a fundamental way people can so easily interact with the board right out of the box in a way that they can simply manipulate and reliably validate. Kind of a shared basic sense. Just a consideration of which I'm sure are bombarded with. Might make a good automated production test too.

http://www.digikey.com/product-detail/en/HSDL-5420%23011/160-1918-6-ND/3711429

Disclaimer I don't work for digikey or liteon and the pico is brilliantly cool non the less.

Well, it turns out that LEDs themselves can be used to detect light: http://makezine.com/projects/make-36-boards/how-to-use-leds-to-detect-light/

The wavelengths are narrow but it might be enough. I'll see if I can make one of the pins that's connected to an LED also be one with an ADC - not sure how easy that will be though.

I think it would be good if possible. I'm just thinking of enabling some neat experiments with simple code that people would be able to do with it straight away. I always find the more success new users see up front the more they are willing to endure as the learning curve steepens. Congrats on the kickstarter.

Hi there !

Just a quick update on this post:

Here's a project by Adam Kumpf I stumbled upon a while back now, which I think may be still quite of interest:
Even if it only work for "slow" baudrates, I was wondering if we could achieve that using only an Espruino board ( without any external component, aside from the electret mike ( with or without a breakout ) )
http://research.kumpf.cc/2015-ProgrammingWithHeadphones/

I also don't know if a DTMF tone(s) decoder is easily doable using the waveform class ..

Last but not least, being able to replace the jack plug with either a mike, a bluetooth audio receiver, a SIM breakout spk line or an intercom spk line 'd be quite cool ( ex: playing a waveform, either a pre-recorded one or directly from the WebAudioApi, to an intercomm's mike & getting the door opened, .. ) :)

so .. WaveformSerial module anyone ? ;)

From Bluetooth audio or GSM you'd probably be able to connect direct using the same arrangement. Also I'd have thought you could decode a sine wave or silence using just a diode + capacitor?

You could do some kind of decoding in software, but you'd almost certainly want that to happen in C code rather than JS... It would also pretty much destroy any hopes you might have for power efficiency :)

Can I buy this cable somewhere? I don't think I have the skill 0r knowledge to build one myself.

This maybe ?
http://m.ebay.com/itm/191680748939?_mwBanner=1

Will it float?

Otherwise is there a shoppinglist for ebay with the right components? So i can try putting it together?

There's now a page on doing this here: http://www.espruino.com/Headphone

Especially if you only want to send data rather than receive as well, you only need two components. Which country are you in? There is probably some local electrical distributor you can get the parts from - and for the plug itself you can usually dismantle an old set of headphones

@Gordon Hello again. Im in Sweden. I want to send and receive but maybe I can get supplies for them both and start with the simple one. I have several headphones. From what I can gather from the videos you have ordinary jumper cables braided not the audio cable just the a audio plug? Then att the board-end you have the resistor/resistors and the capacitator hooked up?