MIDI over (WiFi) TCP

Looking at the Midi support for Espruino, it appears as though the only ports you can send MIDI using this module is via Serialx. I want to send MIDI over EspruinoWifi module. Is there any support for that (yet)?

Sun 2019.08.04

Agree @indianajones, not much excitement on the MIDI topic yet.

Looking over:

http://www.espruino.com/Midi
http://www.espruino.com/BLE+MIDI
http://www.espruino.com/modules/ble_midi.js
and for WiFi
http://www.espruino.com/Interactive+Web+UI

couldn't a server be created, and mimic the same methods to communicate as in the source for the BLE version? Although I don't know of a well done example, others did help me a while back. Start at the last entry and work backwards with these examples to get an idea of what would be needed.

STA or AP mode browser unable to connect to ESP8266-12

Sidebar: Interesting you bring up MIDI. During the mid seventies, nearly a decade before Bill Gates would be on the radar and a household name, I cobbled together (a full 8 x 10in wire wrap project - no soldering) a sequencer with an 8088 and 8255 running commands written in assembler, along with whopping 2K memory chips to a Roland Juno-60, one of the last analog synths without MIDI, all over serial RS232! This was several years before MIDI would take hold. The Yamaha DX-7 comes to mind that exploited the MIDI benefits.

Wow, you predate me! I remember the iconic DX-7, but I went with the Roland D-50, which came out a few years later I believe. Still have it.

Wow, you predate me!

Is that the new 2019's way of being polite? ;-) Yeah, where did that forty years go?

You should see my 45's and album collection and rotary dial corded phone! The Juno-60 still works, along with that micro, but is packed away after my last move. Still play my Fender Rhodes Mark I suitcase piano.

What are your thoughts on creating a WiFi server to solve your issue?

I'm guessing you have the MIDI command set for the audio mixer? Shouldn't be that much of a diversion from the BLE example.

The target mixer manufacturer (Allen & Heath) published their MIDI spec. It uses NRPN, so minor additional complications, but it's all there.

Re: Wifi server, I don't know yet. Coming from the world of cloud-based software, which is my day job, I think of server as a separate entity with an API and all that. All I need is a listener for MIDI (over TCP) messages from the mixer, and a sender to send the messages. Maybe your definition of server is different from mine. Feel free to expound. I'm checking out the links you provided.

You have a suitcase Rhodes? I'm jealous! Hey you might like my son's music, he's the only kid I know making real music these days - the best of 60's and 70's sound. Check him out: https://open.spotify.com/artist/4ybii90Ofz5HRJyYoTLfiq

I'm certainly not qualified to provide a definitive answer as I only dabbled with that two years ago, but as it is possible to have the server running on EspruinoWiFi for instance, build and deploy web page content in the browser, listen to button click events and send data back-n-forth using query string name/value pairs, I don't see why it wouldn't be possible to perform similar tasks using send and recv functions in the same manner.

I was hoping others with more experience here could help out with links and ideas, but surprised no one has chimed in yet.

Will check out the 'real music' link tomorrow

Hi! As you say, for MIDI you just need a Serial transmit pin.

You're totally fine to use Espruino WiFi for that. One serial port (Serial2) is used for the Wifi, but the other one (Serial1) is totally free to use and is available on B6 - so you should be sorted.

More info on pins here: https://www.espruino.com/WiFi#pinout

Hope that's some help?

@Gordon wait, confusion still reigns, my fault not yours. Are you saying that Serial2 is the wifi connection on the Espruino wifi, by default? I don't understand how that would work. I have to send TCP packets over wifi that contains MIDI commands in them, because the target device is listening on TCP for MIDI data. If the target device was a serialport-listening device like normal MIDI, serial port makes sense. Please (try to) clear the fog in my brain. Thanks.

Currently, my plan is to roll my own MIDI implementation using require('Wifi') and require('net'). And by "roll my own" I mean "just enough to get it to work".

Ahh ok, so you're basically saying:

• There's some input device which is wired up to an Espruino WiFi (not via MIDI)
  
• You want the Espruino WiFi to create MIDI data based on the input from that device and then send it over WiFi?
  
  

The existing MIDI module only receives MIDI commands by the look of it, so if you're sending them you'll have to roll your own, but that's not too hard... And I assume that MIDI over TCP really is just sending the same raw MIDI data but over TCP not Serial?

All you need to do is follow the code at https://www.espruino.com/Internet#sockets to create a socket connection, then when you're connected you send the raw MIDI data with client.write(); - you just want to make sure you send the data as binary, so something like:

function sendNote(channel, on, pitch, velocity) {
  socket.write(E.toString([
    (on?0b10010000:0b10000000)|channel,
    pitch,
    velocity]))
}

Yes, MIDI over TCP is just the same MIDI command sequence, only over a TCP connection instead of serial connection. I'm sure there's some kind of "hey this is a MIDI packet" header or something, which I'm still researching, but the MIDI part is typical MIDI.

Thanks for the info, I'll start working on it. And I'd be happy to submit the module to the Espruino project when I'm finished.

And I'd be happy to submit the module to the Espruino project when I'm finished.

That'd be great - thanks! If your module were just to use .write (and assume a connection was already established) then nothing stops it from being used for Serial MIDI devices too, because Serial also has a write method.

@Gordon it looks like the MIDI device (Allen & Heath mixer) communicates using UDP, not TCP. I didn't see a way to specify protocol (TCP, UDP, etc) for the .net library, but I see the choice in NetworkJS. Unless you know something I don't.

Hmm, looks like there aren't specific docs on it - I should add that. UDP is handled in the same way as is done for Node.js.

As an example: https://github.com/espruino/Espruino/blob/master/tests/test_dgram_socket.js

let client = dgram.createSocket('udp4');
client.on('message', function(msg, info) {
  console.log(">"+JSON.stringify(msg));
  console.log(">"+JSON.stringify(info));
  // you probably don't care about this
});
// to send a midi note, you do:
client.send('your data', port, 'servername');

I'll give that a go, thanks!

Turns out I'm mostly wrong -- the mixer sends out a UDP packet containing only its name ('QU-SB'). Apparently that's merely for identification, not sure. But real communication happens on TCP. So back to TCP.

Saw this on Nordic's site: : MIDI over Bluetooth LE
Not Espruino specific, not over Wifi, but might contain some useful info

Thanks, @AkosLukacs, I'm sure there will be some helpful info in there.

@Gordon, to establish and maintain connection to my MIDI device over TCP, I have to send packets with various TCP flags set. In particular, I have to initiate with a SYN, and then respond to future requests with ACK. So it goes like this:

Me: SYN
MIDIDevice: PSH, ACK
Me: ACK
MIDIDevice: PSH,ACK
Me: ACK

That goes on forever. I assume this is just normal keep-alive stuff for TCP connections (I haven't played with TCP at this level for a while).

I got this data by using wireshark and watching how the A&H TCP MIDI driver connected to the mixer. The ACK back-and-forth is what goes on after they connect. When I connect with the Espruino Wifi, it connects and sits there. I assume because I have to send some establishing packet like the A&H driver did (the SYN packet).

Any thoughts on what I'm missing?

Are you sure that doesn't just happen automatically with TCP? Did you try basically just using the example I'd posted?

Exactly! Yeah I'm using your code, tweaked for connecting to the particular mixer, but pretty much the same. I prolly need to do some more digging before I publicly whine about it.

It should go like this:

Me: SYN
MIDIDevice: SYN, ACK
Me: ACK
MIDIDevice: PSH,ACK
Me: ACK

And having access to Wireshark you can look at the data sent.

@maze1980 yeah I forgot to add the data, which is super-important, and I've made it a little bit further because I forgot to check wireshark for broadcast packets. So the real A&H driver comms go like this:

Me: UDP 'Qu Find' on 255.255.255.255:51320
MidiDevice: UDP 'Qu-SB' and { address: 10.11.12.13, port: 51320, size: 6 }
Me: TCP connect to 10.11.12.13:51325 // note that 51325 is the TCP port
MidiDevice: TCP 0xEF // <- this is MIDI ActiveSense bit, sent every 300ms forever
Me: TCP ACK

So the data you were asking for is the 0xEF byte, which is what's sent (and ACK'd by 'me') forever. Here's how my Espruino code runs:

Me: UDP 'Qu Find' on 255.255.255.255:52320
(nothing happens, so instead...)
Me: UDP 'Qu Find' on 10.11.12.13:51320 // because I know the IP address already
MidiDevice: UDP 'Qu-SB' and { address: 10.11.12.13, port: 51320, size: 6 }
(cool! so far, so good)
Me: TCP connect to 10.11.12.13:51325
Me: (I get the 'connected' callback, and client.on('data', ...) is never called

So I'm doing something wrong still, because I don't get the 0xEF message ever.

Hey, why don't I also include my code?

		let udpClient = this.dgram.createSocket('udp4');
		udpClient = udpClient.bind(51320, function() {
			udpClient.on('message', function (msg, info) {
				console.log(">"+JSON.stringify(msg)); // <- here is where I get 'Qu-SB ...' from the mixer
				console.log(">"+JSON.stringify(info));

				let tcpClient = require('net');

				tcpClient.connect({port: 51325, host: "10.1.64.22"}, function() {
					console.log("connected");
				});
		
				tcpClient.on('data', function(chunk) {
					console.log("tcp data is " + JSON.stringify(chunk));
				});

				tcpClient.on('end', function() {
					console.log('client disconnected');
				  });
			});
			udpClient.send("QU Find", 0, 7, 51320, "10.1.64.22");
		});

10.1.64.22 is the Qu Mixer, which is the Midi device I'm trying to reach.

Here is a forum posting from an arduino project that is connecting to a Qu-16 (same series as mine), and I'm doing basically what he's doing: https://forum.arduino.cc/index.php?topic=230326.0

One more item...the blue LED on the Espruino wifi starts, and continues to, flash/flicker after I get the 'connected' message. So I might actually be getting the 0xEF message, but I'm never getting the tcpClient.on('data', ...) call.

And how do I send a message with tcpClient? tcpClient.write() and tcpClient.send() throw 'undefined' errors.

@Gordon, thoughts?