433 mhz RF comms using external AVR MCU (advice on serial protocol needed)

We're all familiar with the challenges of using those el cheapo RF transmitters on the Espruino. Really, the problem is more the receiver than the transmitter - the Espruino just struggles to handle the noise fast enough to receive a signal. Listening continually is out of the question.

I'd already made AzzyRF, my terribad RF OOK protocol. Now that I'd been playing with ATtiny841's over in the Arduino world, I realized that the cheap, serial-capable chips could be used as an RF gateway for the Espruino (or other device).

It seems to work reasonably well - I can have it listen to the receiver, and forward any valid packets to serial, or it can respond to commands over serial and transmit them.

My question now concerns the serial protocol - what is a good format for this that would make sense?

Right now I've got a very crude system that just prints out successfully received packets as $(data)\r , while commands like S(4 bytes)\r, L(7 bytes)\r, etc send packets (AzzyRF has 4/8/16/32 byte packets), and C(27 bytes)\r to write new config data to the EEPROM. I expect that I'd have a few more config commands (to allow configuring more-frequently-changed options independently) and stuff. Does anyone have any recommendations on how to write a sensible protocol? I'm particularly interested in conventions that I should be following - is there a recommended command format? Newline conventions? Other stuff I should think about?

The sketch, and some trivial espruino js to enable sending arbitrary bytes down serial, are in my github:

https://github.com/SpenceKonde/AzzyProjects/tree/master/433mhz/txrxtoserial21

I'm going to have a few boards made that just have an '841 and headers for the RF modules + comms - I figure if nothing else, I'm planning to use my AzzyRF a lot more now that I've got all the infrastructure in place.

Sounds like a really good idea...

I'd suggest a (maybe optional) RTS/CTS implementation (it could always be removed by shorting the 2 wires together). It'd allow someone to use the setDeepSleep mode of Espruino, with a small bit of wrapper code:

setWatch(function(e) {
  digitalWrite(CTS, e.state);
  setDeepSleep(!e.state);
}, RTS, {repeat:true, edge:"both});

Just some ideas:

• Maybe go for AT commands? That seems pretty standard for serial
  
• Ciseco had their LLAP protocol. It's a similar idea, and might be worth a quick read just to see if they had any ideas
  
• As in LLAP, it might be worth making sure $(data) is actually just hex characters - it'll make it a lot safer (for instance how would you send \r otherwise).
  
  

Finally, I know this goes against your plan a bit, but I whether anything could be done in Espruino to make it better at decoding RF... Maybe I could add the ability to drop any state changes that happened too quickly, before they even got to JS.

AT commands are clearly the way to go but I don't know any of the things that I need to know to go forward here.

Where can I go for information about making an AT+ command set?

Like, what should I call the commands?
What are the conventions? What for newlines? What to do to signify a successfully received message?

What is LLAP?

What do you mean about hex characters? Oh shit, I can't just send the binary? This is increasingly becoming a huge fucking deal?

How do I get hex characters out of a serial stream and back into normal text? Do I need separators?

How does CTS/RTS work? I don't have pins, I spent hours designing a board and those pins, the tighest board I'd ever designed because I had so much other stuff on the board to fit but I used all the last pins for indicator leds,. How does CTR/RTS work? I would need to reroute everything, I guess I coudd do that, but I don't know how CTS/RTS work?

What sort of commands should I use for AT command set? I don't really know how AT commandset works, or what the guidelines for making it are? Is newline \r or \r\n or \n\r or \n? How do I send data that's a newline?

Man... I thought I understood what I was doing here---- I clearly have no clue :-(

Yeah, honestly I know as much as you about that. From my impression of ESP8266, you just take your protocol and put AT+ on the start of every command :)

I think:

• \r ends a line, but you probably want to cope with \r\n too
  
• OK\r returned on success, ERROR\r returned on failure
  
• When you're 'pushing' data back, prefix the line with +
  
  

What is LLAP?

It's just Ciseco's home-made protocol. It's worth following the link in my original post though - you might get some ideas.

Personally I'd just forget everything and do what's simple though. As long as it's easy for people to use it doesn't really matter if it's like any other protocol.

Oh shit, I can't just send the binary?

Well, you can. My guess though is in your receiver you'd do something like:

char buffer[256];
int bufLen;

void loop() {
 while (Serial1.available()) {
  char ch = Serial1.read();
  if (ch=='\r') { 
    buffer[bufLen++] = 0;
    executeCommand(buffer);
    bufLen=0;
  } else {
    buffer[bufLen++] = ch;
  }
 }
 // do other stuff
}

And then you're stuffed if you want to send a command likeHi\r\0... You can obviously work around it though - you just might find that sending/decoding hex is actually easier.

RTS/CTS

• RTS = ready to send
  
• CTS = clear to send
  
  

It's almost like an IRQ and a ready line. So:

• your receiver gets data and wants to send it to Espruino
  
• It raises the RTS line
  
• Espruino wakes up, gets its act together, and raises the CTS line
  
• Data is sent
  
• It lowers the RTS line
  
• Espruino lowers CTS and goes back to sleep if it wants to
  
  

edit: but it's not a big deal... Maybe you don't care about sleep power consumption - or you could even hack something up by sending a few characters you didn't care about (which could wake Espruino), then send the data you want later on.

Personally I'd just forget everything and do what's simple though. As long as it's easy for people to use it doesn't really matter if it's like any other protocol.

That's sounds similar as the whole (http/)HTML came to be... good principles to begin with, pragmatic easy to understand approach for each problem. Yes, it got us a bit into a mess - HTML 4.0 - HTML 4.1/XHTML 1.0, a dragging and finally failed XHTML 2.0. But HTML5 - like Phoenix out of the Ashes - kept it moving mightily. Yes, some is on the back of the Browsers which have to work even harder to keep accomodating all the legacy and do a good job on the new stuff...

Thanks. I think that inital response triggered a bit of an anxiety attack for me...

In AT commands, does either side typically send something to indicate that it's expecting something? (ie, master sends AT+SEND, and now is going to send some data - what would one expect that the device send back between those? Or should the master just send 'AT+SEND (data data data)'

Is punctuation or a delimiter normally done between, say, the end of AT+SEND and the beginning of the data being sent? Like AT+SENDdatadatadata or AT+SEND=datadatadata? etc...

Doing CTS/RTS would have some advantages, I think that's the way to do it. It will require significant reworking of my PCB to support though, as the board is already packed; I'm already very short on space. I hope I'll be able to get those two more pins on there without crippling the board in some other way.

Wait, couldn't that be done with 1 pin? Like, have a very weak pulldown on the flow control pin, then to signal that there was something to send, set it to INPUT_PULLUP, and then the other device could force the pin low to tell it to transmit, and let go after the response... ? Wouldn't conform to the standard though.

AT+SEND

If you google for AT+CIPSEND (the TCP/IP send command) you might get some ideas, but I'm not sure it's much of a standard :(

The way ESP8266 does it is (for single sockets):

• Send AT+CIPSEND=nBytes\r
  
• Wait for receive > (in reality you probably don't actually wait for that)
  
• Send the bytes
  
• Wait for receive SEND OK
  
  

So there's actually no need for hex encoding with that at all.

However for SIM900 GPRS here they use AT+CIPSEND without a length parameter, and it looks like you have to send Ctrl+Z to break out and actually send the data - I don't know how they escape it if you want to actually send Ctrl+Z though.

CTS/RTS... Wait, couldn't that be done with 1 pin?

Maybe... I guess once the pin is forced low it's hard to tell when the transmitter has then stopped trying to pull it high... You could do it a bit differently:

• Have an interrupt line
  
• Raise that when you have received data
  
• Espruino wakes up, and actually asks for the data that has been received (rather than being sent it).
  
  

Or you could even just raise the IRQ line 10ms before you start sending data - that'd be good enough too.

Well, I shoehorned those two traces in, so I've got CTS and RTS pins. Very tight board. Had to downsize everything to 0805, so it'll be a fun assembly job ;-) Getting ready for another order from dirtypcbs.

Okay, cool. I think I know what to do on the firmware side. This is very helpful.

Got this coming along well...

Firmware works, and I just fixed a bug with extended packets that had gone unaddressed for a long time (did you know that 32 bytes has 256 bits in it? Whoda thunk it - means a byte just wouldn't do it; Things you don't have to deal with on Espruino)

My commandset for sending will be:

AT+SEND
AT+SENDM
AT+SENDL
AT+SENDE
(for the 4 sizes)

Data returned is prefaced with a '+'

I've got it working with both binary, where the + is followed by a byte indicating the length of the bytes to follow, or where they're represented in hexadecimal and terminated with a \r.

CTS/RTS in a simple form should be easy to add. I sent out the boards to be fabbed by our friends in china yesterday. These, I think, will make the RF stuff a lot easier to do, since now I can easily bridge the divide between high level and low level devices - not to mention giving me an easy way to manually send data over it.

Also, strings in Arduino are hellish to work with compared to Espruino. Good god...

Sounds good - do you think: AT+SEND1/AT+SEND2/AT+SEND4/AT+SEND8 might be more informative for the data size?

Great news about the PCB - you're connecting to the existing cheap RF transmitters and receivers? It'll be a great little tool.

Just thinking, now you have the CTS/RTS pins as well, you could maybe have a mode where your module can be turned into a self-contained RF system... eg:

• send a message when an IO line goes high
  
• change state of IO lines in response to received messages
  
  

Did you see the discussion about encryption here?

The TEA cipher looks great, and should be trivial to add to your radio system if you wanted it...

strings in Arduino are hellish to work with compared to Espruino.

:)

Oh of course, if you wanted it to not convert to serial, and instead do stuff, that's just a matter of different firmware on the '841 - and the code is mostly written, would just need tweaking to the specific IO lines available.

Maybe. I don't like the idea of send 1/2/4/8, because that implies that the size will be 1, 2, 4, or 8 bytes, when that's not really the size.

It uses those cheapo little RF transmitter/receiver units. I ordered some of the good ones (good = crystal on receiver) - they cost about what a normal transmit/receive pair costs, but you only get the receiver. The normal receivers don't seem any better than the nice ones (which are also harder to get)

You could certainly encrypt the data before sending it - I've been watching that thread with interest, but not with this project in mind. The chip I'm using for the RF-Serial interface only has 8k of flash, and I've got less than 1k left after serial, send, and receive ;-) so there will be no on-board encryption.

If you have people monitoring the 433mhz band around your home with an interest to decode the data you're sending, I think you have a more fundamental issue to address.

If you have people monitoring the 433mhz band around your home with an interest to decode the data you're sending, I think you have a more fundamental issue to address

To be honest that is my feeling as well for things that I make, but if your radio became popular then it's actually very possible you may find people 'wardriving' trying to find homes that use it.

For instance it's trivial to write a bit of code that will turn all remote control sockets using this protocol on or off. If you got a powerful 433Mhz transmitter I reckon you could annoy a lot of people - I don't know what radius it'd cover, but I bet it's miles if you pushed a few watts out (illegally). In a city that would affect a lot of people.

And LightwaveRF that is being sold in a lot of retail stores in Europe has pretty bad problems too. While you can't really brute-force it (24 bit address), it's trivial to sniff - and you can then actually disable people's light switches!

...just like you could open the first remote control locked cars... Only serious encription with shared secret or private/public key would fix that... and we end up with more securing data in the 'payload' than functional data. Trying to avoid that 'inefficiency' will, I guess, always compromise encription/privacy.

The first dedicated AzzyRF gateway boards are here:

Despite my rather poor record with the protoboard, I seem to do pretty well with my designs working first try when there's actually something to them. Funny how that works.

Wow, nice - so it all works properly? The LED(s) on it look quite blinding :)

Somehow I didn't respond to this at the time. Yeah, I've got it working pretty well now. With defines in the code on the AzzyRF, I can build it to take commands in binary (ie, AT+SEND\r and the next 4 bytes sent are used raw) or hex (AT+SEND\r and then something like 1440FF02). I added a way (via #defines) to have it send an acknowledgement packet when receiving messages sent to it.

I've got one deployed in operation at my appartment, with plans to add more. It's going to be connected to the same Pico as the voice recognition unit, when the replacement arrives, and handle control of the RF devices that aren't handled by my desk light.

The LED's aren't actually that bright, I just keep my room fairly dim*. They're 0805 LEDs going through I think 1k resistors (most people like 220 ohm - I'd need sunglasses if I used the resistor values most people use)

Still thinking about how to do flow control (ie, to have it raise an IO pin to request to send). I don't think proper CTS/RTS is viable (it doesn't really do what you want here). I think instead, maybe we put the pin into INPUT_PULLUP on the AzzyRF side, and then wait for it to go low before sending anything, then put it back to INPUT when done. And if it's jumpered to ground, it'll send it immediately so you can easily disable that feature if you don't need.

*That works with people too - you may have noticed that you look brighter the dimmer the people around you are. I need all the help I can get, you see ;-)

Do you have any docs on your AzzyRF protocol (maybe Espruino code to transmit?). I'm actually planning on doing some demos with something on 433Mhz (using a PC to decode, and an Espruino to encode) and it'd be nice to publish the JS for that.

From the code I can find it looks like:

• 30x 0.2ms on, 0.2ms off training pulse
  
• 2ms delay (while off)
  
• Transmit bits MSB first, on for 0.55ms for 1, 0.3ms for 0, then off for 0.42ms
  
• Delay 2ms before a repeat
  
  

packet format is (lifted from your code):

The first byte of packets is:
| SZ1 | SZ0 | A5 | A4 | A3 | A2 | A1 | A0 |
A5~0: Address bits - 6 bit address of the device, compared to MyAddress. Only packets matching address are processed.
SZ1~SZ0: Size setting
SZ1=0, SZ0=0: 4 bytes
SZ1=0, SZ0=1: 8 bytes
SZ1=1, SZ0=0: 16 bytes
SZ1=1, SZ0=1: 32 bytes
The next two bytes go into MyCmd and MyParam, respectively, if the transmission is accepted (ie, device address matches and checksum is OK)
In a 4 byte packet, the first 4 bits of the last byte goes into MyExtParam (extended parameter), and the last 4 bits are the checksum.
For all longer packets, the whole fourth byte goes into MyExtParam, and the final byte is the checksum.
For 4 byte packets, checksum is calculated by XORing the first three bytes, then XORing the first 4 bits of the result with the last 4 bits, and the first 4 bits of the fourth byte.
For longer packets, checksum is calculated by XORing all bytes of the packet.

Yup, that's the protocol I used. I also have a slower one in the comments somewhere, but it actually didn't work as well with long batches of 0's or 1's with some receivers due to the AGC getting confused)

I think this is the working transmit code (and pre compiled-js receive code - I haven't gone back to try it out with "compiled") I used to test that:

https://github.com/SpenceKonde/AzzyProjects/blob/master/433mhz/espruino_RFtxrx21.js

I suspect it would now work much better with compiled JS, especially with a SYN470-based superhet receiver with data-squelching enabled, probably even well enough for practical use. I'm having some SYN470 boards made that will break out the pins used to set configuration parameters, and pads for a resistor if you want to squelch (what a fun word to say and type - and that's what the manufacturer calls i too). I think with 5meg squelch I was getting a single spurrious output every few seconds, instead of... a lot more without it, and TONS more, enough to hang the espruino, with the superregenerative receivers (which also have poor range).

Yes, I'll get hold of some of proper receivers - that should help a lot. Once I figure out how to get compiled code running in an irq working reliably, Espruino should be able to handle even the garbage coming in pretty well though.

I actually spent a while yesterday with this: https://github.com/gfwilliams/RadioReceiver

Trying to get it to decode AzzyRF. Not sure if I'm doing something wrong, but I had no luck at all - I guess it's possible it's just a bit too fast for the cheap receiver + audio input :(

If it helps I did this (rough) code for transmission from Espruino which seems to do about the right stuff - albeit without the actual packet formation:

function transmit(pin, data, callback) {
  var pulses = [];
  // 30 training pulses
  for (var i=0;i<30;i++)
    pulses.push(0.2,0.2);
  // 2ms delay off after training pulses
  pulses[pulses.length-1] += 2;
  // output data, MSB first
  data.forEach(function(byte) {
    for (var b=7;b>=0;b--) {
      // on for 0.55ms if 1, 0.3ms if 0, then off for 0.42ms
      pulses.push((byte&1)?0.55:0.3, 0.42);
      byte<<=1;
    }
  });
  // make sure we end on a low signal
  pulses.pop();
  // nowstart pulses
  digitalPulse(pin,1,pulses);
  // work out how long to wait for and call callback after  
  if (callback) {
    var msecs = E.sum(pulses)+2;
    setTimeout(callback, msecs);
  }
}