MQTT Client and longer messages

I've been doing some experiments with MQTT but I'm having an issue receiving messages with a longer payload, ideally 320bytes of binary or 468bytes base64 encoded.
I wrote a tests and it seems like the mqtt client does unpredictable things over ~100 chars in the payload.

This is the test client on a pixl running 2v13, with an ESP8266 connected for WiFi

var server = "192.168.1.10"; 
var mqtt = require("tinyMQTT").create(server);
mqtt.on('connected', function() {
      console.log("MQTT Connected\n");
      mqtt.subscribe("/test");
});

mqtt.on('message', function (msg) {
    console.log(JSON.stringify(msg));
});


digitalWrite(D11,1); // enable ESP8266
  Serial1.setup(115200, { rx: D12, tx : D10 });
  var wifi = require("ESP8266WiFi_0v25").connect(Seri­al1, function(err) {
    if (err) throw err;
    console.log("Connecting to WiFi");
    wifi.connect(SSID, PASSWORD, function(err) {
      if (err) throw err;
      console.log("WiFi Connected");
      mqtt.connect();
    });
});

My test source publishing a message once a second where the first characters are a number then its padded with that number of dots eg 5....

Result is in the attached file, as you can see it starts to loose parts of the payload around 100chars and at 120chars the topic becomes corrupted.
These points seemed to vary a little with different runs, and the length of the topic so I suspect its something in the overall size of the message thats the limiting factor.

According to the MQTT spec it should support messages up to 256mb! while I realise thats totally crazy for Espruino it would be nice if it was able to handle a biut more. even 1kb would be enough for my needs

1 Attachment

• mqtt.log

Well, tinyMQTT can not handle messages larger than 127 bytes.

https://github.com/MaBecker/tinyMQTT

Doh! just noticed I'm using tinyMQTT rather than the regular client that I was looking at the docs for, that will teach me to copy code from another project

:) does it work better using the normal MQTT library?

kind of!
I'm now getting FIFO full errors which I think is down to the interface between the nrf52 and the ESP8266,

Trying with espruinio on an ESP32 seems to work much better although I'm now into issues with the waveform class and that.

Argh, that's a pain. On the Espruino Wifi we enabled flow control (https://github.com/espruino/EspruinoDocs­/blob/master/devices/EspruinoWiFi.js#L25­2-L288)... You could set up CTS:

Serial1.setup(115200, { rx: ..., tx : ..., cts : WIFI_CTS_PIN });
/// and enable (after WiFi inited) with
at.cmd('AT+UART_CUR=115200,8,1,0,2\r\n',­500,function(d) { // enable flow control
                  if (d!="OK") throw new Error("UART_CUR failed: "+(d?d:"Timeout"));
                  // do the next thing now
                });

Trying with espruinio on an ESP32 seems to work much better although I'm now into issues with the waveform class

Hmm, yes, I'm not sure how well that works on ESP32...

Waveform, just checked the list (https://github.com/espruino/Espruino/iss­ues/1777) of open ESP32 issues. If it's broken please share some info to be added to the list of issues.

@MaBe I've just tested with the sample output code on https://www.espruino.com/Waveform and that plays back fine on an ESP32 so looks like thats ok its just my encoding that I need to look at

@Gordon is there any more detail on the audio encoding for waveform, at the moment I have PCM frames with 16bit 8KHz samples, little endian
But trying to play these back with waveform produces noise

I can't remember about the exact 16 bit encoding, but I'd strongly recommend just using 8 bits - it's not like you'll get anything over 8 bits worth of quality using PWM as a DAC :)

For 8 bits I seem to recall I had to export from Audacity as 'unsigned' or something like that. Basically you want a value from 0..255, not -128..127. You could always just add 128 to the numbers in Espruino when you load the buffer, but if you can it's worth getting the data right at the start.