Espruino with Glediator & LED matrix

The 'out of memory' error sounds like you're not getting a newline, so are just making a huge string with all the data from multiple frames?

Actually, if you look at that Arduino file you sent, it seems to me like it's sending binary data and is using a simple 1 as the 'new frame' character (line 110) - presumably it ensures that it just never sends '1' normally.

Try:

function onInit() {
    SPI2.setup({baud:3200000, mosi:B15});    
    var cmd = "";
    Serial3.setup(1000000, {tx:C10,rx:C11});
    Serial3.on('data', function (data) {
      cmd+=data;
      var idx = cmd.indexOf("\1");
      while (idx>=0) {
        var line = cmd.substr(0,idx);
        cmd = cmd.substr(idx+1);
        idx = cmd.indexOf("\1");
        SPI2.send4bit(line, 0b0001, 0b0011);    
      }
    });
  }
  onInit();