string to hex string /hex string to string conversion

Posted on
  • The Quectel BC95 module expects the data payload as hex strings. Are the following methods, the most memory efficient way of doing this?

    //convert to a hex string
        for (var i = 0; i < data.length; i++) {
          var s = data.charCodeAt(i).toString(16);
          while (s.length < 2) {
            s = '0' + s;
          }
          msg += s;
        }
    

    and conversely:

    for (let ndx = 0; ndx < (recvData[1] * 2); ndx += 2) {
        let i = parseInt("0x" + recvData[2].substring(ndx, ndx + 2)); //ascii hex to integer
        sockData[connection] += String.fromCharCode(i);          //build a string
      }
    
    

    From my limited understanding, this code would create a string, append and create a new string, rinse and repeat. This seems like a bit of overhead. I did consider typed arrays, but how do I convert these back to a string in one go?

  • How much data are you looking at sending? If you don't mind a bit of (short-term) memory overhead you can do:

    // to hex
    data.split("").map(x=>(256+x.charCodeAt(­)).toString(16).substr(-2)).join("")
    // from hex
    E.toString("01020304102030".match(/../g)­.map(x=>parseInt(x,16)))
    

    This should be reasonably fast, but it'll allocate an array so for anything over a few hundred bytes it could cause problems.

    String append is actually pretty fast & efficient in Espruino, so using your idea with a few of the hacks from above you might find this is better:

    // from hex
    for (var x of data)msg+=(256+x.charCodeAt()).toString(­16).substr(-2);
    // to hex
    var l = recvData[1]*2;
    var s="";
    for (var i=0;i<l;i+=2) s+=String.fromCharCode(data.substr(i,2))­;
    sockData[connection]+=s;
    

    It might be that you can configure the module to output data in binary though? That might be an easier way of working.

  • That code is like decoding a puzzle! I saw the 256 + bit and I'm thinking What!! Then I twigged that it is to ensure there's at least two hex chars. Thanks for the tips.

    Unfortunately the Quectel BC95 uses hex ascii. Why? Who knows. I can't think of any other AT module that does it that way.

    We were having some memory problems especially when receiving, so I had thought my code was not too efficient. I had it in my mind that there might be a simple way to use a byte array and cast it to a string. That way there's no copy and allocate.

  • I put together this implementation using a byte array without having to copy and allocate new strings.

    var buff = new Uint8Array(1024);
    
    //ascii string to byte array
    for (let i = 0; i < data.length; i++) {
          let c = ('0' + data.charCodeAt(i).toString(16)).slice(-­2);
          buff[i * 2] = c.charCodeAt(0);
          buff[i * 2 + 1] = c.charCodeAt(1);
    }
    
    //Ascii string from byte Array - Note 'apply' only works for 64 arguments
    function fromByteArray(len) {
          if (len < 64) return String.fromCharCode.apply(String, buff.slice(0,len));
          for (var result = [], i = 0; i < len; i += 64) {
                result.push(String.fromCharCode.apply(St­ring, buff.slice(i, i + 64)));
          }
          return result.join("").slice(0,len); 
    }
    
    //Hex string to byte array
    function toByteArray(data) {
          for (let i = 0; i < data.length/2; i++) {
                buff[i] = parseInt((data.substring(i*2,i*2+2)),16)­
          }
          buff.slice(0,data.length /2);
    }
    

    After running a process.memory() command before and after these changes, I can't see any significant memory savings. Should we expect any improvements in memory usage by using a byte array, or does the string append provide the best use of memory allocation?

    What improvements could we make to this implementation to improve memory allocation?

  • String append with += is definitely pretty efficient - as the string will be appended rather than copied to a new string each time.

    That's a great use of String.fromCharCode.apply in fromByteArray - however there is actually E.toString which allows you to do that in one big chunk (because there didn't seem to be a nice way in JS itself).

    @Kartman yes, the code is a bit cryptic! Unfortunately there's nothing built-in. Node.js has Buffer.toString(encoding) which would have been perfect, but we don't have Buffer in Espruino - just DataView which is depressingly similar, but without a .toString(encoding) option

  • Post a reply
    • Bold
    • Italics
    • Link
    • Image
    • List
    • Quote
    • code
    • Preview
About

string to hex string /hex string to string conversion

Posted by Avatar for Kartman @Kartman

Actions