GPRS or Edge Howto?

I have reproducable problems again with the Pico. [CTRL+C] echo(1) doesn't help here. I just get it working when disconnecting it from USB and from the power source. It happens in the sim900 driver when the method init is getting called.

I currently have the problem very often using other code but I was not able to reproduce it.

This is the output (after that it hangs):

 _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v79.19 Copyright 2015 G.Williams
>echo(0);
=undefined
Inititialize sim900 module
> 

As well the Pico sends some weird characters sometimes when connecting it to USB (no init function has been set which could produce that):

Connected
>à ‘Ö—2|

Regarding the HTTP Status code: I can see that some of the data is not getting passed correctly to the attributes of the Response-Object. See here:

Response:  {
  "headers": {
    "Date": "Mon, 08 Jun 2015 12:09:31 GMT",
    "Server": "Apache/2.2.14 (Ubuntu)",
    "Last-Modified": "Fri, 15 Nov 2013 15:42:26 GMT",
    "ETag": "\"c01036-d-4eb390b8a8d18\"",
    "Accept-Ranges": "bytes",
    "Content-Length": "13",
    "Vary": "Accept-Encoding",
    "Connection": "close",
    "Content-Type": "text/plain"
   },
  "httpVersion": "P/1.1 200 OK\r\nDate: Mon, 08 Jun 2015 12:09:31 GMT\r\nServer: Apache/2.2.14 (Ubuntu)\r\nLast-Modified: Fri, 15 Nov 2013 15:42:26 GMT\r\nETag: \"c01036-d-4eb390b8a8d18\"\r\nAccept-Ran­ges: bytes\r\nContent-Length: 13\r\nVary: Accept-Encoding\r\nConnection: close\r\nContent-Type: text/plain\r\n\r\nHello World",
  "statusCode": "\nHTTP/1.1",
  "statusMessage": "200 OK\r\nDate: Mon, 08 Jun 2015 12:09:31 GMT\r\nServer: Apache/2.2.14 (Ubuntu)\r\nLast-Modified: Fri, 15 Nov 2013 15:42:26 GMT\r\nETag: \"c01036-d-4eb390b8a8d18\"\r\nAccept-Ran­ges: bytes\r\nContent-Length: 13\r\nVary: Accept-Encoding\r\nConnection: close\r\nContent-Type: text/plain\r\n\r\nHello World"
 }

edit: sorry - I didn't see these extra posts until my response appeared. I'll take a look tomorrow.

Thank you. I attached the most recent version of the sim900 lib.

Regarding the problem that Espruino hangs do you think that has something todo with Serial1 as discussed here?

http://forum.espruino.com/conversations/­255152/#comment12325361

I experienced that when I first connect the pico to USB and then start the sim900 module the problem doesn't exist. I'll try to set USB.setConsole() in onInit tomorrow and check if that solves the problem.

1 Attachment

• sim900a.js

I see in there references to username and password - I'm confused, I thought that the SIM900 only required the SIM card. How do you get the APM, username and password?

I'm planning to use one of these once my plan expires so I can get my parents and I onto a family plan so I can get SIMs for less than $40-60/month.

The apn, username and password depends on the service provider you are using. In the most cases these are the same for all users and you can find it on the support pages of the providers. Here is a list with some apn settings:

http://www.hw-group.com/products/HWg-Are­s/HWg-Ares_GSM_APN_en.html

Your hanging could be something to do with the console moving to Serial1 I guess. Have you tried forcing it to USB in onInit?

If some code hung, the the input buffer could get full from the responses from the SIM900, which might mean that Ctrl+C no longer worked. It seems strange though.

The errors you were getting with the newline at the start of the HTTP request - do they also happen on the very first HTTP request you make? I have a hunch that it might be rubbish that's left over from previous HTTP requests.

Hi Gordon,

the USB.setConsole() in onInit seems to solve the problem. I can't reproduce it now.

Yes the problems with the HTTP request happen in the very first request as well. It's not just the newline. There should be the status code instead of the HTTP version.

Best,

Tobias

Let me know when I should test something again regarding the HTTP problem.

There should be the status code instead of the HTTP version.

Yes, but that's probably a result of the newline at the start - I think the code that splits the HTTP response up got confused.

Do you think you could do a request with sim.at.debug() called, and attach the result so I can take a look at what gets sent and can then run it through the code here? Sorry, but it's a pain to do here without working hardware to try it on.

Sure. Here is the output:

 _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v79.19 Copyright 2015 G.Williams
>echo(0);
Connecting to sim900a module
["ATE0\r\n"
=undefined
] "A" <--- "A"
] "ATE0\r\n\r\nOK\r\n" <--- "TE0\r\n\r\nOK\r\n"
["AT+CPIN?\r\n"
] "\r" <--- "\r"
] "\n+CPIN: READY\r\n\r\nOK\r\n" <--- "\n+CPIN: READY\r\n\r\nOK\r\n"
["AT+CGATT?\r\n"
] "\r" <--- "\r"
] "\n+CGATT: 1\r\n\r\nOK\r\n" <--- "\n+CGATT: 1\r\n\r\nOK\r\n"
["AT+CIPSHUT\r\n"
] "\r" <--- "\r"
] "\nSHUT OK\r\n" <--- "\nSHUT OK\r\n"
["AT+CIPSTATUS\r\n"
] "\r" <--- "\r"
] "\nOK\r\n\r\nSTATE: IP INITIAL\r\n" <--- "\nOK\r\n\r\nSTATE: IP INITIAL\r\n"
["AT+CIPMUX=1\r\n"
] "\r" <--- "\r"
] "\nOK\r\n" <--- "\nOK\r\n"
["AT+CIPHEAD=1\r\n"
] "\r" <--- "\r"
] "\nOK\r\n" <--- "\nOK\r\n"
["AT+CSTT=\"pinternet.interkom.de\", \"\", \"\"\r\n"
] "\r" <--- "\r"
] "\nOK\r\n" <--- "\nOK\r\n"
["AT+CIICR\r\n"
] "\r" <--- "\r"
] "\nOK\r\n" <--- "\nOK\r\n"
null
["AT+CIFSR\r\n"
] "\r" <--- "\r"
] "\n10.47.223.198\r\n" <--- "\n10.47.223.198\r\n"
IP:10.47.223.198
["AT+CIPSTART=0,\"TCP\",\"http://www.pur­3.co.uk\",80\r\n"
] "\r\nOK\r\n" <--- "\r\nOK\r\n"
] "\r\n0, CONNE" <--- "\r\n0, CONNE"
] "0, CONNECT OK\r\n" <--- "CT OK\r\n"
["AT+CIPSEND=0,64\r\n"
] "\r\n> " <--- "\r\n> "
] "\r\n0" <--- "\r\n0"
] "0, SEND OK\r\n" <--- ", SEND OK\r\n"
["AT+CIPSEND=0,34\r\n"
] "\r\n> " <--- "\r\n> "
] "\r\n0, SEN" <--- "\r\n0, SEN"
] "0, SEND OK\r\n" <--- "D OK\r\n"
] "\r\n+RECEIVE" <--- "\r\n+RECEIVE"
] "+RECEIVE,0,289:\r\nHTTP/1.1 200 OK\r\nDate: Tue, 09 Jun 2015 11:07:06" <--- ",0,289:\r\nHTTP/1.1 200 OK\r\nDate: Tue, 09 Jun 2015 11:07:06"
] "+RECEIVE,0,239: GMT\r\nServer: Apache/2.2.14 (Ubuntu)\r\nLast-Modified: Fri, 15 Nov 2013 15:42:26 GMT" <--- " GMT\r\nServer: Apache/2.2.14 (Ubuntu)\r\nLast-Modified: Fri, 15 Nov 2013 15:42:26 GMT"
] "+RECEIVE,0,157:\r\nETag: \"c01036-d-4eb390b8a8d18\"\r\nAccept-Ran­ges: bytes\r\nContent-Length:" <--- "\r\nETag: \"c01036-d-4eb390b8a8d18\"\r\nAccept-Ran­ges: bytes\r\nContent-Length:"
] "+RECEIVE,0,86: 13\r\nVary: Accept-Encoding\r\nConnection: close\r\nContent-Type: text/plain\r\n\r\nHello World!\n" <--- " 13\r\nVary: Accept-Encoding\r\nConnection: close\r\nContent-Type: text/plain\r\n\r\nHello World!\n"
Response:  {
  "headers": {
    "Date": "Tue, 09 Jun 2015 11:07:06 GMT",
    "Server": "Apache/2.2.14 (Ubuntu)",
    "Last-Modified": "Fri, 15 Nov 2013 15:42:26 GMT",
    "ETag": "\"c01036-d-4eb390b8a8d18\"",
    "Accept-Ranges": "bytes",
    "Content-Length": "13",
    "Vary": "Accept-Encoding",
    "Connection": "close",
    "Content-Type": "text/plain"
   },
  "httpVersion": "P/1.1 200 OK\r\nDate: Tue, 09 Jun 2015 11:07:06 GMT\r\nServer: Apache/2.2.14 (Ubuntu)\r\nLast-Modified: Fri, 15 Nov 2013 15:42:26 GMT\r\nETag: \"c01036-d-4eb390b8a8d18\"\r\nAccept-Ran­ges: bytes\r\nContent-Length: 13\r\nVary: Accept-Encoding\r\nConnection: close\r\nContent-Type: text/plain\r\n\r\nHello World",
  "statusCode": "\nHTTP/1.1",
  "statusMessage": "200 OK\r\nDate: Tue, 09 Jun 2015 11:07:06 GMT\r\nServer: Apache/2.2.14 (Ubuntu)\r\nLast-Modified: Fri, 15 Nov 2013 15:42:26 GMT\r\nETag: \"c01036-d-4eb390b8a8d18\"\r\nAccept-Ran­ges: bytes\r\nContent-Length: 13\r\nVary: Accept-Encoding\r\nConnection: close\r\nContent-Type: text/plain\r\n\r\nHello World"
 }
--->Hello World
] "!\n\r\n" <--- "\r\n"
] "0, CLOSED\r\n" <--- "0, CLOSED\r\n"
> 

I can see there that the last character of the HTTP-Body as well is not parsed correctly. The correct string is "Hello World!"

Ok, thanks! that's really handy. So it's actually not the GSM code at all, it's the SIM900 that's sending the extra newlines:

"\r\n+RECEIVE"
",0,289:\r\nHTTP/1.1 200 OK\r\nDate: Tue, 09 Jun 2015 11:07:06"
//      ^^^^

Now I can't find any info on +RECEIVE at all, so I don't know whether it's actually meant to do that after every RECEIVE or not... Maybe if you request a bigger file (>1.5kb?) we'll get two RECEIVEs, and can check... But for now let's assume that it always goes: +RECEIVE,X,Y:\r\n<data>

I've had to add a second handler called receiveHandler2, because unfortunately the newline after the +RECEIVE would totally confuse the AT handler if we left it in the buffer.

function receiveHandler(line) {
  var colon = line.indexOf(":\r\n");
  if (colon<0) return line; // not enough data here at the moment
  var parms = line.substring(9,colon).split(",");
  parms[1] = 0|parms[1];
  var len = line.length-(colon+3);
  if (len>=parms[1]) {
   // we have everything
   sockData[parms[0]] += line.substr(colon+1,parms[1]);
   return line.substr(colon+parms[1]+3); // return anything else
  } else { 
   // still some to get
   sockData[parms[0]] += line.substr(colon+3,len);
   return "+D,"+parms[0]+","+(parms[1]-len)+":"; // return +D so receiveHandler2 gets called next time    
  }
}

function receiveHandler2(line) {
  var colon = line.indexOf(":");
  if (colon<0) return line; // not enough data here at the moment
  var parms = line.substring(3,colon).split(",");
  parms[1] = 0|parms[1];
  var len = line.length-(colon+1);
  if (len>=parms[1]) {
   // we have everything
   sockData[parms[0]] += line.substr(colon+1,parms[1]);
   return line.substr(colon+parms[1]+1); // return anything else
  } else { 
   // still some to get
   sockData[parms[0]] += line.substr(colon+1,len);
   return "+D,"+parms[0]+","+(parms[1]-len)+":"; // return +D so receiveHandler2 gets called next time    
  }
}

// ....

exports.connect = function(usart, resetPin, connectedCallback) {
  rst = resetPin;
  gprsFuncs.at = at = require("AT").connect(usart);
  require("NetworkJS").create(netCallbacks­);
  at.register("+RECEIVE", receiveHandler);
  at.register("+D", receiveHandler2);
  gprsFuncs.reset(connectedCallback);
  return gprsFuncs;
};

That works!!!! Great! I tested it with small and big files and it seems to send that \r\n after each +RECEIVE

Thank you so much Gordon!

This is the entire driver which is working for me. If you like you can publish it as official module.

1 Attachment

• sim900a.js

Wow, great - thanks! Yes, I'll try and stick it online as a module.

Hello Gordon,

I have a problem with the use of at.register('>') in the following code of the GPRS driver:

  send: function(sckt, data) {
    if (at.isBusy() || socks[sckt]=="Wait") return 0;
    if (!socks[sckt]) return -1; // error - close it
    var f = function(d) {
      at.unregister('>');
      if (d != sckt + ", SEND OK") return;
    };
    at.register('>', function() {
      at.write(data);
      return "";
    });
    at.cmd('AT+CIPSEND='+sckt+','+data.lengt­h+'\r\n', 10000, f);
    return data.length;
  }

How can it handle multiple sockets with it? In the AT lib I can just register for some chars once. The next call of the send function for another (or for the same) socket tries to register for '>' again or sends a new AT+CIPSEND without waiting for ">" which leads the communication to an inconsistent state. I tried to set the socket to "Wait" until ">" comes in for the particular socket, but requests for other sockets were still able to send.

In the ESP8266 driver the problem was not existing because you send the data directly after AT+CIPSEND, which is - unfortunately - not working with the SIM900.

Do you have an idea how to solve the problem?

Best,

Tobias

No, it is a problem on ESP8266 as well; the moment more than one connection is open at once, everything hits the fan.

The ESP8266 problem is because of their stupid use of Linked/Unlinked with no connection number. The SIM900 shouldn't have that.

I guess the problem here is that at.isBusy() is reporting false because it's not actually busy, just waiting for >.

Why not add your own variable to track whether you're waiting or not?

send: function(sckt, data) {
    if (at.isBusy() || this.sendWaiting || socks[sckt]=="Wait") return 0;
    if (!socks[sckt]) return -1; // error - close it
    this.sendWaiting = true;
    var f = function(d) {
      at.unregister('>');
      this.sendWaiting = false;
      if (d != sckt + ", SEND OK") return; // <------- this doesn't seem to do anything?
    };
    at.register('>', function() {
      at.write(data);
      return "";
    });
    at.cmd('AT+CIPSEND='+sckt+','+data.lengt­­h+'\r\n', 10000, f);
    return data.length;
  }

Hello Gordon,

I tried to solve it like this yesterday night:

  /* Send data. Returns the number of bytes sent - 0 is ok.
  Less than 0  */
  send: function(sckt, data) {
    if (busy || at.isBusy() || socks[sckt]=="Wait") return 0;
    if (!socks[sckt]) return -1; // error - close it
    busy = true;
    at.register('>', function() {
      console.log('READY');
      at.unregister('>');
      at.write(data);
      return "";
    });
    at.registerLine(sckt + ', SEND OK', function() {
      at.unregisterLine(sckt + ', SEND OK');
      busy = false;
      return "";
    });
    at.cmd('AT+CIPSEND='+sckt+','+data.lengt­h+'\r\n');
    at.debug();
    return data.length;
  }
};

But for some reason I ran into a deadlock situation where busy was set to true but no SEND OK came in to remove it in some cases.

I will try your solution without waiting for SEND OK and give you a feedback.

Okay we have to wait for SEND OK, otherwise data can be lost. My solutions from yesterday works now. I saw that the LOW_MEMORY flag was set during execution. I compressed the code now and it seems to work stable.

Here you find the current version of the driver.

1 Attachment

• sim900a_community.js

It might be worth adding a timeout - Maybe just use the callback argument of at.cmd instead of the registerLine that you added? That includes a timeout if nothing is received in a given time period, so you should never be able to get in a deadlock.

Please see the post above.

Okay, If I experience further problems with it, I will add a timeout.

Thanks - I posted just a few secs after you, so didn't realise you'd sorted it out :)

Hi Gordon... It is annoying... Again problems with GPRS. When the sim900 is ready for receiving data after AT+CIPSEND it sends:

] "\r\n> "

When I use the following code:

at.register('> ', function() {
  at.unregister('> ');
  at.write(data);
  return "";
});

It removes the "> " from the AT buffer but leaves "\r\n" in it which can cause problems for subsequent requests (maybe for the +RECEIVE handler as well). I tried as well to register for "\r\n> ". That didn't solve that for me. How would you handle that? There are as well situations like

] "\r" <--- "\r"
] "\n> " <--- "\n> "

or

["AT+CIPSEND=1,26\r\n"
] "\r\n>" <--- "\r\n>"
] "> " <--- " "