iPhone notifications for Bangle.js

Posted on
  • iPhone notifications for Bangle.js

    I have been working on an app to get Bangle.js to display notifications from an iPhone such as calls, emails etc on the Bangle using Apple's ANCS - Apple Notification Center Service. I have made some progress and hit an obstacle which I outline in the following in the hope that someone can suggest a way forward - other than buying an Android phone:-)

    The first step is to get the iPhone to see the Bangle.js in Settings for bluetooth. To do this the advertising packet from Bangle.js needs to contain the UUID of the solicited ANCS service. This is published by Apple as:- 7905f431-b5ce-4e33-a455-4b1e122d00d0.

    The following code achieves this and allows the iPhone to connect to the watch.

    Bangle.setLCDTimeout(0);
    g.clear();
    Terminal.println("Starting Advertising");
    
    NRF.setAdvertising([
      0x02, //length
      0x01, //flags
      0x06, //
      0x11, //length
      0x15, //solicited Service type code
      0xD0,0x00,0x2D,0x12,0x1E,0x4B, //UUID
      0x0F,0xA4,
      0x99,0x4E,
      0xCE,0xB5,
      0x31,0xF4,0x05,0x79],{});
    
    NRF.setSecurity({passkey:"123456",mitm:1­,display:1});
    
    NRF.on('connect',function(addr){
      Terminal.println("connect from ");
      Terminal.println(addr);
      Terminal.println("----------------------­");
      var sec = NRF.getSecurityStatus();
      Terminal.println("connected: "+sec.connected);
      Terminal.println("encrypted: "+sec.encrypted);
      Terminal.println("mitm_protected: "+sec.mitm_protected);
      Terminal.println("bonded: "+sec.bonded);
    });
    
    NRF.on('disconnect',function(reason){
      Terminal.print("disconnect ");
      Terminal.println(reason);
    });
    
    

    When connected the watch can display the iPhone's bluetooth address for the ANCS service. As can be seen from the screen photo this is a private resolvable address. Note that the watch does not start advertising until it is disconnected from the Web IDE - the first disconnect in the photo.

    Now for the problem. Bluetooth specifies both that a device may be Master/Central or Peripheral/Slave as well as whether it is a Client or Server. The Espruino examples all have the Master as Client and the Slave as Server. The Master initiates the connection and the Client accesses the data provided by the Server. The problem is that the ANCS has the Master as Server and the Client as Slave. In other words, the iPhone initiates the connection and also provide the ANCS service. The watch is the Slave which responds to the connection and then accesses the ANCS service as a client. The Espruino API as far as I can ascertain does not have a way to let you create a BluetoothDevice and then a BluetoothRemoteGATTServer to access the remote server for this Slave/Peripheral - Client combination. I tried an NRF.connect() to the private address, however, this immediately drops the connection.

    The Espruino Bluetooth API is a joy when compared with the complexity of the Nordic or Arduino APIs, so I do hope that I am wrong and there is a way of doing this or that it can be easily extended.

  • In other words, the iPhone initiates the connection and also provide the ANCS service.

    Wow, you're totally sure about this?

    This is the first time I've come across any device that actually does this. Right now Bangle.js/Espruino can't do it without firmware mods :(

    There's potentially a hack where you set m_central_conn_handle to m_peripheral_conn_handle in the firmware, but I think it'd be worth handling properly. I've just filed an issue for it here: https://github.com/espruino/Espruino/iss­ues/1800

  • Thanks for the quick response.

    Yes, I am reasonably certain about this. I have had a look at the Arduino ESP32 implementation of ANCS access and in fact, I have traced its connection protocol with NRf Sniffer and Wireshark and it is pretty clear that the iPhone has both the Master/Central role and that it also provides the service.

    Someone tried to do this for a Puck two years ago - http://forum.espruino.com/conversations/­314651/ - and I am sure this and the fact that you have to advertise the solicited Service UUID is why they failed.

    Resolution of this along the lines you suggest in the issue would be very elegant indeed. I have managed to build the firmware from source, so I may try to hack something in the interim.

  • I implemented the hack suggested by Gordon in the firmware for a Puck.js and it does work, however, after some experimentation, I have realised that while interesting - it is not necessary!

    Essentially, after getting the address of the iPhone using NRF.on("connect",function(addr){}), you can get a connected BluetoothRemoteGATTServer object using NRF.connect(addr). This causes a disconnect event with reason 8, however, the method returns a connected object that can then be used for bonding. I was able to discover this as I connected the REPL to the physical Puck UART and could thus interrogate objects while bluetooth was connected to the iPhone.

    So, I have now been able to get the iPhone notifications on the Bangle without any firmware changes. The only issue with this method is that the startBonding() method promise is fulfilled before bonding is actually complete. This can be solved by waiting a bit and checking the security status too ensure the connection is encrypted.

  • @jeffmer: I am really impressed. I have an iPhone too and would be really interested to see what kind of application you will end up with~~

  • I have made quite a lot of progress with this and then hit another roadblock:

    Progress: The Bangle - or actually Puck.js which I am using for testing - can connect and bond reliably to the iPhone such that the iPhone automatically reconnects when it's Bluetooth is turned off and then on again. The Puck can create the required remote service and characteristic objects and send commands to start notifications etc. I can decode the data I get back from the iPhone on messages, calls etc.

    Problem: Data is transferred by the iPhone ANCS service using notifications. Each notification message can transfer 20bytes of user data. The problem arises when the data to be transferred is greater than 20 bytes. In this situation the iPhone sends multiple notification messages. The Puck gets the correct number of characteristicvaluechanged events, however usually, for two events received consecutively,the data associated with the first event will be the data from the last notification message. In other words, each of the two events has the same data values.

    @Gordon: - I have tried increasing the minimum interval which can be set during connection, however, the iPhone can send up to four packets per interval and it is obviously doing this when the data to be transferred is greater than the MTU of 20 bytes. The soft device is clearly getting these messages as I get the right number of events, however somewhere either in the soft device or the transfer to Espruino, the data is getting lost - any suggestions as to where in the code this might be happening?

  • Thanks for pushing further with this! I've actually had another report of this happening with something else (can't remember the link right now) so I'd love to get it fixed.

    So as far as I know what happens is:

    I'll have a think about trying to reproduce it here as well. I think it's probably not possible to do with Espruino itself as with the promises it doesn't like transmitting more than one frame per interval

  • Thanks for the pointers. I had a look through the code and it occurred to me that one might see exactly the behaviour I describe if the second IRQ signalling notification interrupted the first. I read in the Nordic documentation that softdevice event handlers usually run at priority 2 and I see that the Espruino IO queueing routines disable and enable interrupts to preserve exclusive access to shared queue structures, however, it was not clear to me as to whether the enable routine set a fixed priority or returned it to the priority that the IRQ handler was running at before the queueing operation was called.

    Do you think this might be a possibility or I have I got this completely wrong?

  • Wow, that's an interesting thought... jshPushEvent does call jshInterruptOff/On and that calls __set_BASEPRI(0). I'd have thought that something like that would have caused all kinds of other issues though.

    One thing that might help might be to look at the contents of the event queue to see what's actually in there?

    Do you have any thoughts about reproducing this easily? Would nRF Connect or something do it?

  • It is easy to reproduce if you have an iPhone! Seriously though, when first connected, ANCS sends a stream of the outstanding notifications and they have sequence numbers so it’s easy to see that messages are overwritten. Android also can send up to 6 packets per interval so you it should be possible to reproduce using Android. Not sure if nRF connect has the facility to quickly repeat notifications.

    I plan to experiment by creating cloned versions of the two queueing routines involved that do not use interruptOff and interruptOn and use these in the notification handler. I will let you know the result. The situation with notifications is fairly unique I think as usually interruptOn is only called after an event is queued and you are more or less finished with the IRQ. For reads of course, these are request response and the multiple message per interval probably does not happen.

    Is there an easy way to look at the contents of the IO event queue without using a Jlink debugger etc?

  • Sadly that did no good - no harm either, however, it looks like __set_BASEPRI(0) is not the problem.

  • Thanks for checking! I couldn't see how to convince nRF connect to write, so I'll have a go at forcing Espruino to send multiple packets

    I'm afraid as far as I know there isn't an easy way to get at the contents. I guess you could peek it by getting the address from the built firmware. While the data would be handled by that point it's a circular buffer so chances are you'd still be able to read it.

  • Looking at the following piece of code from targets/nrf5x/bluetooth.c, line 518 - see below -which handles the IOEvent, it looks to me like the data is associated with the characteristic not the event, so that if the second IOEvent is processed before a callback occurs, the second set of data will be installed in the characteristic value attribute and the callbacks for both events will thus see the same data. This implementation - if my understanding is correct - will always mean that the client sees the latest value of the server data, however, it means intermediate values are lost. It is reasonable if the client simply needs to monitor the value of a changing server variable, however, it leads to the current problem with ANCS. Not sure the best way to fix this, however, rather than create another queue, it might be better to defer processing of the second Bluetooth IOEvent until the first callback has occurred.

    case BLEP_NOTIFICATION: {
         JsVar *handles = jsvObjectGetChild(execInfo.hiddenRoot, "bleHdl", 0);
         if (handles) {
           JsVar *characteristic = jsvGetArrayItem(handles, data/*the handle*/);
           if (characteristic) {
             // Set characteristic.value, and return {target:characteristic}
             jsvObjectSetChildAndUnLock(characteristi­c, "value",
                 jsvNewDataViewWithData(bufferLen, (unsigned char*)buffer));
             JsVar *evt = jsvNewObject();
             if (evt) {
               jsvObjectSetChild(evt, "target", characteristic);
               jsiQueueObjectCallbacks(characteristic, JS_EVENT_PREFIX"characteristicvaluechang­ed", &evt, 1);
               jshHadEvent();
               jsvUnLock(evt);
             }
           }
           jsvUnLock2(characteristic, handles);
         }
    
  • Ahh - that's right! Thank you!

    Sorry, I'd been thinking about setServices for some reason (hence the EVT_HVX).

    Annoyingly it's the Web Bluetooth spec so I can't just change the way it works (eg for the event to have a data field for current data).

    But actually the fix should be quite easy. Just changing jsiQueueObjectCallbacks to jsiExecuteObjectCallbacks - as you say it'll just defer the processing until after.

    If that causes issues we could queue a function call that looks a bit like:

    function handleData(characteristic, data) {
      characteristic.value = data;
      characteristic.emit('characteristicvalue­changed',{target:characteristic});
    }
    
  • Ok, perfect. You can reproduce it just by occupying Espruino for a while and connecting to another Espruino that outputs data over BLE UART:

    var gatt;
    NRF.connect("fc:cc:b8:22:b0:42 random").then(function(g) {
      gatt = g;
      return gatt.getPrimaryService("6e400001-b5a3-f3­93-e0a9-e50e24dcca9e");
    }).then(function(service) {
      return service.getCharacteristic("6e400003-b5a3­-f393-e0a9-e50e24dcca9e");
    }).then(function(characteristic) {
      characteristic.on('characteristicvaluech­anged', function(event) {
        console.log("RX: "+JSON.stringify(E.toString(event.target­.value.buffer)));
      });
      return characteristic.startNotifications();
    }).then(function() {
      console.log("Done!");
    });
    setInterval(function() {
      for (var i=0;i<1000;i++);
    },50);
    
  • Progress! I will try the first suggestion and let you know. I will need a little help with translating the second idea into C calls to try the second.

  • It's ok - I'm on it :)

  • Ok, pull now and give it a go. I think it's fixed!

  • Ok, will do, many thanks

  • Amazing - all works perfectly. Expect ANCS app & widget soon - touch wood!

  • \o/ I'll have a think about implementing the central mode nicely

  • Apologies for the length of this post, however, I would really like to find out if the widget works for other people and I would appreciate feedback on some of the issues mentioned in the following.

    iPhone ANCS widget available for testing

    This widget allows you to answer or cancel iPhone incoming calls and also displays messages. It connects to the Apple Notification Center Service which is already on all iPhones, so you do not need to install any additional iPhone apps to use this widget.

    Warning - this is very much a beta release and although so far it appears quite stable, you may get the odd crash when moving between apps which will require a reboot - pressing BTN1 & BTN2 for 6 seconds. See issues at the end of this file.

    Installation

    First,you must update to the latest Bangle firmware - v2.05v27 or later.

    This beta version can be installed from this repository's app loader - https://jeffmer.github.io/JeffsBangleApp­sDev/. The widget will only run with a compatible clock app - see below. There are three listed in the App loader so install one or all three as they are linked and pressing BTN1 or BTN3 if they are not all installed will cause a black screen of death. Once installed, the widget will only appear in a compatible app when enabled in the Bangle Settings app - ANCS Widget will appear in APP/Widget settings.

    iPhone Pairing

    Once enabled, the widget icon should be displayed coloured grey (its green in the photo). Go to the phone's Bluetooth settings menu and your Bangle should appear under Other devices. If this is the first time you have connected with the Bangle from your iPhone, it may be named Accessory. Click on the name and the iPhone should connect and start pairing. The widget icon will turn red and the iPhone will ask you to enter a pairing code - the traditional 123456. After that, the iPhone may also ask to allow the device access to ANCS. Once pairing is complete, the widget icon should go blue and eventually green. The range of colours is:

    • Grey - not connected - advertising
    • Red - connected - not paired.
    • Blue - paired and connected - getting services
    • Yellow - got Services - discarding old notifications.
    • Green - waiting for new notifications - calls and messages only in this implementation.

    After pairing the first time, the Bangle should connect automatically when the widget is running. Sometimes you may need to click on the Bangle name in Settings:Bluetooth:My devices on the iPhone or disable and then enable Bluetooth to start connection. You may also need to get the iPhone to forget the bond if you want to connect the App Loader or Web IDE.

    Messages & Calls

    Messages are displayed as shown above until BTN2 is pressed to dismiss it. I strongly advise disabling the BTN2 LCD wake function in the Settings App as otherwise when the screen times out and you press BTN2 to wake the LCD, the screen will turn on and the Message Alert will be dismissed!. Calls can be answered or dropped.

    .

    Issues

    1. With GadgetBridge, the Android phone has a Central-Client role with the Bangle as Peripheral-Server. With the ANCS widget there is the fairly unusual situation in which the Bangle is Peripheral-Client to the iPhone's Central-Server role. Firstly, since Espruino does not deal explicitly with Bangle as Peripheral-Client, we have adopted the bodge outlined in an earlier posting and immediately connect back to the iPhone after the iPhone initiates the connection due to its Central role. This usually works, however, sometimes and especially when phone and Bangle are quite far apart, the Bangle connects in Peripheral-Server mode and the widget cannot get the Peripheral-Client connection it needs. The symptom is that the Bluetooth widget goes blue indicating connection from Central and the ANCS widget icon is grey indicating it has no connection. This should be fixed when Gordon deals with Issue 1800. Currently - the solution is to bring the Bangle near the phone and switch apps to force a reconnection attempt.

    2. When the Bangle switches apps, all state - including widget state - is lost unless explicitly stored. The consequence of this is that when the Bangle switches apps, the connection to iPhone has to be re-established to restore the remote GATT server and characteristics state. This is quite slow. To minimise reconnection, the widget needs to grab the screen from the running app to signal messages and calls. To allow this to work, the app needs to implement the SCREENACCESS interface. In essence, the widget only connects when running with compatible apps that implement this interface. An example of an implemented interface is:

      var SCREENACCESS = {
        withApp:true,
        request:function(){
          this.withApp=false;
          stopdraw(); //clears redraw timers etc
          clearWatch(); //clears button handlers
        },
        release:function(){
          this.withApp=true;
          startdraw(); //redraw app screen, restart timers etc
          setButtons(); //install button event handlers
        }
      }
      
      Bangle.on('lcdPower',function(on) {
      if (!SCREENACCESS.withApp) return;
      if (on) {
      startdraw();
      } else {
      stopdraw();
      }
      });
      

    I would be very interested in a less instrusive way of doing this, however, the solutions for full screen alerts that I have seen involve switching apps.

  • This looks great! Yes, I'm afraid right now claiming the entire screen is pretty tricky, since you'd have to stop apps trying to redraw everything (not just watches, but any input you could get from the system) and then restore it. One hack might be just doing g=Graphics.createArrayBuffer(1,1,1) to force everything else to draw to an offscreen buffer, but that doesn't help with buttons. Anyway, it'd be better to start a new thread for that :)

  • Post a reply
    • Bold
    • Italics
    • Link
    • Image
    • List
    • Quote
    • code
    • Preview
About

iPhone notifications for Bangle.js

Posted by Avatar for jeffmer @jeffmer

Actions