Most recent activity
-
- 40 comments
- 3,847 views
-
Ok, then I understand your train of thought!
If I would use a very old (but very reliable) IBM keyboard I would probably need more force, compared with my current not so great MS Surface Pro 4 keyboard where keys move only 1 mm or so. With my way of typing, I'm however doubting the accuracy would be much better than randomly right, of course the Bangle/Bangles would need to be very tight to reduce any slipping.
Interesting idea anyway, I'll put it behind my ear.PS Similar solutions can be found by searching for
make any surface a keyboard, e.g.:
https://www.youtube.com/watch?v=cQqgm7PGSXA&ab_channel=Akshay
https://mashable.com/article/wearable-surface-keyboard -
And you use all fingers, not only index fingers? If using all fingers, how would it be possible to know what finger you pressed a key with, even if you are moving your wrists?
-
Not sure I understand, do you mean that Bangle could be trained to recognise what I'm typing on a keyboard? And this could be used later when I'm typing on e.g. a table surface, and Bangle connected to a device without external keyboard, like a phone?
At least when I'm touch typing with two hands, my wrists are resting on a surface and sometimes not moving at all, hence any accelerometer readings are infinitesimal and most probably unusable. Using one hand, and one finger, it might work as long as you "calibrate", i.e. know which x and y positions you are starting from. Could be doable even without AI.Or perhaps you mean something else...?
-
Thx @Serj !
Well, let's see what's next, no promises, what I have in mind will in worst case not include any Bangle or Puck, unless I can sneak them in somehow in the equation. Not revealing anything more for now, don't have a clue if I even can get the human interface and tech to work together :-) -
Thx! Somehow it's not replacing my keyboard either :D
Right before Christmas I actually put a Bangle on my ankle and taught it to recognize if I had shoes or socks on my feet when walking. While the accuracy was not great, it was better than random. I guess by tweaking the gesture sensitivity, using other type of shoes, and collecting lots more data, the accuracy would've been better. And no, I don't need AI to tell me what I'm wearing, HI (Human Intelligence) is good enough for that :)
So far Bangle & AI has for me mainly been a solution looking for a problem to solve, but you gotta start somewhere. -
Just an update to this for other AI & ML nerds like me. With Bangle.js2 I'm now able to spell the English alphabet in the air by "drawing" characters in the air (CAPS only +
spaceandbackspace, left outW). If you want to see it in action, check a short video I uploaded to my LinkedIn account, the link should work for anyone.The main changes I've done compared to the tutorial I wrote, is putting below code snippet in the beginning of the programs used to collect the gestures and to recognise them. The code sets the sensitivity for starting and ending the gestures. Depending on your use case, you might want to tweak the settings accordingly.
Bangle.setOptions({gestureEndThresh: Math.pow(700, 2), gestureStartThresh: Math.pow(600,2), gestureInactiveCount: 6, gestureMinLength: 15});If you want to connect your Bangle to e.g. a computer, and write in any application, the below code is what I uploaded to Bangle. I'm sure it could be optimized and cleaned (and commented!), but as a proof of concept, it works good enough for me. Feel free to provide improvements though, that way I'll learn JS better myself.
var storage = require('Storage'); var kb = require("ble_hid_keyboard"); NRF.setServices(undefined, { hid : kb.report }); const settings = storage.readJSON('setting.json',1) || { HID: false }; var sendHid, next, prev, toggle, up, down, profile; var time_on_screen = 500; Bangle.setOptions({gestureEndThresh: Math.pow(700, 2), gestureStartThresh: Math.pow(700,2), gestureInactiveCount: 6, gestureMinLength: 15}); if (settings.HID=="kb" || settings.HID=="kbmedia") { profile = 'Keyboard'; if (settings.HID=="kbmedia") { sendHid = function (code, cb) { try { NRF.sendHIDReport([2,0,0,code,0,0,0,0,0], () => { NRF.sendHIDReport([2,0,0,0,0,0,0,0,0], () => { if (cb) cb(); }); }); } catch(e) { print(e); } }; } else { sendHid = function (code, cb) { try { NRF.sendHIDReport([0,0,code,0,0,0,0,0], () => { NRF.sendHIDReport([0,0,0,0,0,0,0,0], () => { if (cb) cb(); }); }); } catch(e) { print(e); } }; } next = function (cb) { sendHid(0x4f, cb); }; prev = function (cb) { sendHid(0x50, cb); }; toggle = function (cb) { sendHid(0x2c, cb); }; up = function (cb) {sendHid(0x52, cb); }; down = function (cb) { sendHid(0x51, cb); }; } else { E.showPrompt("Enable HID?",{title:"HID disabled"}).then(function(enable) { if (enable) { settings.HID = "kb"; require("Storage").write('setting.json', settings); setTimeout(load, 1000, "hidkbd.app.js"); } else setTimeout(load, 1000); }); } function drawApp() { g.clear(); g.setFont("6x8",2); g.setFontAlign(0,0); g.drawString(profile, 120, 120); const d = g.getWidth() - 18; function c(a) { return { width: 8, height: a.length, bpp: 1, buffer: (new Uint8Array(a)).buffer }; } g.drawImage(c([16,56,124,254,16,16,16,16]),d,40); g.drawImage(c([16,16,16,16,254,124,56,16]),d,194); g.drawImage(c([0,8,12,14,255,14,12,8]),d,116); } if (next) { Bangle.on('aiGesture', (v) => { E.showMessage(v); switch (v) { case 'A': kb.tap(kb.KEY.A, 0); // next(() => {}); break; case 'B': kb.tap(kb.KEY.B, 0); // next(() => {}); break; case 'BACKSPACE': kb.tap(kb.KEY.BACKSPACE, 0); break; case 'C': kb.tap(kb.KEY.C, 0); // prev(() => {}); break; case 'D': kb.tap(kb.KEY.D, 0); break; case 'E': kb.tap(kb.KEY.E, 0); break; case 'F': kb.tap(kb.KEY.F, 0); break; case 'G': kb.tap(kb.KEY.G, 0); break; case 'H': kb.tap(kb.KEY.H, 0); break; case 'I': kb.tap(kb.KEY.I, 0); break; case 'J': kb.tap(kb.KEY.J, 0); break; case 'K': kb.tap(kb.KEY.K, 0); break; case 'L': kb.tap(kb.KEY.L, 0); break; case 'M': kb.tap(kb.KEY.M, 0); break; case 'N': kb.tap(kb.KEY.N, 0); break; case 'O': kb.tap(kb.KEY.O, 0); break; case 'P': kb.tap(kb.KEY.P, 0); break; case 'Q': kb.tap(kb.KEY.Q, 0); break; case 'R': kb.tap(kb.KEY.R, 0); break; case 'S': kb.tap(kb.KEY.S, 0); break; case 'SPACE': kb.tap(kb.KEY[" "], 2); break; case 'T': kb.tap(kb.KEY.T, 0); break; case 'U': kb.tap(kb.KEY.U, 0); break; case 'V': kb.tap(kb.KEY.V, 0); break; case 'X': kb.tap(kb.KEY.X, 0); break; case 'Y': kb.tap(kb.KEY.Y, 0); break; case 'Z': kb.tap(kb.KEY.Z, 0); break; } setTimeout(drawApp, time_on_screen); }); setWatch(function(e) { var len = e.time - e.lastTime; if (len > 0.3 && len < 0.9) { E.showMessage('prev'); setTimeout(drawApp, 1000); prev(() => {}); } else { E.showMessage('up'); setTimeout(drawApp, 1000); up(() => {}); } }, BTN1, { edge:"falling",repeat:true,debounce:50}); setWatch(function(e) { var len = e.time - e.lastTime; if (len > 0.3 && len < 0.9) { E.showMessage('next'); setTimeout(drawApp, 1000); next(() => {}); } else { E.showMessage('down'); setTimeout(drawApp, 1000); down(() => {}); } }, BTN3, { edge:"falling",repeat:true,debounce:50}); setWatch(function(e) { E.showMessage('toggle'); setTimeout(drawApp, 1000); toggle(); }, BTN2, { edge:"falling",repeat:true,debounce:50}); drawApp(); } -
Thx, I'll give it a try when I need to upgrade the other watches. Did not know about this app, but then I've not been following the app development that closely either.
-
@Gordon
What's the fastest and easiest way to upgrade the firmware from Windows?
The Web IDE says it can't perform upgrades at the moment, so I finally did it through nrfToolbox, not easy for me and my "fat fingers" to fiddle with my phone, and in addition I needed to do as mentioned above, pressReply100 times to get it through, 1% at a time.
Now, it's one thing to do it once, and for one watch only, but as I have 11 watches (for a workshop in February) I'm simple not going to do it this way for all of them, in worst case several times per year, so that's why I'm looking for a quicker and easier way.
Thx @Gordon !
Yes, the machine learning part is completely done with Edge Impulse, and I agree, it's an impressive platform, and for many use cases the time limit of 20 minutes (for the free tier) is more than enough. You can also do inference on a mobile, e.g. with camera/images, audio, or accelerometer.
I also tried to learn Tensorflow some years back, but got stuck on too many details here and there in Python, so Edge Impulse and other platforms hiding the inner parts of the engine help bringing ML "to the crowds".
For ML nerds: Other platforms I've found are Google's Teachable Machine (browser based) and Microsoft's Lobe.Ai (client), those are a bit quicker to get started with if you are only interested in ML with images or audio.