Using an IMU to control an airplane image

Posted on Thu 15th, June 2017

• #1

ClearMemory041063
Introduction

Having developed an Espruino module for the BNO055 IMU chip, this project will attempt to use the Euler angle of Yaw, Pitch and Roll that the chip provides to position a video rendering of an airplane on the computer screen. It is intended that as the orientation of the BNO055 chip is changed that the airplane image on the screen will follow.
http://forum.espruino.com/conversations/305928/

Implementation

The progress so far serves a webpage that display the airplane and HTMP post commands obtain updated Euler angles that the PICO increments. Integration with the BNO055, will follow in a few days.
A PICO with an ESP8266 attached by a shim is the platform. The BNO055 IMU chip is also connected to the PICO.

The attached files

Flight4.html contains the web page that will display the airplane, but it is set to simulate the changes of the Euler angles. If opened in a browser, it demonstrates the airplane in motion.
Flight4a.html is similar to flight4.html but is set to use HTML Post messages to the server to obtain the Euler angles.
Jnet.js is the Web credentials file that you will need to edit and place in the WebIDE projects module subdirectory. Edit the ssid and pw for your local wifi network.
```
/* Jnet.js */
exports = { id: "...", ssid: "funny", pw: "1234567890", ip: "..." 
```
ServePage3a.js contains the server code. It connects to the WiFi using jnet.js credentials.
It displays the IP address that you will use to access via a browser.
```
1v92 Copyright 2016 G.Williams
>Start
Start connection process
Try again
=undefined
Reset the ESP8266
Connecting to WiFi
IP=  192.168.1.6
null
Wi-Fi Connected
```
Once a browser connects you will see:
```
1v92 Copyright 2016 G.Williams
>Start
Start connection process
Try again
=undefined
Reset the ESP8266
Connecting to WiFi
IP=  192.168.1.6
null
Wi-Fi Connected
{"st":3333,"yaw":0,"roll":10,"pitch":0}
A
{"st":3333,"yaw":0,"roll":20,"pitch":0}
A
{"st":3333,"yaw":0,"roll":30,"pitch":0}
A
{"st":3333,"yaw":0,"roll":40,"pitch":0}
A
{"st":3333,"yaw":0,"roll":50,"pitch":0}
```
Screenshot1.jpg shows a screenshot of the webpage.

Future work
1. Use six switches, two for each axis. If a switch is closed increment or decrement the angle
2. Use 3 potentiometers two in a joystick and one for the rudder , use the values read by an analog port to increment or decrement the Euler angles.
3. Write the code to uses the BNO055 module to change the Euler angles.
```
Msg.roll+=10;
if(Msg.roll>360){
Msg.roll=0;
Msg.yaw+=45;
if(Msg.yaw>360){
Msg.yaw=0;
Msg.pitch+=15;if(Msg.pitch>360)Msg.pitch=0;
}//endif yaw
}//endif roll
var M=JSON.stringify(Msg);
console.log(M);
res.writeHead(200, {'Content-Type': 'text/plain','Content-length':M.length});
//res.writeHead(200);
res.end(M);
```
5 Attachments
• #2

Gordon

Nice! Are you thinking you might eventually use it to control a real model aeroplane?
• #3

ClearMemory041063
Using http Post is a bit slow. I'm working on using web sockets instead. I hope web socket data rates are faster. Model airplane control would be nice. I've seen some examples on the web. But that's another hobby to support.

I would like to try recording the heading and linear acceleration and then using integrations to get velocity and position, and plotting the course taken and see how well it would match the actual course. (Walk around the block for instance). This sort of thing could be useful to map a cave or tunnel.

1 Attachment
- websockserve.js
• #4

ClearMemory041063
The HTML POST problem

If you try to post and then process the reply too fast, the header information winds up in the message from the client to the server.

Are Websockets faster?

To find out I went to the Espruino site to try a Websocket example
https://www.espruino.com/ws
In the Websocket server example code:
```
var page = '<html><body><script>var ws;setTimeout(function(){';
page += 'ws = new WebSocket("ws://" + location.host + "/my_websocket", "protocolOne");';
page += 'ws.onmessage = function (event) { console.log("MSG:"+event.data); };';
page += 'setTimeout(function() { ws.send("Hello to Espruino!"); }, 1000);';
page += '},1000);</script></body></html>';
```
This is the HTML code sent to the client. Note the line page += 'ws.onmessage = function (event) { console.log("MSG:"+event.data); };';
Console.log is not a valid way to display a message in HTML/javascript.
The example works but nothing appears on the client screen.
A better example is in the attached file websockserve1a.js

Websockets seem to be faster

Reworked the HTML file into the attached file flight5a.html to use Websockets. It is incorporated into the server code as a string using the file converter site at
https://www.espruino.com/File+Converter
The attached file websocketserver2.js contains the server code using Websockets.

Bugs

The airplane flies and flies but eventually stops communicating.
Using the debugger on the client side the following two errors were occurring
```
ws.onmessage=function(event){
   var S=JSON.parse(event.data);
(index):1 Uncaught SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at WebSocket.ws.onmessage ((index):214)


VM44:37 WebSocket connection to 'ws://192.168.1.6:8000/my_websocket' failed: A server must not mask any frames that it sends to the client
```
The JSON.parse bug has been fixed.

The file trycatch.js shows how this was done in the client code. The messages sometime come in garbled and the JSONparse throws an error. The code catches the error and discards the message.
```
ws.onmessage=function(event){
try{
   var S=JSON.parse(event.data);
}
catch(e){return;}
   CalStat=S.st;
```
The other bug I have no clue. Helpful suggestions are welcome.

Is this project the start of a 3D mouse?
The POST method fails with 500 ms sample intervals.
The Websockets method works well at 300ms and maybe 200ms.

4 Attachments
• #5

Gordon

Console.log is not a valid way to display a message in HTML/javascript.

If you just open the Web browser's devtools you should still see a console with those messages?

The other bug I have no clue. Helpful suggestions are welcome.

The websocket disconnect one about server masking? And it doesn't happen immediately?

When the parse error occurs, please could you dump JSON.stringify(event.data) so we can see what's in it? I imagine that for some reason the data getting sent to the server is getting corrupted.
• #6

ClearMemory041063
I reworked the communications link last night. Removed the timer approach and went to the approach that the client sends a character, the server replies with the JSON string, on receipt by the client, but before the JSON parse, the client sends another character, then does the JSON parse within a try/catch framework. The results run much faster and much longer without errors

The port used in the browser is 8000 not 8080. (you can edit it if you like). So the IP address line of the browser reads:
http//192.168.1.6:8000
The latest are attached.

2 Attachments
- flight6a.html
- websockserve3.js
• #7

ClearMemory041063
Success

Combined two programs with some editing and the resulting program connects to the BNO055 IMU, then is connects to the local WiFi to serve the display page. As you move the BNO055 chip around the airplane on the screen seems to follow. (Just got it working so there are likely some gotchas, but I feel pretty good for now).
Put the BNO055.js in the modules directory of the WebIDE project (sandbox)
The WiFi is an ESP8266 on a Pico shim
The BNO055 is I2C. See code for pins.

2 Attachments
- 3DMouse.js
- BNO055.js
• #8

Gordon

Great! I guess it's possible that somewhere along the line a buffer got full and dropped some characters?

• #9

Gotcha 1 Glitches in the data from the BNO055.

Examine the pitch values which should be around 180 degrees but on occasion read near 0.

Cal= 3033
{"st":3033,"yaw":181.6875,"roll":18.5625,"pitch":-179.3125}
Cal= 3033
{"st":3033,"yaw":181.9375,"roll":16.875,"pitch":177.0625}
Cal= 3033
{"st":3033,"yaw":180.875,"roll":6.875,"pitch":178.25}
Cal= 3033
{"st":3033,"yaw":179.8125,"roll":3.375,"pitch":178.9375}
Cal= 3033
{"st":3033,"yaw":181.3125,"roll":3.1875,"pitch":179.9375}
Cal= 3033
{"st":3033,"yaw":181.75,"roll":2.3125,"pitch":-0.0625}
Cal= 3033
{"st":3033,"yaw":181.25,"roll":3.8125,"pitch":-0.0625}
Cal= 3033
{"st":3033,"yaw":182.625,"roll":5.1875,"pitch":-179.6875}
Cal= 3033
{"st":3033,"yaw":183.0625,"roll":6,"pitch":-0.0625}
Cal= 3033
{"st":3033,"yaw":183.375,"roll":7,"pitch":-177.75}
Cal= 3033
{"st":3033,"yaw":183.125,"roll":7.25,"pitch":-177.625}
Cal= 3033
{"st":3033,"yaw":183.9375,"roll":7.875,"pitch":-176.8125}
Cal= 3033
{"st":3033,"yaw":184.3125,"roll":-0.0625,"pitch":-0.0625}
Cal= 3033
{"st":3033,"yaw":185.0625,"roll":9.25,"pitch":-175.75}

Try using a median filter on the Yaw, Roll and Pitch data.

See attached file medianFilter.js and 3Dmouse8b.js

var MM=new xyzMF(7); //global
…
  W.getIMU();
  MMM=MM.getMF(W.Head);
  Msg.yaw=MMM[0];
  Msg.roll=MMM[1];
  Msg.pitch=MMM[2];
  console.log(JSON.stringify(Msg));

And the output:

Cal= 3003
{"st":3003,"yaw":178.625,"roll":2.875,"pitch":179.75}
Cal= 3003
{"st":3003,"yaw":178.6875,"roll":2.8125,"pitch":179.75}
Cal= 3003
{"st":3003,"yaw":178.6875,"roll":2.8125,"pitch":179.75}
Cal= 3003
{"st":3003,"yaw":178.625,"roll":2.75,"pitch":179.625}
Cal= 3003
{"st":3003,"yaw":178.5625,"roll":2.6875,"pitch":179.5625}
Cal= 3003
{"st":3003,"yaw":178.625,"roll":2.625,"pitch":179.6875}
Cal= 3003
{"st":3003,"yaw":178.6875,"roll":2.625,"pitch":179.75}
Cal= 3003
{"st":3003,"yaw":178.625,"roll":2.75,"pitch":179.75}
Cal= 3003
{"st":3003,"yaw":178.8125,"roll":2.625,"pitch":179.8125}
Cal= 3003
{"st":3003,"yaw":178.875,"roll":2.75,"pitch":179.875}
Cal= 3003
{"st":3003,"yaw":179.0625,"roll":2.75,"pitch":179.9375}
Cal= 3003
{"st":3003,"yaw":179.0625,"roll":2.8125,"pitch":179.875}
Cal= 3003
{"st":3003,"yaw":179.125,"roll":2.875,"pitch":179.875}

3DMouse8c.js reconnects the web interface and uses the median filter

This version eliminates the image radically changing positions on the screen.
I still wonder if I missed a data ready status in the BNO055 module that might be causing the problem.
The hypothesis BNO055 is updating values at the same time the PICO is trying to read them. I’ll have to revisit this issue.

Other gotchas relating to the position of the screen image and the position of the chip still remain to be resolved.

testws7a.js allows one to type yaw=23; roll=45; or pitch=-179 (supply your value) into the left pane of the WebIde and have the position of the airplane change on the web page.

5 Attachments

• #10

Gordon

Very strange about the pitch - I'd wondered whether pitch only varies between 0 and 180 degrees (so rolls over), but it seems you have around - 180 in there too!

Maybe you could try comparing the pitch you have with the raw values? It looks like you're using twos complement, but some devices just use 15 bits + sign for signed values?
• #11

ClearMemory041063

From the BNO055 datasheet

The output data format is based on the following convention regarding the rotation angles
for roll, pitch and heading / yaw (compare also section 3.4):

Table 3-13: Rotation angle conventions

Rotation angle Range (Android format) Range (Windows format)

Pitch +180° to -180° -180° to +180°

Roll -90° to +90° -90° to +90°

Heading / Yaw 0° to 360° 0° to 360°

Rotation angle	Range (Android format)	Range (Windows format)
Pitch	+180° to -180°	-180° to +180°
Roll	-90° to +90°	-90° to +90°
Heading / Yaw	0° to 360°	0° to 360°

• #12

ClearMemory041063

Found the fix to gotcha 1

Recall that glitch1 produces Pitch values that fluctuate for a steady position of the IMU chip.
The BNO055.js has been changed to fix this problem and the median filter has been removed from the 3dMouse8aDrain.js.
The original approach was to read a series of BNO055 registers in one big scoop.

BNO055.prototype.getIMU=function(){
 var IMU=new Int16Array(13);
 this.selectPage(0);
 IMU=this.ReadBytes(this.Page0Regs.EUL_Heading_LSB,26);
 this.Head[0]=IMU[0]/this.EULscale;
 this.Head[1]=IMU[1]/this.EULscale;
 this.Head[2]=IMU[2]/this.EULscale;
 this.Quant[0]=IMU[3]/2;
 this.Quant[1]=IMU[4]/2;
 this.Quant[2]=IMU[5]/2;
 this.Quant[3]=IMU[6]/2;
 this.Lin[0]=IMU[7]/this.ACCscale;
 this.Lin[1]=IMU[8]/this.ACCscale;
 this.Lin[2]=IMU[9]/this.ACCscale;
 this.Grav[0]=IMU[10]/this.ACCscale;
 this.Grav[1]=IMU[11]/this.ACCscale;
 this.Grav[2]=IMU[12]/this.ACCscale;
};//end getIMU

By changing the way the data are read from the BNO055 the data value flutter issue has been resolved.
(The Art of Computer Programming)

BNO055.prototype.getIMU=function(){
 var IMU=[];
 this.selectPage(0);
 var R=this.Page0Regs.EUL_Heading_LSB;
 for(var i=0;i<13;i++){
  IMU.push(this.ReadBytes(R,2));
  R=R+2;
 }//next i
 this.Head[0]=IMU[0]/this.EULscale;
 this.Head[1]=IMU[1]/this.EULscale;
 this.Head[2]=IMU[2]/this.EULscale;
 this.Quant[0]=IMU[3]/2;
 this.Quant[1]=IMU[4]/2;
 this.Quant[2]=IMU[5]/2;
 this.Quant[3]=IMU[6]/2;
 this.Lin[0]=IMU[7]/this.ACCscale;
 this.Lin[1]=IMU[8]/this.ACCscale;
 this.Lin[2]=IMU[9]/this.ACCscale;
 this.Grav[0]=IMU[10]/this.ACCscale;
 this.Grav[1]=IMU[11]/this.ACCscale;
 this.Grav[2]=IMU[12]/this.ACCscale;
};//end getIMU

The module will be further updated to extend this discovery to other block reads of data.

2 Attachments

• #13

ClearMemory041063
3DMouse8adrainB.js

This version uses the update BNO055.js module from the WebIDE sandbox.
It requires BNO055.js (sandbox), FlashEEPROM, and ESP8266WiFi_0v25 (Espruino modules).
It runs on a PICO with an ESP8266 connected via a shim.
The BNO055 module is connected as before.
// define the I2C connection I2C3.setup({ scl :A8, sda: B4} );
A normally open pushbutton is connected to ground and pin B7. Once the calibration status read 3333 the button is pressed to write the calibrations to the PICO EEROM
The calibration offsets and radiae are saved in PICO EEROM page var CalPage=89;
The html page is divided into two portions page1 and page2 to avoid out of memory errors
The web address uses port 8000. Given a router assigned IP address 192.168.1.6 use
"http//192.168.1.6:8000" in the browser. (Note older browsers may noe support the WebSockets used in this program. ( My old Android Tablet loads the page but doesn't support the websockets)
The drain command is used to serve the two pages.
```
function onPageRequest(req, res) {
  res.writeHead(200, {'Content-Type': 'text/html'});
 res.on('drain',function(){
  res.write(page2);
  res.end();//Page);
 });
  res.write(page1);
}
```
Some sample left pane output:
```
A
{"st":333,"yaw":341.125,"roll":16.625,"pitch":101.3125}
Usage= 4369
A
{"st":333,"yaw":341.0625,"roll":16.4375,"pitch":101.5}
Usage= 4369
```
The BNO055.js module and savCalBNO055.js can be found at:
http://forum.espruino.com/conversations/305928/

4 Attachments

Post a reply
- Bold
- Italics
- Link
- Image
- List
- Quote
- code
- Preview
Formatting Help

Don't worry about formatting, just type in the text and we'll take care of making sense of it. We will auto-convert links, and if you put asterisks around words we will make them bold.

Tips:

Create headers by underlining text with ==== or ----

To *italicise* text put one asterisk each side of the word

To **bold** text put two asterisks each side of the word

Embed images by entering:
![](https://www.google.co.uk/images/srpr/logo4w.png)
That's the hard one: exclamation, square brackets and then the URL to the image in brackets.

* Create lists by starting lines with asterisks

1. Create numbered lists by starting lines with a number and a dot

> Quote text by starting lines with >

Mention another user by @username

For syntax highlighting, surround the code block with three backticks:

```
Your code goes here
```
Just like Github, a blank line must precede a code block.

If you upload more than 5 files we will display all attachments as thumbnails.

For a full reference visit the Markdown syntax.

Using an IMU to control an airplane image

Introduction

Implementation

The attached files

Future work

The HTML POST problem

Are Websockets faster?

Websockets seem to be faster

Bugs

The JSON.parse bug has been fixed.

The other bug I have no clue. Helpful suggestions are welcome.

Success

Gotcha 1 Glitches in the data from the BNO055.

Try using a median filter on the Yaw, Roll and Pitch data.

3DMouse8c.js reconnects the web interface and uses the median filter

Other gotchas relating to the position of the screen image and the position of the chip still remain to be resolved.

Found the fix to gotcha 1

3DMouse8adrainB.js

About