Using Puck.js as controller for MakeCode Arcade games

Hi there! I'm Bre, a game developer at a children's hospital. I'm exploring using Puck.js as a game controller for MakeCode Arcade games, so that we can use them for adaptive gaming, procedure support, etc. I'd like to show you guys what I've done so far, and see if I can get a little help as well.

You can try the following examples I've put together - just click "Connect Puck.js" at the bottom of the page to connect the Puck, and then tilt the Puck to play.

Space Destroyer - tilt Puck left/right (about accelerometer X axis) to move left right, tilt forward (about accelerometer Y axis) to shoot.

Hot Air Balloon - tilt Puck forward (about accelerometer Y axis) to move the hot air balloon up.

I'm using the Madgwick AHRS algorithm to estimate orientation, but only using the accelerometer/gyroscope data as input. Here's a demo that rotates a 3D model from the Puck accel/gryo data only. I'm pretty happy with the pitch/roll calculations, but heading has always been off. I was hoping to use the magnetometer to improve my orientation calculations.

If I add in the magnetometer data to the Madgwick algorithm like I do in this example, the orientation is way worse. The heading always seems to want to drift to the same direction, regardless of what direction it's pointing. I've been searching datasheets and documentation and code for answers, but for the life of me, I cannot get the magnetometer and accelerometer data to play well together.

Has anyone been able to combine the accel/gyro/mag data to estimate orientation? I have the Puck v2.1a, just FYI. I don't really know much about sensor fusion and I've been trying to figure this out for a while, so any help would be much appreciated! Thank you!

Hi - that sounds great! It's a really nice idea to add in some extra control methods for kids.

I think it's possible that what's happening is the Madgwick is expecting relative magnetometer values, but what you're getting are the absolute (effectively uncalibrated) ones.

Actually getting good, calibrated values from the magnetometer itself can be a total pain as it can change even if you move away from a metal desk or so on, but what I'd suggest is you do is store min/max values, work out a central value from those and then subtract that:

var Vec3 = require("Vec3");
var magMin = new Vec3();
var magMax = new Vec3();

Puck.magOn();
Puck.on('mag', function(xyz) {
  //console.log(xyz); // {x:..., y:..., z:...}
  var v = new Vec3(xyz);
  magMin = magMin.min(v);
  magMax = magMax.max(v);
  var diff = v.sub(magMax.add(magMin).mul(0.5));
  // ... diff contains the value you want
});

If you then pass that value in, you might find that works well - but you will have to roll the Puck around and around a bit when you start so that the min/max values get set up right

Thanks so much @Gordon! I'm still playing with it, but so far your suggestion seems to produce much better results when using the magnetometer. I'll update once I solidify the results a bit more.

I have also been looking at the Madgwick algorithm, and implemented a calibration program. I have set up a little webpage which displays relevant data, and is intended to measure steps.

https://baldursigurds.github.io/gamrun_dev/

@bre, I hope you don't mind that I based some of the graphics on your code which uses three.js, I have started reading their documentation.

What happens when you press the button that says "Calibrate magnetometer", it does just that, and gives you a translation vector and a symmetric matrix which together map values from the sensor to a sphere centered at the origin, using least squares.

What the program does is it switches between the MARG and the IMU algorithms depending on whether the magnetometer is close to the sphere or not.

I should mention, that for some reason, the MARG algorithm still doesn't seem to be working perfectly, although I just basically used the code from Madwick's paper.

One thing that I haven't figured out is that if I just record a hundred values from the magnetometer, and then calibrate according these values, then they fit very close to a sphere. In other words, the least squares are very small. When running everything with the accel/gyro as well, there are many jumps, where the coordinates suddenly jump to 32k. That's why in the code, it discards values with entries very big in absolute value. Even more, the values that then follow don't seem to fit the sphere perfectly. The value labelled as "Radius Ratio" gives the ratio between the length of the calibrated magnetometer and the radius of the sphere it should fit on. It typically fluxuates from about .5 to 2 or 3, when it should stay very close to 1. @Gordon any ideas? Maybe I should take this outdoor to do more testing, where there are fewer magnets around.

When running everything with the accel/gyro as well, there are many jumps, where the coordinates suddenly jump to 32k.

That's very odd - do you think you could provide some code that exhibits it, and what values to you have exactly? If it's something like 32767 then that might indicate a read error that maybe we could do something about in firmware.

do you think you could provide some code that exhibits it

You can try it on the webpage gamrun_dev that I linked to. Connect a puck to it, while the ide is running. Each datapoint is nine values, first three are acceleration, next three are the gyroscope, then the magnetometer. These values get console.log'ed out on the terminal there, they are directly from the sensors. You can then copy them into a csv file, and visualise them with gnuplot, or analyze them directly. I have done similar things, where I am only running the magnetometer, and this way, the data comes out clean. In fact, I have forked the BangleApps repository and added an app called calibrtr, it is available here:
https://baldursigurds.github.io/BangleApps/
The data that comes out of this typically looks much cleaner, and fits very well to an ellipsoid. I don't think I've seen any bad values coming from this method, having looked at a few datasets. I don't know what the protocols are for pull requests, but I may want to clean this code up a bit before sending it off. But you're welcome to have a look :)

Maybe I should add that I am now using an ML2032 rechargable battery, I should perhaps also try it with a classical CR2032, and see if that is the issue.

Thanks! I just looked into this and I think it may be conflict between calling mag and having the magnetometer turned on and taking timed readings but then just requesting data with Puck.mag() which currently makes its own reading. If the timing is wrong then it ends up reading twice.

As an example, this exhibits the problem:

Puck.accelOn(104);
Puck.magOn(80);

mag = {};

Puck.on('accel',function(a) {
    mag = Puck.mag();
    if (mag.x==-32768) print(mag);
    else print("ok");
});

but this doesn't:

Puck.on('mag', function(m) {
  mag = m;
  if (mag.x==-32768) print(mag);
    else print("ok");
});

I've fixed it in the cutting edge builds now, but ideally just use the second method and it should all work fine even on current builds.

Yes, this does indeed seem like a more sensible approach. When I wrote the Puck.mag() command, I though, "hmm, this can't be the best way to do this", but no second thought stopped me. I will modify the code when I get the chance, and test it out. This is great.