Testing new builds - approach ??

Thanks! I know the actual code's a bit of a mess - it was more about trying to get something working quickly - but if you fancy working on it and want to change it around I'm more than happy :)

Your approach would allow small test result stats files (eg JSON) to be stored on the host over time, in a boards structure, and analysed with a tool there

IIRC it's what it does, but I'm not sure if that's really the best route. It's what I did originally, and then I had a script which ran over them and generated one big JSON file of relevant info that was viewable on a website online - but in the end with the tens of thousands of commits it took quite a long while to do that after each run.

... but then having said that we have all the firmwares in https://www.espruino.com/binaries/travis/master/ so one file per commit which listed all the test reports would probably work quite well as a starting point.