You are reading a single comment by @Robin and its replies.
Click here to read the full conversation.
Espruino is a JavaScript interpreter for low-power Microcontrollers. This site is both a support community for Espruino and a place to share what you are working on.
@Robin
Tue 2019.08.27
Anyone with a running 'C' compiler and debugger able to cross check these values please? Or any ideas to provide other helpful means that will assist in pin pointing what is going on?
It is puzzling that Espruino and the Chrome browser produce near identical numerals to the right of the D.P. when hand calculated, or using the output value from the @valderman Feb 2012 snippet.
Value
1234.505Espruino - Single precision 32 bit:
0.50499999523Chrome - Single precision 32 bit:
0.5049999952316284The online calculator produces a slightly greater value:
Single precision 32 bit:
1234.505004882812Double precision 64 bit:
1234.5050000000001091393642127513885498046875I originally thought this might be that double precision was more accurate, as there are an additional 32 bits that when summed, would cause a higher value. But it can be seen that is not the case using the online calculator. surprisingly, the value is smaller