Thank you for the response @Gordon and Welcome Back! Hope it was fun and relaxing. Me, summer break is over and back to the salt mine this week, so limited available time now.
I'll gather some more data that fails, in order to prove out the update when I get around to flash. I'll also create some cross check code after looking into the compare suggestion from #15 using Float64Array().
Then, I'll update the firmware and test, test, test. Also want to re-visit the other two posts involving rounding.
From the line that was modified, seems to confirm my hunch that it wasn't in the underlying mechanism that creates the floating point equivalent, which was what I initially thought it appeared to be, two weeks ago. Was it a number typo (then) or was there a particular reason that value was chosen? Seems a possible typo. Still puzzled though, how that online calculator provides a slightly different higher mantissa value that does either the near matching Browser or Espruino tests. To confirm, to debug this would require the steps Wilberforce and AkosLukacs #6 #7 #8 #9 point out? Maybe will have time to push through that this winter.
© Espruino, powered by microcosm.
Report a problem