Number.toFixed() not rounding per specification

Posted on
  • Sun 2019.08.11

    toFixed() Does not round as does at the MDN site

    https://developer.mozilla.org/en-US/docs­/Web/JavaScript/Reference/Global_Objects­/Number/toFixed
    "toFixed() returns a string representation of numObj that does not use exponential notation and has exactly digits digits after the decimal place. The number is rounded if necessary, and the fractional part is padded with zeros if necessary so that it has the specified length."


    >var kwh = 1234.505;
    >kwh.toFixed(2)
    "1234.50"
    

    This rounds correctly in the try it window at Mozilla link above


    From MDN Correct string representation and rounding

    >(1234.505).toFixed(2)
    ="1234.51"
    


    >process.env
    ={
      VERSION: "2v04",
      GIT_COMMIT: "3956264e",
      BOARD: "ESPRUINOWIFI",
    



    EDIT:
    @AkosLukacs documented two other samples in #4 #8

    http://forum.espruino.com/conversations/­336605/


    Had a thought, could this anomaly be related to the recent discovery of floating point number of digits to right of zero not matching different browsers and node.js output? hmmmm. . . .

    http://forum.espruino.com/conversations/­336605/
    test files #1



    EDIT: Tue 2019.08.20
    Unable to verify with toPrecision()

    https://developer.mozilla.org/en-US/docs­/Web/JavaScript/Reference/Global_Objects­/Number/toPrecision

    >var kwh = 1234.505;
    kwh.toPrecision(2);
    
    Uncaught Error: Function "toPrecision" not found!
     at line 1 col 5
    kwh.toPrecision(2);
    

    Verified not a member of Number

    https://www.espruino.com/Reference#t_Num­ber

  • Tue 2019.08.20

    This might be of some use to rule out a floating point issue, or cross check a 'C' snippet, by observing the actual mantissa value used at the point of desired rounding:

    by @valderman Feb 2012
    https://stackoverflow.com/questions/9383­593/extracting-the-exponent-and-mantissa­-of-a-javascript-number

    Adapted from snippet above for the Espruino tests

    function getNumberParts(x) {
        var float = new Float64Array(1),
            bytes = new Uint8Array(float.buffer);
    
        float[0] = x;
    
        var sign = bytes[7] >> 7,
            exponent = ((bytes[7] & 0x7f) << 4 | bytes[6] >> 4) - 0x3ff;
    
        bytes[7] = 0x3f;
        bytes[6] |= 0xf0;
    
        return {
            sign: sign,
            exponent: exponent,
            mantissa: float[0],
        }
    }
    
    tests.forEach(function(x) {
        var parts = getNumberParts(x),
            value = Math.pow(-1, parts.sign) *
                        Math.pow(2, parts.exponent) *
                        parts.mantissa;
    
        console.log( "sign= " + parts.sign + "  exp= " + parts.exponent + "  man= " + parts.mantissa );
        console.log( "Testing:  arg= " + x + "  calc= " + value );
    });
    

    Value under test:

    var tests = [1234.505];
    
    
    sign= 0  exp= 10  man= 1.20557128906
    
    Testing:  arg= 1234.505  calc= 1234.505
    
    =undefined
    > 
    

    Still a bit of work to do with the actual manipulation of the mantissa.

    Also may be used for and related to post:

    Math.round() doesn't follow specification for exact 0.5



    References:

    nice pictorial on bit layout in F.P. register
    https://en.wikipedia.org/wiki/IEEE_754-1­985
    https://en.wikipedia.org/wiki/Floating-p­oint_arithmetic

  • Wed 2019.08.21

    Using the IEEE 754 specification link in #2 above, and using those primitive tools, a pencil and paper, made an attempt at viewing the bits that are used to create the floating point representation of our discovered value that appears to be problematic.

    GOAL: Make a determination if the issue is with the functions that are performing the conversion, or isolate if the underlying floating point math is problematic.

    Using our initial value, we split into the fractional and the integer parts. Using the integer value, convert that to base 2 using a calculator, and also the hex equivalent for reference. The fractional part uses a iterative process of simple multiplication, and records the carry value, either a one or zero.


    Step A
    Convert the Integer part to it's equivalent in base 2

    var kwh = 1234.505;
    
    Convert to F.P.
    Step A   Integral part 1234 base 10 = 0100 1101 0010 base 2 == 0x04D2
    
    Fractional Part 0.505
    0.505 x 2 = 1.01  1 
    0.01 x 2  = 0.02  0 
    0.02 x 2  = 0.04  0 
    0.04 x 2  = 0.08  0 
    
    0.08 x 2  = 0.16  0 
    0.16 x 2  = 0.32  0 
    0.32 x 2  = 0.64  0 
    0.64 x 2  = 1.28  1 
    
    0.28 x 2  = 0.56  0 
    0.56 x 2  = 1.12  1 
    0.12 x 2  = 0.24  0 
    0.24 x 2  = 0.48  0 
    
    0.48 x 2  = 0.96  0 
    0.96 x 2  = 1.92  1 
    0.92 x 2  = 1.84  1 
    0.84 x 2  = 1.68  1
    

    I only manually calculated 16 places as we really don't know how many bits will be required and quite frankly, is a tedious process.


    Step B
    Normalize that integer part (move the D.P. in this case left) so we have a value of one multiplied by it's base 2 power equivalent. We are after the resultant power which is used as an index.

    0100 1101 0010
    
    Step B Normalize - move D.P. left 10 places
    
    // we drop the leading zero in the initial value
    
     1.00 1101 0010  base2 x 2 ^10
    
    We moved the D.P. ten places left which is the equivalent of multiplying one and the fractional part by 2 ^ 10
    

    We mentally save ten, the base 2 power and record both to use in subsequent calculations. Looking back at the @valderman Feb 2012 stackoverflow.com getNumberParts(x) function from #2 above, we now can validate that the power 10 does match.



    Step C
    Now we may start to assemble our F.P. register value. Bit 31 is the sign. B23-B30 is the exponent. Bit0-B22 holds the mantissa

    1 bit sign  8 bit exponent  23 bit mantissa
    reg 0  00000000  0000000000.....bit0
       31 30        22                 0
         msb    lsb msb              lsb 
    

    Assemble the base 2 fractional part. Using the manual calculations from step 2

    mantissa is formed using the original adjusted exponent
    00 1101 0010  followed by
    1000 0001 0100 0111
    
    23 bits mantissa
    reg 0  10001001  0011010010.....bit0
       31 30        22                 0
         msb    lsbmsb               lsb 
    



    Step D
    Convert the stream of bits into the base we understand in hex notation

    F.P.  0  1000 1001   0011 0100 1010 0000 0101 000
    
    Separating by four bits, 0xF we now have
    
    0100 0100 1001 1010 0101 0000 0010 1000
    
    we can easily see the hex equivalent of
    
    0x449A 5028
    
    // this is our Floating Point equivalent value in hex for
    //  our base 10 value of 1234.505
     
    // that is the value used in the register performing F.P. calculations
    //  make sure not to confuse with the human readable
    //  hex equivalent 0x04d2 or 1234
    
    which when converted to human readable form is
    0x449A 5028  base 16
    equiv - as a floating point value not the original number 1234.505
    1150963752  base 10
    



    Now off to hand calculate the reverse process and build a simple snippet to check our work.

    Back in a bit (a long while)

  • Sat 2019.08.24

    Took a bit longer than I anticipated

    Attempting to uncover the source of this rounding error reminds of a book I read thirty years ago that had an impact on my career, if not more awareness to the world of computing. I was buried in electronics at the time, barely post tube era. (I hear the laughter now) oops, sorry proper web etiquette ROTF LOL

    Even remember the author, Clifford Stoll. Anyone with me? The Cuckoos Egg A true story that Cliff unravels tracking down $0.75 accumulated rounding error that leads him down the path tracking the crime in progress. And across a 1200 baud modem! Remember those days? Mesmerised over the step-by-step process to find the computer anomaly. Can't immediately put my hand on the hard copy I picked up later in the nineties, but did find a link. Isn't the web a marvelous place!!

    Grab a copy - worth the read

    http://bayrampasamakina.com/tr/pdf_stoll­_4_1.pdf



    Back to our Cuckoos Egg that is plaguing this discovered rounding issue.

    While cranking out routines to assist in building some data to analyze, I uncovered a slick online tool to create Floating Point values.

    https://www.exploringbinary.com/floating­-point-converter/

    I used this to cross check my manual register byte creation. Still need some looping functions to solve this anomaly though.





    From the following we can visually verify that the @valderman Feb 2012 snippet does in fact return consistent accurate results.

    Output from Chrome browser

    significand= 0011010010100000010100011110101110000101­00
    significand= 01234567890123456789012345678901      
    //    array element index reference only
    
    
    // we have a sign bit, eight exponent bits, followed by the significand, 
    // the string of bits we are after starting at element 10
    
    
    2^0 = 1 add leading implicit 1
    // we would use the above for our decimal representation
    
    str = [ 10000001010001111010111 ] 
    
    [ 2^ -1 ] 0.5
    [ 2^ -8 ] 0.00390625
    [ 2^-10 ] 0.0009765625
    [ 2^-14 ] 0.00006103515625
    [ 2^-15 ] 0.000030517578125
    [ 2^-16 ] 0.0000152587890625
    [ 2^-17 ] 0.00000762939453125
    [ 2^-19 ] 0.0000019073486328125
    [ 2^-21 ] 4.76837158203125e-7
    [ 2^-22 ] 2.384185791015625e-7
    [ 2^-23 ] 1.1920928955078125e-7
    
    sum = 0.5049999952316284
    



    Note the number of digits the browser presents to the right of the D.P.

    To determine if there might be any issues while adding, (remember we use the PC to perform the addition, which is using the same questionable Floating Point to perform that calculation) I added the summation intermediate step.

    I've added the element location beneath the addend to ease locating elements ref  D:0123456

    // Chrome browser
    2^0 = 1 add leading implicit 1
    str = [ 10000001010001111010111 ] 
    
    [ 2^-1 ]
    0.5
    - - - - - - - - - - - - - - - - - - - -
    0.5
    
    [ 2^-8 ]
    0.00390625
    D:12345678901234567890
    - - - - - - - - - - - - - - - - - - - -
    0.50390625
    
    [ 2^-10 ]
    0.0009765625
    D:12345678901234567890
    - - - - - - - - - - - - - - - - - - - -
    0.5048828125
    
    
    [ 2^-14 ]
    0.00006103515625
    D:12345678901234567890
    - - - - - - - - - - - - - - - - - - - -
    0.50494384765625
    
    [ 2^-15 ]
    0.000030517578125
    D:12345678901234567890
    - - - - - - - - - - - - - - - - - - - -
    0.504974365234375
    
    
    [ 2^-16 ]
    0.0000152587890625
    D:12345678901234567890
    - - - - - - - - - - - - - - - - - - - -
    0.5049896240234375
    
    
    [ 2^-17 ]
    0.00000762939453125
    D:12345678901234567890
    - - - - - - - - - - - - - - - - - - - -
    0.5049972534179688
    
    
    [ 2^-19 ]
    0.0000019073486328125
    D:12345678901234567890
    - - - - - - - - - - - - - - - - - - - -
    0.5049991607666016
    
    
    [ 2^-21 ]
    4.76837158203125e-7
    D:12345678901234567890
    - - - - - - - - - - - - - - - - - - - -
    0.5049996376037598
    
    [ 2^-22 ]
    2.384185791015625e-7
    D:12345678901234567890
    - - - - - - - - - - - - - - - - - - - -
    0.5049998760223389
    
    [ 2^-23 ]
    1.1920928955078125e-7
    D:12345678901234567890
    - - - - - - - - - - - - - - - - - - - -
    
    - - - - - - - - - - - - - - - - - - - -
    - - - - - - - - - - - - - - - - - - - -
    D:12345678901234567890
    0.5049999952316284
    
    toFixed(2)
    0.50
    
    // Chrome browser
    



    Note that the browser output renders up to 19 base ten numerals to the right of the D.P. and will default to 'E' notation when significantly small a value.

        document.write( sum.toString(10) );
    
      // it appears that the toString() function has no effect on the formatting in this case
        document.write( sum );
    
    Findings:
    
    Chrome output no format specifier, but 16 numerals past the D.P. and up to 19 used during rendering to get to this final value
    0.5049999952316284
    
    
    Espruino output no format specifier, but truncates (up to) 11 numerals
    
    0.50499999523
    





    Now let's compare to the Espruino output

    // Espruino output
    2^0 = 1   add leading implicit 1
     
     
    [ 2^- 1 ]
    0.5
     
    D:12345678901234567890
    -------------------------
    0.5
     
     
    [ 2^- 8 ]
    0.00390625
     
    D:12345678901234567890
    -------------------------
    0.50390625
     
     
    [ 2^-10 ]
    0.0009765625
     
    D:12345678901234567890
    -------------------------
    0.5048828125
     
     
    [ 2^-14 ]
    0.00006103515
     
    D:12345678901234567890
    -------------------------
    0.50494384765
     
     
    [ 2^-15 ]
    0.00003051757
     
    D:12345678901234567890
    -------------------------
    0.50497436523
     
     
    [ 2^-16 ]
    0.00001525878
     
    D:12345678901234567890
    -------------------------
    0.50498962402
     
     
    [ 2^-17 ]
    0.00000762939
     
    D:12345678901234567890
    -------------------------
    0.50499725341
     
     
    [ 2^-19 ]
    0.00000190734
     
    D:12345678901234567890
    -------------------------
    0.50499916076
     
     
    [ 2^-21 ]
    0.00000047683
     
    D:12345678901234567890
    -------------------------
    0.50499963760
     
     
    [ 2^-22 ]
    0.00000023841
     
    D:12345678901234567890
    -------------------------
    0.50499987602
     
     
    [ 2^-23 ]
    0.00000011920
     
    D:12345678901234567890
    -------------------------
    0.50499999523
    
    toFixed(2)
    0.50
    
    // Espruino output
    



    Using our online floating point generator, we can see that our input of 1234.505 using either precision presents a number that should round to the hundredths place. The value 1234.5050048828125 contains enough bits to the right of our last thousandths numeral '5' to allow for a round up. During the conversion back to decimal, several bits beyond 2^-23 are lost, leaving us with a value that is just shy of our original '5' thousandths place. 1234.50499999

    As I have hand coded two examples and we have the @valderman stackoverflow snippet as an additional cross check, I'm 0.9987654321 x 10^2 % certain that the Floating Point mechanism works near flawlessly, as one would expect, making the assumption that proven underlying 'C' Math libraries are accurate and used.



    But, still a bit puzzling is that Espruino and the Chrome browser produce near identical numerals to the right of the D.P.
    Espruino - Single precision 32 bit: 0.50499999523
    Chrome - Single precision 32 bit:0.5049999952316284

    The online calculator produces a slightly greater value:

    Single precision 32 bit: 1234.505004882812
    Double precision 64 bit:1234.50500000000010913936421275138854980­46875



    As the conversion to and from F.P. introduces a slight bit of error, It would be nice if we had a right of the D.P. adder that works using integers only. This could assist in resolving if there is additional slop during the summation of the equivalent right of D.P. bits. Anyone up for the task to create that little wonder, while I battle cleaning up these snippets to allow posting here?



    The Html code file presents the conversion to a floating point value and uses color hinting to show the concatenation of bits to form the register value used in math calculations. It was a quick hack to compare summation output against Espruino output. Nothing fancy. Needs a bit of refactoring and renders more of a what's going on under the hood. Can be used to check conversions back to decimal along with toFixed() checks.


    1 Attachment

  • Sat 2019.08.24

    Although I have spent a great deal of time in an attempt to resolve this rounding issue, I am currently behind on the skills and don't have the tools needed to perform a correction. I'm also waaaaaaay short on time to face the learning curve involved.

    In case I enter one of my sadistic moments to venture down a path of creating compilations,

    I've scanned

    https://github.com/espruino/Espruino
    https://github.com/espruino/Espruino/blo­b/master/README_BuildProcess.md

    and it appears one could perform this task on a Windows10 PC. If I'm understanding this, I need a virtual PC to run a Linux OS image. Ubuntu is used to compile the files that are written in 'C'. Ubuntu runs Python scripts that perform the compilation.

    So far, am I on the right track?

    Then what is not clear, what tools are needed to perform debugging? My background from a decade ago was VStudio pre community edition, building web services in C/C++/C#.

    Any other intuitive assistance to get a good overview would be appreciated.

  • If you are on 64 bit Windows the best bet is to install WSL. Windows subsystem for Linux. It's in the Windows shop app.

    This gives you an Ubuntu shell window without needing to install Linux in a virtual machine.

    Once you are in the shell - do the apt-get to install the build tools as per the Linux instructions. Then you can just a make and build the Linux version of espruino, which you can then run with ./esprunio --telnet

    In your ide, you can then connect to localhost:2323 to connect to this install running in Linux on top of Windows.

  • '64 bit Windows'

    Yes, have that.

    Thank you @Wilberforce, that helps in clearing that up. I'm sure others reading along will appreciate that info also. By "In your ide" does this imply a browser of my choice, or is this an IDE that will be available once Ubuntu shell is installed? Is a debugger available, or do we use the Chrome debugger?

    The biggest obstacle I see is attempting to traverse the 'C' logic in performing the rounding. I've uncovered these links:

    TO BE ADDED SHORTLY - EDIT see post #10 for functions and entry points

    BTW: This will be more of a winter project for me, as I really don't want to start meddling with the underlying source. Will also need to re-build my Linux skills along with telnet commands, quite an undertaking. However, as implied by post #4, this has been a tremendous learning experience in the understanding of how complex, just a simple addition instruction truly is under the covers. Most probably don't give a rat's expletive but it has clued me in to the complexity of what you guy's actually do. Thank's for that effort!!

  • Use WSL as @Wilberforce already recommended. Or of course you can use an Ubuntu VM. You can use VS Code with the remote WSL extensions, makes modifying files much easier...

    Just install WSL, VSCode, "Remote - WSL" extension

    • Start a new VSCode & launch WSL
    • open the terminal in VSCode
    • cd ~ (if you are not there by default)
    • git clone https://github.com/espruino/Espruino.git­
    • Press the "Open Folder" button
    • and follow the Linux instructions :)

    One rookie mistake I did: don't try to use WSL in your regular windows file system (by default you are at something like /mnt/c/something == c:\something). Clone the Espruino repo into a WSL directory, otherwise bash scripts just went crazy.

    As for debugging: AFAIK you won't be able to single step debug the Espruino code as easily as you could do from VS. There gdb, but that itself has a steep learning curve afaik.

  • Sun 2019.08.25

    Thank you @AkosLukacs for the additional tidy summary.

    'There gdb, but that itself has a steep learning curve afaik.'

    Found this ten minute overview of GDB, which supplies the basics,

    https://www.youtube.com/watch?v=sCtY--xR­UyI

    and there is VSCode, Git and Bash to learn on top of it all.



    With only a basic introduction with RedHat over a decade ago, this process will feel like starting from scratch in an entirely new environment. Planning on chunking, maybe an hour a week, until I'm up to speed.

    With three Espruino projects in the works, will even have to delay that thought. A winter starter perhaps.

  • Sun 2019.08.25

    Links to areas of source that appear to be the entry points that will need a looksy:

    Definition

    https://www.espruino.com/Reference#l_Num­ber_toFixed

    GitHub source file for toFixed()

    https://github.com/espruino/Espruino/blo­b/master/src/jswrap_number.c#L120

    JsVar *jswrap_number_toFixed(JsVar *parent, int decimals) {
      if (decimals<0) decimals=0;
      if (decimals>20) decimals=20;
      char buf[JS_NUMBER_BUFFER_SIZE];
      ftoa_bounded_extra(jsvGetFloat(parent), buf, sizeof(buf), 10, decimals);
      return jsvNewFromString(buf);
    }
    



    JS_NUMBER_BUFFER_SIZE    is suspect, but I'm not able to locate where that constant is defined



    ftoa_bounded_extra

    https://github.com/espruino/Espruino/blo­b/master
    https://github.com/espruino/Espruino/blo­b/master/src/jsutils.c
    https://github.com/espruino/Espruino/blo­b/master/src/jsutils.c#L614

    void ftoa_bounded_extra(JsVarFloat val,char *str, size_t len, int radix, int fractionalDigits) {
    
    // possible areas that end of array may be encountered
    

    jsvGetFloat

    https://github.com/espruino/Espruino/blo­b/master
    https://github.com/espruino/Espruino/blo­b/master/src/jsvar.c#L1867

    JsVarFloat jsvGetFloat(const JsVar *v) {
    
    // too many lines and entry points to list here
    
  • Sun 2019.08.25

    And just when one thinks every stone is over turned, more web searching digs up that others have found that toFixed(2); is problematic between different browsers and have concluded the safest method is to use your own rounding function, 'just to be sure!'

    https://stackoverflow.com/questions/1001­5027/javascript-tofixed-not-rounding

    A link there over turns this relatively recent 'fix' - "Revision 1383484 of Math.round()"

    https://developer.mozilla.org/en-US/docs­/Web/JavaScript/Reference/Global_Objects­/Math/round$revision/1383484

    "Rounding 1.005 should have an expected result of 1.01, but it returns 1."



    So, what do we know so far for function toFixed()?

    by definition:

    'The number is rounded if necessary, and the fractional part is padded with zeros if necessary so that it has the specified length'

    https://developer.mozilla.org/en-US/docs­/Web/JavaScript/Reference/Global_Objects­/Number/toFixed

    EXCEPT when it doesn't, then one must apply a patch that is only discovered when locating the recommended fix as suggested in revision 1383484.

    Pardon me while I go scream!!

  • Mon 2019.08.26

    I'm so hoarse, can't be heard, so decided to create that patch 'fix' as suggested. Used the standard rounding function which is recognized by most:

    var roundedNum = ( Math.round( num * 100 )  /  100 ).toFixed( place );
    
    



    Interactive fiddle Demo

    http://jsfiddle.net/jfriend00/kvpgE/

    Tried it out in Chrome browser

    num= 1.005 place=2
    roundedNum= 1.00
    
    
    num= 1234.505 place=2
    roundedNum= 1234.51
    



    Now we are on the right track, or are we???

    Test using Espruino WebIDE

    num= 1.005  place=2
    roundedNum= 1.00
     
      
    num= 1234.505  place=2
    roundedNum= 1234.50
    



    And, SURPRISE!!   that failed. Is it the toFixed() or the actual Math.round() this time???


    2 Attachments

  • Mon 2019.08.26

    Now running on sheer insanity, and a will and determination to find an acceptable answer, found a jsFiddle solution that MDN recommends. A Closure that wraps the Math.round() function.

    Interactive

    http://jsfiddle.net/cCX5y/2/


    Tried that out in the Chrome browser with it's own with embedded revealed steps

    1.005     p2
    inside Math.round10(value, exp); val=1.005 exp=-2
    inside decimalAdjust() type=round val=1.005 exp=-2
    inside decimalAdjust() shift val=1.005
    inside decimalAdjust() shift val=101
    inside decimalAdjust() shift back val=101
    inside decimalAdjust() shift back ret=1.01
    inside Math.round10(value, exp); decimalAdjust() ret=1.01
    Math.round10(1.005, -2); = 1.01
    
    
    
    1234.505     p2
    inside Math.round10(value, exp); val=1234.505 exp=-2
    inside decimalAdjust() type=round val=1234.505 exp=-2
    inside decimalAdjust() shift val=1234.505
    inside decimalAdjust() shift val=123451
    inside decimalAdjust() shift back val=123451
    inside decimalAdjust() shift back ret=1234.51
    inside Math.round10(value, exp); decimalAdjust() ret=1234.51
    Math.round10(1234.505, -2); = 1234.51
    
    



    So we can confirm the confirmed MDN rounding of 1.005 now works in the browser.


    Now we have to be on the right track track this time, Right?

    Test using Espruino WebIDE

    1.005    p2
      inside  Math.round10(value, exp);   val=1.005  exp=-2
      inside decimalAdjust() type=round  val=1.005  exp=-2
      inside decimalAdjust() shift  val=1.005
      inside decimalAdjust() shift  val=100
      inside decimalAdjust() shift back  val=100
      inside decimalAdjust() shift back  ret=1
      inside Math.round10(value, exp);  decimalAdjust() ret=1
      Math.round10(1.005, -2);  = 1
    
    
    1234.505    p2
      inside  Math.round10(value, exp);   val=1234.505  exp=-2
      inside decimalAdjust() type=round  val=1234.505  exp=-2
      inside decimalAdjust() shift  val=1234.505
      inside decimalAdjust() shift  val=123450
      inside decimalAdjust() shift back  val=123450
      inside decimalAdjust() shift back  ret=1234.5
      inside Math.round10(value, exp);  decimalAdjust() ret=1234.5
      Math.round10(1234.505, -2);   = 1234.5
    



    This time it appears to be the combination of the toString() function and toFixed()

    So there ya' have it. Even the MDN recommended closure that * IS * the Javascript solution to this (lack of) rounding anomaly, also fails.

    Don't shoot the messenger here. Remember, a month ago all I wanted was a reliable way to shift one bit left. Sheer persistence and digging got us this far.


    2 Attachments

  • Tue 2019.08.27

    Anyone with a running 'C' compiler and debugger able to cross check these values please? Or any ideas to provide other helpful means that will assist in pin pointing what is going on?

    https://www.exploringbinary.com/floating­-point-converter/



    It is puzzling that Espruino and the Chrome browser produce near identical numerals to the right of the D.P. when hand calculated, or using the output value from the @valderman Feb 2012 snippet.

    Value 1234.505

    Espruino - Single precision 32 bit: 0.50499999523
    Chrome - Single precision 32 bit:0.5049999952316284

    The online calculator produces a slightly greater value:

    Single precision 32 bit: 1234.505004882812
    Double precision 64 bit:1234.50500000000010913936421275138854980­46875

    I originally thought this might be that double precision was more accurate, as there are an additional 32 bits that when summed, would cause a higher value. But it can be seen that is not the case using the online calculator. surprisingly, the value is smaller

  • Hi, thanks for looking into this. I just had a look at the code involved and I believe I just fixed it: https://github.com/espruino/Espruino/com­mit/3c700f9d92e73d08dc28a60abd29915a8abd­9499

    If you try with the latest builds hopefully you'll have some luck.

    Floating point is a bit of a nightmare as any error could be:

    • conversion of a text string to floating point
    • the actual floating point maths
    • conversion of floating point to text

    If this comes up in future one solution might be to compare (new Float64Array([1234.50])).buffer which gives you the dependable, non-fuzzy binary representation of the underlying floating point number.

  • Tue 2019.08.27

    Thank you for the response @Gordon and Welcome Back! Hope it was fun and relaxing. Me, summer break is over and back to the salt mine this week, so limited available time now.

    I'll gather some more data that fails, in order to prove out the update when I get around to flash. I'll also create some cross check code after looking into the compare suggestion from #15 using Float64Array().

    Then, I'll update the firmware and test, test, test. Also want to re-visit the other two posts involving rounding.

    From the line that was modified, seems to confirm my hunch that it wasn't in the underlying mechanism that creates the floating point equivalent, which was what I initially thought it appeared to be, two weeks ago. Was it a number typo (then) or was there a particular reason that value was chosen? Seems a possible typo. Still puzzled though, how that online calculator provides a slightly different higher mantissa value that does either the near matching Browser or Espruino tests. To confirm, to debug this would require the steps Wilberforce and AkosLukacs #6 #7 #8 #9 point out? Maybe will have time to push through that this winter.

  • it wasn't in the underlying mechanism that creates the floating point equivalent

    Correct, yes.

    Was it a number typo (then) or was there a particular reason that value was chosen?

    I believe at the time it was a badly thought out attempt to ensure that digits further down the number didn't cause rounding - for example 0.5549 -> 0.56. Of course because it works in at least 90% of cases it wasn't immediately apparent that there was an issue.

    how that online calculator provides a slightly different higher mantissa value that does either the near matching Browser or Espruino tests.

    https://www.exploringbinary.com/floating­-point-converter/ says:

    It is implemented with arbitrary-precision arithmetic, so its conversions are correctly rounded

    I'm not entirely sure on what it means there but on Espruino the stringToFloatWithRadix function uses repeated multiply and add with floats to convert the number, and multiply/divide by 10 is a particularly bad thing to do for accuracy.

    The converter could quite happily store the integer and fractional parts as two arbitrarily long integers and then combine them at the end, and that would probably provide a more accurate number in a lot of cases.

    To confirm, to debug this would require the steps Wilberforce and AkosLukacs #6 #7 #8 #9 point out?

    It's worth playing with WSL anyway - it's really simple to compile Espruino in it, and you could then change the Espruino source to output the binary values of the number at each stage of the arithmetic.

  • Post a reply
    • Bold
    • Italics
    • Link
    • Image
    • List
    • Quote
    • code
    • Preview
About

Number.toFixed() not rounding per specification

Posted by Avatar for Robin @Robin

Actions