[Bug libc/4943] Inconsistent rounding behaviour for sprintf and IEEE doubles

["paul at inet dot co dot za" <sourceware-bugzilla at sourceware dot org>]

------- Additional Comments From paul at inet dot co dot za  2007-09-23 07:19 -------
(In reply to comment #26)
> (In reply to comment #25)
> > Since the late 80's I could quote the IEEE storage format without a second
> > thought.
> 
> Knowing the format is not the same as knowing how FP works.
<snip>
> FYI, you need 17 digits to make sure that no information is lost in the
> conversions. Your "15 decimal digits of precision" is no more an approximation.
<snip>
> > When you break it down, though, it's not even 52 bits, as you cannot ignore
> > calculation and storage error.
> 
> No, you mix up "accuracy" and "precision". The precision is always 53 bits for
> the double format (well, not taking subnormals into account). After
> calculations, the precision remains the same, but the accuracy can decrease (and
> it will, in general).
<snip snip >
> one, I could write it and find values that gave different rounding directions
> because I understand floating point.

Vincent, Now you are just arguing semantics.  You cannot propel yourself forward
by patting yourself on the back.

Your example does not produce an unexpected result due to a rounding error (as
per the examples in my previous query) but rather produces an unexpected result
due to an actual inaccuracy in the storage.  

>This is not 5000.524999999999600, but a 53-bit value. The exact value (written
>in decimal) is: 5000.524999999999636202119290828704833984375. But if you
>understand floating point, you should know that...

Sorry, Vincent, but I do not agree with that bit of blurb.  The actual value
when written in decimal cannot take more than 15 decimal digits into
consideration.  The correct way to interpret the value is 
    5000.524999999999636 == 5000.52499999999 
with a possible error of
       0.000000000009636

If you round the value 5000.524999999999 to 3 decimal points, you get 5000.525.
 If you round the number 5000.524999999999 to 2 decimal points, you get 5000.52.
This is not due to inconsistency, but inaccuracy.

In fact floating point storage format (mantissa, exponent and sign) is no
different from scientific notation which is the decimal equivalent of the binary
floating point, other than that the precision of floating point numbers is fixed
at 15 decimal digits of precision.

But if you understood floating point, you would know this.

For now, it seems that a pocket calculator does a better job of converting
binary floating point to decimal text representation than either GLibC,
MSVCRT.DLL or Vincent Lefèvre .


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=4943

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.