Re: Rounding of the double

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

actually, it doesn't store it in hex, it stores it in binary, with an implied high-order
bit. Since a normalized floating-point number ALWAYS has a 1 in the high order position,
that is assumed, and the first mantissa bit we see is actually the SECOND mantissa bit,
since the first bit is always implied.

IBM got into trouble with numerical analysts when they introducted the /360, because it
did store floating point values in hex, that is, the normalization was to 4 bits of
precision (the high-order nybble being nonzero) and there was no implied decimal point, so
it was intrinsically less precise than the binary representations that were in use at the
time.

Otherwise, I agree completely with your points. Floating point always seems to come as
such a surprise to programmers.

In my numerical analysis course, the professor spent 6 weeks telling us how to do
numerical analysis right, then spent the next 6 weeks telling us how all computers did it
wrong and if we wanted to write numerical algorithms we had to take these errors into
account. The theory of numerical analysis assumes essentially infinite precision.

When I was working on my PhD dissertation, I went to one of the foremost numerical
analysts (who was on the CMU faculty at the time) and asked him what techniques should be
used to optimize floating point computations. His answer was "none". The explanation was
that either the code was written by someone who was clueless, in which case optimization
didn't matter, or it was written by a numerical analyst, in which case any transformation
from the original computation would distort the results and introduce significant errors.
"We work with algorithms whose errors are sometimes six orders of magnitude larger than
the result during certain points of the computation, but which then all cancel out,
leaving us with a result to the intended precision". Which made my life easier, because I
ended up with a whole lot of work I didn't need to do.
joe

On Thu, 31 May 2007 18:35:14 +0100, "David Webber" <dave@xxxxxxxxxxxxxxxxxxxxxxxxxxx>
wrote:


"Alex" <alsim123@xxxxxxxxxxx> wrote in message
news:1180624216.737392.276140@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

I have double d1 = 80.00
double d2 = 100.00

double result = d1/d2;

result = 0.80000000000000004 !!??
....
Is there any way actually if I divide 80 by 100 to get 0.80 ?

1. Some unpleasant truths about floating point numbers:

(In what follows I'll use an ellipsis to mean recurring indefinitely.)

0.80000000000000004 *is* 0.80

"0.80" means a number between

0.79500000000000...
and
0.80499999999999...

"0.800" means a number between

0.799500000000000...
and
0.800499999999999...

You want the answer

0.8000000000000...

But in fact the numbers you divided were only specified here to two decimal
places, and so technically the answer is not known to that accuracy.

2. To the point: I suspect you are using a convention where you can write
0.80 and assume that this means 0.80000... with zeroes going on for ever.
This is a very dangerous way of looking at things, and runs into deep
trouble in various ways. One of the ways it runs into trouble is that the
computer stores numbers hexadecimally and the number you want (8 tenths +
no hundredths + no thousandths + no ten-thousandths +...) in general has no
particular reason to be represented by a finite string of hexadecimal
digits.

Another problem is when you divide two numbers an inaccuracy can occur in
the final significant figure stored by the computer. It would have to
store an infinite number of digits to avoid this.

The upshot of these and other considerations is that you should *never*
trust the least significant digits of floating point variables. A
corollary is that you should also never test them for equality.

So if you're using floating point, you have no right to make demands on the
least significant figures and you must write your code in a way which
manages them.

Dave
Joseph M. Newcomer [MVP]
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
.

Relevant Pages

  • Re: BigNum -- Floating Point
    ... <snip English description of method> ... Mixed-base floating point, perhaps? ... use a big amount of space to store it. ... Then either you take the loss of precision like a man; ...
    (comp.programming)
  • Re: Floating Point Precision
    ... > I have a question about floating point precision in C. ... The relative error (loosely speaking, the precision) ... It is the difference between 1.0f and the "next" float value, ... a field known as "Numerical Analysis." ...
    (comp.lang.c)
  • Re: Conformance
    ... where 100 digits of precision is useful. ... I've read a discussion of an algorithm which failed after N iterations, where N was about 10 when using 32-bit floating point. ... The point is, the analyst had to be pretty clever to figure out that alternative algorithm, and not everyone can afford the time needed to become an expert at numerical analysis. ...
    (comp.std.c)
  • Floating point load-store behaviour.
    ... I am trying to predict the behaviour of floating point load and store ... -> Is that particular bit getting flipped coz of rounding or something ...
    (comp.lang.c)
  • Floating point load-store behaviour.
    ... I am trying to predict the behaviour of floating point load and store ... -> Is that particular bit getting flipped coz of rounding or something ...
    (comp.arch)