Re: This calculation is just wrong / computer can't count!

Maybe this will clear some of it up:
http://www.google.com/search?q=Significant+digits+double&hl=en

AliR.


"GT" <ContactGT_removeme_@xxxxxxxxxxx> wrote in message
news:00237e30$0$5087$c3e8da3@xxxxxxxxxxxxxxxxxxxx
I have been debugging something for ages now. I have a method that does
some complex maths, but right at the beginning it works out a proportion
and a few ratios and the maths is simply wrong. In my code I (obviously)
use variables and the values vary each time the method is called, but there
seems to a problem with the maths. I have narrowed the problem down to the
following. Can someone else please try this simply calculation and see what
their computer gets.

Code Line 1:
double effortChangeProportion = (55.0 - 30.0) / 30.0;


This first line does 55-30 and divides the result by 30. In other words
25/30, which is 0.8333 (recurring 3s).

The computer manages to give the answer 0.83333333333333337 !!

Code Line 2:
effortChangeProportion++;

or

effortChangeProportion = effortChangeProportion + 1.0;

The second line of code (both alternatives give the same result) builds on
the first by simply adding 1 (so I can then multiply other numbers by this
proportion).

In this case 0.8333 becomes 1.8333 , but again the computer gets this
wrong. It tries to add 1 to 0.83333333333333337 and gets
1.8333333333333335.

Obviously this can easily be done in 1 line of code, but it is broken down
to demonstrate the maths going wrong twice!

Can anyone shed some light on this for me please?

GT



.

Relevant Pages