Re: This calculation is just wrong / computer can't count!

"Les" <l.neilson@xxxxxxxxxxxxxxxxxxx> wrote in message
news:fe327n$10cj$1@xxxxxxxxxxxxxxxxxxxx

"GT" <ContactGT_removeme_@xxxxxxxxxxx> wrote in message
news:00238cdd$0$5102$c3e8da3@xxxxxxxxxxxxxxxxxxxx
"AliR (VC++ MVP)" <AliR@xxxxxxxxxxxxx> wrote in message
news:En7Ni.30834$eY.28056@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Maybe this will clear some of it up:
http://www.google.com/search?q=Significant+digits+double&hl=en

That info on significant digits was all very interesting, but how do I
get my code to work properly?!? I want to force the computer to calculate
and store as many digits as it can handle and no extra spurious digits,
so that my calculations get nice accurate results. At the moment part of
my calculation ends up with -0.00000000000007 (haven't counted the zeros
here, but you get the idea!), where it should be zero.


If your algorithm is susceptible to differences at the 17th decimal place
(as in your original post) then your algorithm is wrong.
There are an infinite number of numbers between 0.830 and 0.840 (ie
between *any* two numbers) unfortunately computers only have a limited
number of bytes in which to store these numbers, and a "standard defined"
way of storing them if it uses for example IEEE format.
Many numbers cannot be represented exactly in binary and so a compromise
is assumed.
If you were to try "by hand" your calculation, then the recurring 3's of
25.0/30.0 would cause you to run out of paper before you could do the
actual math. So you mentally adjust the *correct* number to (an incorrect)
one which you can handle.

Almost - your last part is wrong. You don't *adjust* the number, you
*truncate* the number to a certain level of precision. Even if you truncate
the number to 2 decimal places it is 0.83 not 0.87 !!

In the case of double the level of precision seems to be 16 digits, but this
is actually irrelevant. Whatever the number of digits used, the last digit
should be a 3 not a 7 !! My algorithm is fine, the calculation is producing
an incorrect answer. At a point later in my calculation I end up subtracting
one number from another and the answer should be zero, but instead (due to
the above problem) the answer is not == 0.


.

Relevant Pages

  • Re: How to increase Working Precision?
    ... I don't get credit for letting "generally outstanding" slip by? ... and 9.95, with three correct digits. ... It printed as zero because it ... is deemed to have too little precision to print otherwise ...
    (sci.math.symbolic)
  • Re: Singles to Doubles
    ... In internal promotion from lower precision to higher, there isn't any information as to what the missing decimal digits should be -- that they should be zero is only one of a large set of possible values. ... The way to force that in any modern language I'm familiar with is essentially what we've been discussing here...make a decimal representation that looks like the number you want and then let the i/o routines read that in, doing the hard work of converting its internal representation to match as closely as possible. ...
    (microsoft.public.vb.general.discussion)
  • Re: Calendrical centering
    ... Detonation of the first A-bomb July 16, ... That's a reason I like the end of the Younger Dryas as a zero point. ... Digital watches would need to carry 18 digits, ... Moreover, the setting of today's current time lacks a bit of precision, ...
    (talk.origins)
  • Re: Calendrical centering
    ... Detonation of the first A-bomb July 16, ... That's a reason I like the end of the Younger Dryas as a zero point. ... Digital watches would need to carry 18 digits, ... Moreover, the setting of today's current time lacks a bit of precision, ...
    (talk.origins)
  • Re: Poll: Are PCs Turing Machines?
    ... > In my example, I don't even care about the calculation itself, only ... This is false UNLESS the condition is "I need to see all the digits". ... Instead we can imagine that in Heaven, the Riemann Hypothesis is solved ... If the Turing Abomination (a Turing machine executing such inelagant ...
    (sci.math)