Re: This calculation is just wrong / computer can't count!


"GT" <ContactGT_removeme_@xxxxxxxxxxx> wrote in message
news:00238cdd$0$5102$c3e8da3@xxxxxxxxxxxxxxxxxxxx
"AliR (VC++ MVP)" <AliR@xxxxxxxxxxxxx> wrote in message
news:En7Ni.30834$eY.28056@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Maybe this will clear some of it up:
http://www.google.com/search?q=Significant+digits+double&hl=en

That info on significant digits was all very interesting, but how do I get
my code to work properly?!? I want to force the computer to calculate and
store as many digits as it can handle and no extra spurious digits, so
that my calculations get nice accurate results. At the moment part of my
calculation ends up with -0.00000000000007 (haven't counted the zeros
here, but you get the idea!), where it should be zero.


If your algorithm is susceptible to differences at the 17th decimal place
(as in your original post) then your algorithm is wrong.
There are an infinite number of numbers between 0.830 and 0.840 (ie between
*any* two numbers) unfortunately computers only have a limited number of
bytes in which to store these numbers, and a "standard defined" way of
storing them if it uses for example IEEE format.
Many numbers cannot be represented exactly in binary and so a compromise is
assumed.
If you were to try "by hand" your calculation, then the recurring 3's of
25.0/30.0 would cause you to run out of paper before you could do the actual
math. So you mentally adjust the *correct* number to (an incorrect) one
which you can handle.

Les


.

Relevant Pages

  • Re: Poll: Are PCs Turing Machines?
    ... > In my example, I don't even care about the calculation itself, only ... This is false UNLESS the condition is "I need to see all the digits". ... Instead we can imagine that in Heaven, the Riemann Hypothesis is solved ... If the Turing Abomination (a Turing machine executing such inelagant ...
    (sci.math)
  • Re: Converting binary floating point numbers to decimal?
    ... converting the integer part "1" to decimal, giving 1, then ... giving a sequence of digits: ... example (4 bits of fraction, ... 10^x calculation, ...
    (sci.math)
  • Re: Poll: Are PCs Turing Machines?
    ... > In my example, I don't even care about the calculation itself, only that the ... > such a number without a corresponding amount of memory. ... > I don't think the randomness of digits in pi has anything to do with the RH, ... > a universe in which there is an infinite amount of matter, ...
    (sci.math)
  • Re: The "spinoza" programming language
    ... It is irrational (in the mathematical sense of the word) ... you going to print out a number with an infinite number of digits? ... But lets suppose that you have a calculation, with DecimalPrecision unset, that spits out a variable equal to 0.333...+ ...
    (comp.programming)
  • Re: Poll: Are PCs Turing Machines?
    ... > In my example, I don't even care about the calculation itself, only that the ... > such a number without a corresponding amount of memory. ... > I don't think the randomness of digits in pi has anything to do with the RH, ... > a universe in which there is an infinite amount of matter, ...
    (comp.theory)