Re: Problem in Float Arithmetic

On Mon, 23 Jun 2008 15:53:31 +0200, "Christoph Conrad"
<spamcruncher@xxxxxxxxxxxxxxxxxxxx> wrote:

Hi Doug,

* Doug Harrison [MVP] <dsh@xxxxxxxx> wrote:

So you should use "double".

That will give him a more accurate result, but it still won't be
exact.

That is correct, and the meaning of "precision".

No, I meant "accurate". It won't merely have more digits (precision), the
result will be closer to the mathematical result (accuracy).

--
Doug Harrison
Visual C++ MVP
.

Relevant Pages

  • Re: float bug? perl 5.8, DBI and oracle 10.2.0
    ... precision numbers in oracle, you've got 38 decimal digits to play ... and with minimal coaxing perl will handle them as ... digits from a 32 bit floating point number - I'll go out on a limb ... and hazard that one can expect 12 or so digits from a 64 bit floating ...
    (perl.dbi.users)
  • RE: float bug? perl 5.8, DBI and oracle 10.2.0
    ... I would not characterise 32-bit signed integers as giving 10 digits ... truncate and tell people you get 9 digits of precision. ... perl 5.8, DBI and oracle 10.2.0 ... Floating point values are typically stored in 64 bits or sometimes 96 ...
    (perl.dbi.users)
  • Re: float bug? perl 5.8, DBI and oracle 10.2.0
    ... I would not characterise 32-bit signed integers as giving 10 digits ... truncate and tell people you get 9 digits of precision. ... perl 5.8, DBI and oracle 10.2.0 ... Floating point values are typically stored in 64 bits or sometimes 96 ...
    (perl.dbi.users)
  • Re: 15 Significant Digits Limitation a Mistake for Spatial Informa
    ... DP does not restrict to 15 decimal digits. ... Input and output precision are more tightly linked in Excel ... Decimal data type or roll your own extended precision data types. ...
    (microsoft.public.excel)
  • Re: Salamin-Brent algorithm
    ... fraction-representing digits, which is of course ... might have thought one could dispose of because ... since one's precision is so increasing with the ... S-B algorithm)? ...
    (sci.math)