Re: Bug in excel fundamantal add-function

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance



No, it's not a bug. It's a "feature" of XL using IEEE double precision
floating point math. Most numbers can't be exactly represented in a
finite number of binary digits, just as 1/3 can't be exactly represented
by a finite number of decimal digits.

You can use ROUND() to round to zero if it's necessary.

See

http://cpearson.com/excel/rounding.htm

In article <1142809137.754056.127350@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>,
"snooze" <kze@xxxxxxxxxxx> wrote:

A7 must be 3 - but try to have 13 decimals...!!! is this a BUG? I think
so
.



Relevant Pages

  • Tough problem...
    ... I am hoping somebody here has enough expirience ... in math and floating point round off errors, ...
    (borland.public.delphi.language.basm)
  • Re: weird results when multiplying by a power of 10
    ... I'm not very familiar with how Matlab handles the math for ... I'm having problems multiplying certain numbers by the ... the limited floating point representation by HW (also typically called round ...
    (comp.soft-sys.matlab)
  • Re: Bug in excel fundamantal add-function
    ... floating point math. ... finite number of binary digits, just as 1/3 can't be exactly represented ... You can use ROUND() to round to zero if it's necessary. ...
    (microsoft.public.excel)
  • Re: Floating Point Problems
    ... > I have problems with floating point numbers. ... decimal representation of that value? ... those trailing bits make us round up ... I don't know enough about floating point math to say whether the .NET ...
    (comp.programming)
  • Re: prePEP: Decimal data type
    ... I think it's a good thing that you're forced to round ... >is about rounding rules and it's quite easy to avoid the "fail to meet the ... ISTM these things are 'way beyond the concern of a math package per se, ... for the meaning of "context" in this PEP context, ...
    (comp.lang.python)