Re: Q: When 0 <> 0 ?

Tech-Archive recommends: Fix windows errors by optimizing your registry

Yup. This is an FAQ, and an issue with all applications using IEEE
Double Precision Floating Point math. The problem is based on internal
rounding of values which can't be represented exactly in binary (much as
1/3 can't be represented with a finite number of decimal places:
0.333...). See

http://cpearson.com/excel/rounding.htm

for a more thorough treatment and, for a discussion of workarounds:

http://www.mcgimpsey.com/excel/pennyoff.html



In article <1130424196.344761.31740@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>,
splever@xxxxxxxxx wrote:

> A: When you add it in excel!
>
> Example: Place the following values in a column and sum them.
>
> -116.33
> 98.53
> 17.80
> 0
> =============
> Excel returns: 0.0000000000000035527136788005
>
> Correct Answer: 0
>
> If you remove 0 from the list ( or put it in a different place in the
> list, the proper result, 0 is returned.
>
> I assume this has to do with the known floating point issues, but I've
> never seen an example where 0<>0, so I thought I'd share.
>
> I have a workaround, but would love to see comments...
.

Relevant Pages

  • Re: Unacceptable floating point errors
    ... colleges and many commercial settings, (e.g., Mathematica), can provide ... Some software gives you better accuracy than does Excel. ... Single Precision Floating Points, about at 15 places in Double ... byte Double Precision Floating Point standard published by IEEE. ...
    (microsoft.public.excel.misc)
  • Re: When 0 <> 0 ?
    ... Excel stores numbers in IEEE double precision floating point ... normally working with approximations, an approximation of 15 ... Microsoft MVP - Excel ...
    (microsoft.public.excel)