Re: Glaring Excel calc bug.

"Earl Kiosterud" <some...@xxxxxxxxxxx> wrote...
I agree that neither decimal nor binary can represent values
precisely. I was just referring to the conversion issues. If
we have decimal values to start with, and want decimal results,
why not do the math in decimal? We can certainly afford the
memory and speed hits with todays hardware.
....

So switching to precision as displayed is too difficult for you?

As for arbitrary precision arithmetic, there are several widely and
freely available libraries that provide it, but Microsoft couldn't use
'em because they're licensed under terms Microsoft can't stomach. And
given their track record, I know I wouldn't trust Microsoft to roll
their own and get it right.

.

Relevant Pages

  • Re: Glaring Excel calc bug.
    ... referring to the conversion issues. ... Using BCD does not eliminate the issues of finite precision arthmetic, ... math, instead of having to do binary conversions? ... You get 0 from =1.01-1-0.01 (removing the parentheses) because Excel is ...
    (microsoft.public.excel)
  • Re: Display of Single Data Exported
    ... So, Access provides the currency data type, which is not a floating point ... It stores exactly 4 places of precision after the ... Microsoft introduced a scalable point data type - a Number ...
    (microsoft.public.access.tablesdbdesign)
  • Re: Are f.p. manipulation functions only used in initialization?
    ... After reset control word = 037F ... Copyright Microsoft Corporation. ... precision KIND is quadruple precision. ... control at 53 bits as Microsoft seems to want to do because on Win64 ...
    (comp.lang.fortran)
  • Re: LDBL_MAX -1.#QNAN0e+000 with MinGW?
    ... Microsoft is perfectly right to do that since the C standard EXPLICITELY ... the compiler to the run time library and this is surely not Microsoft's ... there are numbers that are rappresentable with 80bit float ... The CPU is in 64 Bit (double precision), ...
    (comp.lang.c)
  • Re: Xilinx "somethings wrong" error
    ... as these vectors are, without the sizing headaches. ... -- Numeric_Std natively does modulo based math ... -- integer based math does full precision math: ... group - note the enhancement is not the LRM text - LRM ...
    (comp.lang.vhdl)