Re: I don´t view the value right after six decimal place in Excel



The procedure informed does not work. I wish that I send the Schedules so you
can make the tests?
If so, tell me your e-mail. Thank you.

"joeu2004" wrote:

PS....

On Aug 5, 9:02 am, Claudio <Clau...@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote:
From the sixth decimal place of any number in Excel does not present
the accuracy of the system in Cobol.
Would I like to do this in Excel? (have the same accuracy from the sixth
home)

This might help.....

For every calculation, use the ROUND() function to set the result to
the desired precision. You might want to do that around intermediate
results (subexpressions) as well as final results. This ensures that
the internal binary representation is exactly the same as if you had
entered the result as a constant.

That still might have the same effect as COBOL decimal arithmetic.
But it should reduce the effects of "propagation error". Consider
this example: enter =100*ROUND(2.00666666666667,2)-200 and compare
with 1. They are not identical in Excel 2003. Now compare 1 with
round(100*ROUND(2.00666666666667,2),0)-200.

You might consider setting the option Tools > Options > Calculation >
Precision As Displayed. But first, I believe that affects only the
final result, not intermediated calculations (subexpressions). And
second, it might not have the desired effect when using values with
different precisions.

If you experiment with this option, it would be wise to make a copy of
the workbook first because some changes cannot be undone by simply
unsetting the option and recalculating.

.



Relevant Pages

  • Re: demonic numbers !
    ... > corresponding calculation of error intervals. ... Then they're either not doing enough math for rounding errors to ... accumulate or they're not using a wide enough range of magnitudes. ... They don't need more than 53 bits of precision to store those values. ...
    (comp.lang.lisp)
  • Re: ROUNDING RESULT OF CALCULATION UP OR DOWN
    ... In one of the calculation rules we had to round to a whole year. ... I keep repeating we need decimal arithmetic in Excel, ... | ??>> the limits of precision? ...
    (microsoft.public.excel.misc)
  • Re: Problem with a small "exercise" code using compiler xlf
    ... going from REAL to DOUBLE PRECISION solved part of the ... In the case of problems like this, negative square roots when ... in the intermediate calculation gives a negative square root. ... one could choose one rounding mode for one part ...
    (comp.lang.fortran)
  • Floating Point , Wide zero etc
    ... The default precision of 4 is used. ... require 6 places for GST calculation. ... The simple rule is never store unconverted data for accounting. ... the binary approximation converts back to ...
    (comp.databases.pick)
  • Re: For Andrew Breen, plasma
    ... physicists use slide rules and everything. ... you can get almost all the precision you want! ... calculation with it, but have forgotten the usage of the log-log scales ... I had a couple of K&E 20 inch rules. ...
    (sci.military.naval)

Loading