Re: This calculation is just wrong / computer can't count!


"GT" <ContactGT_remove_@xxxxxxxxxxx> wrote in message
news:0057ac41$0$11709$c3e8da3@xxxxxxxxxxxxxxxxxxxx

We all know computers are binary machines, not decimal, but why can't the
computer do the same basic maths as a pocket calculator?!?

Hand calculators (and I believe the calculator in Windows) use internal
decimal arithmetic.

0.833333 is not representable in binary.

If you need your program to work with full accuracy and precision to 17
decimal places then you will need to write your own package working in
decimal math, or as Luke said

"find an arbitrary precision math library on the web and use it."


25/30 is 0.833recurring and it should calculate, store and display as many
digits as can be stored at the desired level of granularity, not randomly
invent an extra digit on the end!

The extra digit is not _random_ it is caused by the binary value being the
closest it can get to the number you specify / calculate.

I repeat my question:

Then we can only repeat the same answers. You really need to know about
floating point math and "internal representation"


How do I get the computer to calculate and store the number of digits that
it can handle and no more. I don't care if it calculates 6, 10, 15, 27
digits, but I do *require* a correct result 25/30 is *NOT* 0.83337 or
0.8333337 or 0.833333333333333333337. and 0.8333337 + 1 is *NOT* 1.8333335
!!


(repeat, sorry)
If you need your program to work with full accuracy and precision to 17
decimal places then you will need to write your own package working in
decimal math, or as Luke said

"find an arbitrary precision math library on the web and use it."


Les


.