Re: This calculation is just wrong / computer can't count!

"Luke alcatel" <luke@xxxxxxxxxxx> wrote in message
news:uyM7l$pBIHA.4476@xxxxxxxxxxxxxxxxxxxxxxx
I'll risk criticism by saying this thread is silly. A middle school child
learns not to express calculation results with more significant digits
than the data or instruments provide. A double provides 14-15 significant
digits so why are you surprised that the result is not what you
theoretically expect at the 15th digit?

I am surprised because I am using a C++ basic data type and it shouldn't
stumble over basic mathematics!

Clue #1 adjust your print formats for fewer significant digits.

This number is the first stage in some complex calculations and the problem
is compounded resulting in a final number that is just wrong.

Clue #2 don't try to land on Jupiter if your flight control software uses
doubles but your navigation system requires 18 significant digits.

I only require about 5 or 6 decimal places, but float and double both see
the same problem, so what am I supposed to do?

Clue #3 find an arbitrary precision math library on the web and use it.

I found one - its called the basic C++ data types. int, float, double etc.


Why is everyone having a go at me? I asked a perfectly simple question that
is confusing 3 of us here. In front of me, I have a casio calculator, a
pencil and paper and a PC. I type some numbers into the calculator and
scribble on the paper and derive 5 numbers. The first stage in the
calculation is 25/30. The calculator gives us range of 5 number that
match/confirm what we worked out on paper, but the PC gives us something
else that is just wrong. We are not working on a low level C system here, we
are writing an MFC application with dialog boxes, menus and the likes. There
is not a printf in sight.

One of the results should be exactly 0, but the computer gives the
result -0.00000000007. Which is then displayed on the screen as -0.0. This
result is displayed on the screen (in an edit boxes) along with all the
positive numbers and is just wrong!

25/33 = 0.833 recurring. 0.833337 is not *innacurate* it is *WRONG*. End of
story.

Even if this 0.83333..3337 were true, then adding 1.0 to it CANNOT BE
1.8333...335 - it has changed the last digit!!! If it can't handle the last
digit, then it shouldn't display/use it.

If the computer can work to 8, 12, 15, 16, 18 digits, then why does it get
the last digit wrong?

I understand all this significant digit stuff, my problem is - Why does the
computer store and use the 'dodgy' last digit if it is actually WRONG and
causes incorrect results?


.

Relevant Pages

  • Re: Complacency
    ... Show them the resulting display of a long line of their favorite digit. ... :: were unable to get sensible answers, so they called in the manager. ... :: The manager went off and brought back a calculator. ... :: I was the only person in the shop and I think all this took at least ...
    (alt.support.diabetes)
  • Re: This calculation is just wrong / computer cant count!
    ... In case you missed the reality check, YOU ARE GETTING THE CORRECT ANSWER ... asked him about issues of optimizing floating point computations. ... there really isn't very much call for three billion digit arithmetic otherwise. ... In front of me, I have a casio calculator, a ...
    (microsoft.public.vc.mfc)
  • What is this problem and where did I find it?
    ... You have a calculator with 0 in the display, ... calculations are done with absolute accuracy and that you can display ... I am not sure that this is the correct statement of the problem. ... left-most digit tells the number of 0's in the whole number, ...
    (sci.math)
  • Re: Is there any possibility to implemented a 32-bit float point calculator in two week ?
    ... What I want is a simple float arithmetic unit for a Calculator. ... and floating-point with powers of 10 instead ... in exactly the same way you were taught at school, digit ... DOULOS - Developing Design Know-how ...
    (comp.lang.verilog)
  • Re: dbm to volts (humax F2foxT)
    ... ITYMM the difference between rounding to the nearest n significant ... digits and truncating to n significant digits ... determine the least significant digit value). ... 2 digit accuracy for db figures suffices for most practical ...
    (uk.tech.digital-tv)