Re: This calculation is just wrong / computer can't count!

"Joseph M. Newcomer" <newcomer@xxxxxxxxxxxx> wrote in message
news:edfgg35g02eh1jiaeb2f8cusas6gtudr7e@xxxxxxxxxx
See below...
On Thu, 4 Oct 2007 17:10:28 +0100, "GT" <ContactGT_removeme_@xxxxxxxxxxx>
wrote:

".rhavin grobert" <clqrq@xxxxxxxx> wrote in message
news:1191512413.771033.139730@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
On 4 Okt., 17:28, "GT" <ContactGT_remove...@xxxxxxxxxxx> wrote:
".rhavin grobert" <cl...@xxxxxxxx> wrote in message

news:1191511188.715360.292900@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

On 4 Okt., 17:00, "GT" <ContactGT_remove...@xxxxxxxxxxx> wrote:
".rhavin grobert" <cl...@xxxxxxxx> wrote in message
On 4 Okt., 16:24, "GT" <ContactGT_remove...@xxxxxxxxxxx> wrote:
What do you mean by the number might be below double granularity on
the
computer? What can I do about this?

your computer stores floatingpoints (float, double, long double) in a
format like

[sign][exponent][fraction]. that means that it can distinguish
between
0.0000003 and 0.0000004 but perhaps not between 700000.0000003 and
700000.0000004. on the other hand, it can handle numbers up to
3.4028235*(10^38) for a 32-bit floating point variable.

But if the computer stores numbers in this format, then why does it add
an
extra digit on the end that it can't handle and more importantly - how
do
I
stop it from adding the extra digit on the end?

it doesnt "add" it.

If you do...
int a = 1.34;

and then look into the memory at &a you'll find a "1", because the
compiler does a

int a = (int) 1.34; => int a = 1;

because the constant is first cast into the appropriate value (1 for
int, because granuarlity of int is 1) and then moved into memory at
&a. For doubles, thats the same. But doubles have granuarlity
antiproportional to absolute (unsigned) size. You can imagine it as a
"floating point" in decimals: guess you have (for example) 10 digits,
a sign (on= + /off = -) and something that states: put decimal point
here...

if you do a = 0.123456789, then it would be printed as 0.123456789, if
you do a = 0.1234567895 than it would be also printet as 0.123456789
because you're below granuarlity.

if you do a a = 1234567890 ten wit will be printed as 1234567890, if
you substact 0.4 then you're below granuarlity and may get 1234567890
or 1234567889 depending on your system.

I understand, but how do I stop the computer from working outwith its
granularity? I understand that there are a finite number of digits in a
double, plus a sign, plus an exponent, but why does the computer display
an
extra digit? I asked the computer to do 25/30 and store the answer as a
double. The answer is 0.8333333. The computer should surely only store as
many of the trailing 3s as it has space for. If it has 8 digits, then it
should store 0.83333333, but instead it shows a 7 as the last digit. This
isn't a rounding error, this is just wrong. How do I stop it from
showing/using the extra digits?

This doesn't even make sense. BY DEFINITION, the computer is forced to
work within the
limits of its physical design. There is no "extra digit" being added; I
have no idea why
you think it is an "extra" digit. It is NOT an "extra" digit, IT IS THE
VALUE THE
COMPUTER HAS! If you ask the computer to print out a floating point
number to maximum
precision, you are going to see the floating point number to its maximum
precision, and
the number you are seeing is the ACTUAL, PRECISE VALUE THAT IS STORED IN
THE COMPUTER.
That value is NOT 0.833,333,333,333,333,3 and it is not
0.833,333,333,333,333,33, it is
0.833,333,333,333,333,37. There is *NO* "extra" digit. This IS the value
that was
computed. (Note that I put , separators in so you can count the digits!).
There is
absolutely NO way you can represent 0.833,333,333,333,333,33 in 64-bit
IEEE 754 floating
point.

You can represent
0.833,333,333,333,333,26
which is represented as the 64-bit hexadecimal number
0xAAAAAAAAAAAAA
and you can represent
0.833,333,333,333,333,37
which is represented as the 64-bit hexadecimal number
0xAAAAAAAAAAAAB

Note that these differ by 1 bit in the low-order position. There are no
other
representations possible, and in particular, you cannot possibly find a
bit pattern that
represents
0.833,333,333,333,333,33

You obviously asked to display the value to maximum decimal places. If
you want to see
fewer decimal places, you would ask it to show the value to fewer decimal
places. You
obviously did not do this, so you get to see the entire value

[snip]

Please read the rest of the thread before jumping on the 'attack the OP'
bandwagon. This 0.83(r) number is not being displayed. I was always taught
to do my calculations to as many decimal places as possible, then only do
rounding at the last (display) stage, so my final calculations are rounded
to 0, 1 or 2 decimal places.

This 0.83(r) number the first step in a series of basic mathematics
calculations. The accuracy (not precision) of this number is relied on in
later calculations and in one example one number was subtracted from another
number and the result should have been zero. In actual fact, the PC
stored -0.00000000000007 (with the appropriate number of zeros).

My question was simple and everyone has jumped on my back with
binary/decimal conversion explanations, quotes about IEEE and standards, I
simply want to know how to get the computer to do 25 / 33 and store the
answer to a reasonable number of decimal places (at least 6). I don't want
it to store so many decimal places that it can't handle the number and
actually ends up storing a number that is mathematically incorrect (25/30 !=
0.833,333,333,333,333,37)! I understand the binary conversion and why
'double' can't store 0.833,333,333,333,333,33 and the last digit is
therefore random/jumbled/wrong/whatever you want to call it. My question is
simply, given that the last digit can never be relied on, why doesn't it
just not use the last digit when performing maths calculations?

A primary child could see that 25 / 30, then multiplied by 30 should be 25,
but the computer gets it wrong because of this incorrect final digit! How do
I get around this problem using the basic, build in C++ data types?


.

Relevant Pages

  • Re: This calculation is just wrong / computer cant count!
    ... extra digit on the end that it can't handle and more importantly - how ... "floating point" in decimals: guess you have 10 digits, ... The computer should surely only store as ... rounding at the last stage, so my final calculations are rounded ...
    (microsoft.public.vc.mfc)
  • Re: INFINITY Revisited
    ... >> since there is no largest natural number at any time before noon (nth ... >> interval) additonal balls always remain to be added and to be removed. ... decimals each with no last digit printed on them (perhaps this is only ... with no last digit must be either a repeating rational or an irrational ...
    (sci.math)
  • Re: Number Of Decimal Digits In Frequency Display
    ... >> Is the signle digit in the decimal poriton of the frequency display, ... newsagents in the city used to stock the South/West and London ... he hasn't seem more than one place of decimals. ...
    (alt.radio.digital)
  • Re: Reduce size of data
    ... AFAIK Jet stores Decimals like dBase does: no nibbles, ... digit, so at least 18 bytes for a decimal of precision 18. ... require 10 bytes to store (add one nibble for a sign and divide by two, ...
    (microsoft.public.access.tablesdbdesign)
  • Re: #999...999: When is a textbook not a textbook? Was: new textbook: "Mathematical-Physics (p&#
    ... your "isoceles triple" for 4,, ... for your infinitieth leftward digit, ... of the ordinary decimals' digit-beginnings, ... "the Peano Axioms model the wrong thing." ...
    (sci.math)