Re: This calculation is just wrong / computer can't count!

See below...
On Thu, 4 Oct 2007 17:10:28 +0100, "GT" <ContactGT_removeme_@xxxxxxxxxxx> wrote:

".rhavin grobert" <clqrq@xxxxxxxx> wrote in message
news:1191512413.771033.139730@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
On 4 Okt., 17:28, "GT" <ContactGT_remove...@xxxxxxxxxxx> wrote:
".rhavin grobert" <cl...@xxxxxxxx> wrote in message

news:1191511188.715360.292900@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

On 4 Okt., 17:00, "GT" <ContactGT_remove...@xxxxxxxxxxx> wrote:
".rhavin grobert" <cl...@xxxxxxxx> wrote in message
On 4 Okt., 16:24, "GT" <ContactGT_remove...@xxxxxxxxxxx> wrote:
What do you mean by the number might be below double granularity on
the
computer? What can I do about this?

your computer stores floatingpoints (float, double, long double) in a
format like

[sign][exponent][fraction]. that means that it can distinguish between
0.0000003 and 0.0000004 but perhaps not between 700000.0000003 and
700000.0000004. on the other hand, it can handle numbers up to
3.4028235*(10^38) for a 32-bit floating point variable.

But if the computer stores numbers in this format, then why does it add
an
extra digit on the end that it can't handle and more importantly - how do
I
stop it from adding the extra digit on the end?

it doesnt "add" it.

If you do...
int a = 1.34;

and then look into the memory at &a you'll find a "1", because the
compiler does a

int a = (int) 1.34; => int a = 1;

because the constant is first cast into the appropriate value (1 for
int, because granuarlity of int is 1) and then moved into memory at
&a. For doubles, thats the same. But doubles have granuarlity
antiproportional to absolute (unsigned) size. You can imagine it as a
"floating point" in decimals: guess you have (for example) 10 digits,
a sign (on= + /off = -) and something that states: put decimal point
here...

if you do a = 0.123456789, then it would be printed as 0.123456789, if
you do a = 0.1234567895 than it would be also printet as 0.123456789
because you're below granuarlity.

if you do a a = 1234567890 ten wit will be printed as 1234567890, if
you substact 0.4 then you're below granuarlity and may get 1234567890
or 1234567889 depending on your system.

I understand, but how do I stop the computer from working outwith its
granularity? I understand that there are a finite number of digits in a
double, plus a sign, plus an exponent, but why does the computer display an
extra digit? I asked the computer to do 25/30 and store the answer as a
double. The answer is 0.8333333. The computer should surely only store as
many of the trailing 3s as it has space for. If it has 8 digits, then it
should store 0.83333333, but instead it shows a 7 as the last digit. This
isn't a rounding error, this is just wrong. How do I stop it from
showing/using the extra digits?

This doesn't even make sense. BY DEFINITION, the computer is forced to work within the
limits of its physical design. There is no "extra digit" being added; I have no idea why
you think it is an "extra" digit. It is NOT an "extra" digit, IT IS THE VALUE THE
COMPUTER HAS! If you ask the computer to print out a floating point number to maximum
precision, you are going to see the floating point number to its maximum precision, and
the number you are seeing is the ACTUAL, PRECISE VALUE THAT IS STORED IN THE COMPUTER.
That value is NOT 0.833,333,333,333,333,3 and it is not 0.833,333,333,333,333,33, it is
0.833,333,333,333,333,37. There is *NO* "extra" digit. This IS the value that was
computed. (Note that I put , separators in so you can count the digits!). There is
absolutely NO way you can represent 0.833,333,333,333,333,33 in 64-bit IEEE 754 floating
point.

You can represent
0.833,333,333,333,333,26
which is represented as the 64-bit hexadecimal number
0xAAAAAAAAAAAAA
and you can represent
0.833,333,333,333,333,37
which is represented as the 64-bit hexadecimal number
0xAAAAAAAAAAAAB

Note that these differ by 1 bit in the low-order position. There are no other
representations possible, and in particular, you cannot possibly find a bit pattern that
represents
0.833,333,333,333,333,33

You obviously asked to display the value to maximum decimal places. If you want to see
fewer decimal places, you would ask it to show the value to fewer decimal places. You
obviously did not do this, so you get to see the entire value, AS IT IS REPRESENTED IN THE
COMPUTER. I cannot emphasize enough that THERE IS NO EXTRA DIGIT BEING ADDED!

Do not think you are doing decimal arithmetic. Youa re not doing decimal arithmetic, and
having expectations that you are doing decimal arithmetic is why you are having
unrealistic expectations about what value you are seeing.

It is not wrong. It is absolutely correct, to within the limits of the design of IEEE 754
floating point arithmetic. There is no way you are going to get any other value, unless
you choose a completely different floating point chip on a completely different platform.
Oh yes, and if you do, you will STILL see phenomena like this. They will just be slightly
different due to different choices of how that particular platform's floating point
arithmetic works.
joe
****

Joseph M. Newcomer [MVP]
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
.

Relevant Pages

  • Re: wprintf() and large exact floats?
    ... It is almost always a mistake to think of floating point representations to a fixed number of decimals as "exact". ... The point is that you have no right to assume that arithmetic involving floating point representations can ever be exact. ...
    (microsoft.public.vc.language)
  • Re: problem with double
    ... While I understand that binary representations of ... floating point numbers are more efficient, ... that is less confusing to users than the fact that most of the short decimals ... a first-class data type. ...
    (comp.lang.java.programmer)
  • Re: Precision
    ... There is nothing wrong with the D3 floating point algorithm or anyone else's ... It's the same if we use Precision, floating point, or ... positions it is IMPOSIBLE to have a result of 4 decimals with the 2 ... remember what the associated actuarial calculations were. ...
    (comp.databases.pick)
  • Re: Precision
    ... There is nothing wrong with the D3 floating point algorithm or anyone else's ... It's the same if we use Precision, floating point, or ... positions it is IMPOSIBLE to have a result of 4 decimals with the 2 ... remember what the associated actuarial calculations were. ...
    (comp.databases.pick)
  • RE: Packed decimal arithmetic in C
    ... >many decimals to ... >> I once used double precision floating point in exactly that manner. ... >applications that track currency and need to do so exactly. ... >A scaled integer meets the requirement of exactness. ...
    (comp.os.vms)
Loading