Re: This calculation is just wrong / computer can't count!

On 4 Okt., 17:28, "GT" <ContactGT_remove...@xxxxxxxxxxx> wrote:
".rhavin grobert" <cl...@xxxxxxxx> wrote in message

news:1191511188.715360.292900@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

On 4 Okt., 17:00, "GT" <ContactGT_remove...@xxxxxxxxxxx> wrote:
".rhavin grobert" <cl...@xxxxxxxx> wrote in message
On 4 Okt., 16:24, "GT" <ContactGT_remove...@xxxxxxxxxxx> wrote:
What do you mean by the number might be below double granularity on the
computer? What can I do about this?

your computer stores floatingpoints (float, double, long double) in a
format like

[sign][exponent][fraction]. that means that it can distinguish between
0.0000003 and 0.0000004 but perhaps not between 700000.0000003 and
700000.0000004. on the other hand, it can handle numbers up to
3.4028235*(10^38) for a 32-bit floating point variable.

But if the computer stores numbers in this format, then why does it add an
extra digit on the end that it can't handle and more importantly - how do I
stop it from adding the extra digit on the end?

it doesnt "add" it.

If you do...
int a = 1.34;

and then look into the memory at &a you'll find a "1", because the
compiler does a

int a = (int) 1.34; => int a = 1;

because the constant is first cast into the appropriate value (1 for
int, because granuarlity of int is 1) and then moved into memory at
&a. For doubles, thats the same. But doubles have granuarlity
antiproportional to absolute (unsigned) size. You can imagine it as a
"floating point" in decimals: guess you have (for example) 10 digits,
a sign (on= + /off = -) and something that states: put decimal point
here...

if you do a = 0.123456789, then it would be printed as 0.123456789, if
you do a = 0.1234567895 than it would be also printet as 0.123456789
because you're below granuarlity.

if you do a a = 1234567890 ten wit will be printed as 1234567890, if
you substact 0.4 then you're below granuarlity and may get 1234567890
or 1234567889 depending on your system.

.

Loading