Re: This calculation is just wrong / computer can't count!

".rhavin grobert" <clqrq@xxxxxxxx> wrote in message
news:1191511188.715360.292900@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
On 4 Okt., 17:00, "GT" <ContactGT_remove...@xxxxxxxxxxx> wrote:
".rhavin grobert" <cl...@xxxxxxxx> wrote in message
On 4 Okt., 16:24, "GT" <ContactGT_remove...@xxxxxxxxxxx> wrote:
What do you mean by the number might be below double granularity on the
computer? What can I do about this?

your computer stores floatingpoints (float, double, long double) in a
format like

[sign][exponent][fraction]. that means that it can distinguish between
0.0000003 and 0.0000004 but perhaps not between 700000.0000003 and
700000.0000004. on the other hand, it can handle numbers up to
3.4028235*(10^38) for a 32-bit floating point variable.

But if the computer stores numbers in this format, then why does it add an
extra digit on the end that it can't handle and more importantly - how do I
stop it from adding the extra digit on the end?


.

Loading