Re: This calculation is just wrong / computer can't count!

Tech-Archive recommends: Fix windows errors by optimizing your registry

".rhavin grobert" <clqrq@xxxxxxxx> wrote in message
news:1191601477.742676.224270@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
On 5 Okt., 18:01, "GT" <ContactGT_remo...@xxxxxxxxxxx> wrote:
".rhavin grobert" <cl...@xxxxxxxx> wrote in message

news:1191599292.427080.100770@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

On 5 Okt., 11:38, "GT" <ContactGT_remo...@xxxxxxxxxxx> wrote:
I would like to know how to use the basic C++ datatypes and where the
last
digit is 'wrong' or 'unreliable', just don't use it. Everyone keeps
saying
'just use the digits you can rely on', but how? I have a double
effortChangeProportion which has 15 or 16 decimal places. HOW do I
tell
it
to just use the first 10 decimal places, thereby dropping the final
digit
which is knocking out my calculations? I can't find any methods on a
double
which tell it to truncate digits.

You cant tell yout computer "drop the last x digits in decimal system"
when your number is stored in BINARY.

if you're interested in the last 5 digits of 0.444444443094876 _as a
number_ then you should do a

int iResult = (0.444444443094876 * 10000);
then do the rest of your calculation and divide by 10000 at the end
(according to your function you of course may do more than this).

Thanks, but I have tried this, but the divide by 10000 then re-introduces
an
erroneous digit on the end of the number, so gains me nothing!


it cant "introduce an erroneous digit". If you do a 1/3 then stating
"1/3" or "3^(-1)" are the only exact results.
stating "0.33333333" is not wrong, it's just not exact. the same
applies to "0.33333334" or even

I disagree:
0.3 is accurate (correct), but not very precise - only 1 decimal place
0.333 is accurate (correct) and precise to 3 decimal places
0.33333333 is accurate (correct) and precise to 8 decimal places
0.33333334 is inaccurate (*WRONG*) and precise to 8 decimal places


.

Relevant Pages

  • Re: Can anyone improve this any further......
    ... Some machines only have a "native" signed integer division, ... suppose the implementation rounds towards -inf. ... Then rest is set to -3277 and j is 2, and converting j to a digit ... required to make hardware FP divide fast, ...
    (comp.lang.c)
  • Re: DECIMAL
    ... My early Intel 4004-based handheld calculator performed floating ... decimal add, subtract, multiply and divide. ... to "support" one digit decimal operations. ...
    (comp.lang.pl1)
  • Re: DECIMAL
    ... No more difficult than multiple-word binary integer division. ... decimal add, subtract, multiply and divide. ... to "support" one digit decimal operations. ... divisor and decimal with a one digit divisor. ...
    (comp.lang.pl1)
  • Re: DECIMAL
    ... but a lot harder for divide. ... divisor and decimal with a one digit divisor. ... by 32 bits is often enough, two digit by one digit rarely. ... Of course, as the 8088 has decimal division, the first step ...
    (comp.lang.pl1)
  • Re: math problem
    ... Notice that the last digit of each line are identical. ... its last digit would be to divide by 4, ... Therefore, since there's no remainder, the last ... and I would be very grateful for a proof (or disproof). ...
    (sci.math)