Re: This calculation is just wrong / computer can't count!

Tech Tip: Click here to run a free scan for Windows Errors and optimize PC performance

On 5 Okt., 18:01, "GT" <ContactGT_remo...@xxxxxxxxxxx> wrote:
".rhavin grobert" <cl...@xxxxxxxx> wrote in message

news:1191599292.427080.100770@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

On 5 Okt., 11:38, "GT" <ContactGT_remo...@xxxxxxxxxxx> wrote:
I would like to know how to use the basic C++ datatypes and where the
last
digit is 'wrong' or 'unreliable', just don't use it. Everyone keeps
saying
'just use the digits you can rely on', but how? I have a double
effortChangeProportion which has 15 or 16 decimal places. HOW do I tell
it
to just use the first 10 decimal places, thereby dropping the final digit
which is knocking out my calculations? I can't find any methods on a
double
which tell it to truncate digits.

You cant tell yout computer "drop the last x digits in decimal system"
when your number is stored in BINARY.

if you're interested in the last 5 digits of 0.444444443094876 _as a
number_ then you should do a

int iResult = (0.444444443094876 * 10000);
then do the rest of your calculation and divide by 10000 at the end
(according to your function you of course may do more than this).

Thanks, but I have tried this, but the divide by 10000 then re-introduces an
erroneous digit on the end of the number, so gains me nothing!


it cant "introduce an erroneous digit". If you do a 1/3 then stating
"1/3" or "3^(-1)" are the only exact results.
stating "0.33333333" is not wrong, it's just not exact. the same
applies to "0.33333334" or even
"0.33333333333333333333333333333333333333333333333333333333333333333"

if you want a "user friendly" output, then you have to format a
string, if you want a result your programm can live with, you have to
code your programm according to your needs. If you need to know wether
toe result comes near zero, you can do something like

bool NearZero(double db)
{
return ((db*db) < 0.00001);
}

rhavin;)

.

Relevant Pages

  • Re: This calculation is just wrong / computer cant count!
    ... digit is 'wrong' or 'unreliable', ... Thanks, but I have tried this, but the divide by 10000 then re-introduces ... erroneous digit on the end of the number, ... stating "0.33333333" is not wrong, ...
    (microsoft.public.vc.mfc)
  • Re: An uncountable countable set
    ... once give a proper definition? ... That every digit can be indexed does not require that the ... You state so, and remain stating so, without any proof. ... number indexability to every digit in an endless string of digits. ...
    (sci.math)
  • Re: This calculation is just wrong / computer cant count!
    ... If you stop saying "introduces an erroneous digit" and replace it with "I am incapable of ... digit is 'wrong' or 'unreliable', ... then do the rest of your calculation and divide by 10000 at the end ... erroneous digit on the end of the number, ...
    (microsoft.public.vc.mfc)
  • Re: This calculation is just wrong / computer cant count!
    ... I am just doing a series of basic calculations. ... Using floating point doesn't help - it just brings the erroneous digit ...
    (microsoft.public.vc.mfc)