Re: This calculation is just wrong / computer can't count!

Tech-Archive recommends: Fix windows errors by optimizing your registry

".rhavin grobert" <clqrq@xxxxxxxx> wrote in message
news:1191599292.427080.100770@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
On 5 Okt., 11:38, "GT" <ContactGT_remo...@xxxxxxxxxxx> wrote:
I would like to know how to use the basic C++ datatypes and where the
last
digit is 'wrong' or 'unreliable', just don't use it. Everyone keeps
saying
'just use the digits you can rely on', but how? I have a double
effortChangeProportion which has 15 or 16 decimal places. HOW do I tell
it
to just use the first 10 decimal places, thereby dropping the final digit
which is knocking out my calculations? I can't find any methods on a
double
which tell it to truncate digits.

You cant tell yout computer "drop the last x digits in decimal system"
when your number is stored in BINARY.
if you're interested in the last 5 digits of 0.444444443094876 _as a
number_ then you should do a

int iResult = (0.444444443094876 * 10000);
then do the rest of your calculation and divide by 10000 at the end
(according to your function you of course may do more than this).

Thanks, but I have tried this, but the divide by 10000 then re-introduces an
erroneous digit on the end of the number, so gains me nothing!

if you're interested in the last 5 digits of 0.444444443094876 _as a
string_ then you should simply do a

CString str;
str.Format("%.5f", 0.444444443094876 );

rhavin;)



.

Relevant Pages

  • Re: This calculation is just wrong / computer cant count!
    ... The guard bits answer has nothing to do with answering your question. ... that these conversions introduce. ... but the final digit is a 3. ... how do I get my calculations to work as accurately as the Windows ...
    (microsoft.public.vc.mfc)
  • Re: An uncountable countable set
    ... but it may depend on the ordering of the calculations. ... the diagonal number it does not matter what digit you calculate first. ... to which every path will lead is always the same: The set of rationals ... You state, explicitly, that they were not binary representations, ...
    (sci.math)
  • Re: This calculation is just wrong / computer cant count!
    ... but ONLY BECAUSE OF THE DIGIT HE THINKS IS WRONG!!!!! ... and they correctly reflect the calculations ... point of calculating something that should be zero when we already know with ... Obviously, GT must have some reason for making these calculations, but he ...
    (microsoft.public.vc.mfc)
  • Re: This calculation is just wrong / computer cant count!
    ... extra digit on the end that it can't handle and more importantly - how ... "floating point" in decimals: guess you have 10 digits, ... The computer should surely only store as ... rounding at the last stage, so my final calculations are rounded ...
    (microsoft.public.vc.mfc)
  • Re: This calculation is just wrong / computer cant count!
    ... extra digit on the end that it can't handle and more importantly - how ... "floating point" in decimals: guess you have 10 digits, ... The computer should surely only store as ... rounding at the last stage, so my final calculations are rounded ...
    (microsoft.public.vc.mfc)