Re: This calculation is just wrong / computer can't count!

Tech-Archive recommends: Fix windows errors by optimizing your registry

Windows calculator is not using 'double'. Try and see how big a factorial it
allows you to calculate. It's beyond FP math.

"AliR (VC++ MVP)" <AliR@xxxxxxxxxxxxx> wrote in message
news:R18Ni.30858$eY.24784@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
I'm not sure yet. I'm search on google for it. I see that windows
calculator program does exactly that. It only shows the significant
digits of a double. So it has to be doable.

AliR.


"GT" <ContactGT_removeme_@xxxxxxxxxxx> wrote in message
news:00238cdd$0$5102$c3e8da3@xxxxxxxxxxxxxxxxxxxx
"AliR (VC++ MVP)" <AliR@xxxxxxxxxxxxx> wrote in message
news:En7Ni.30834$eY.28056@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Maybe this will clear some of it up:
http://www.google.com/search?q=Significant+digits+double&hl=en

That info on significant digits was all very interesting, but how do I
get my code to work properly?!? I want to force the computer to calculate
and store as many digits as it can handle and no extra spurious digits,
so that my calculations get nice accurate results. At the moment part of
my calculation ends up with -0.00000000000007 (haven't counted the zeros
here, but you get the idea!), where it should be zero.





.

Relevant Pages

  • Re: .99999... still=/= 1
    ... > A scientific calculator does approximate convergence with a certain ... > number significant digits. ... i.e. any partial sum, or do you mean the whole series, which is defined ... The expression you wrote with it stopping after the mth term ...
    (sci.math)
  • Re: Thatll Teach Em - Long Division
    ... if only so you are not totally stumped if your calculator fails. ... I remember a teacher when I was at school in the 1970s whose standard ... punishment for any misdemeanour was "cubes". ... boys to call out digits (the number of digits depended on the severity of ...
    (uk.media.tv.misc)
  • HP30S numerical anomaly
    ... I posted about a peculiar HP30S behavior over on MoHP, ... Recently while looking at the calculator forensics web site, ... functions can give substantially less than 24 digits; ...
    (comp.sys.hp48)
  • Re: FAQ 4.2 Why is int() broken?
    ... is much of an issue - 8 or 12 digits are enough. ... For example on my HP-48 calculator. ... can display 0.999999999999 exactly, ...
    (comp.lang.perl.misc)
  • Re: Math errors in python
    ... They don't even share three digits beyond the decimal point. ... > yet not all fractions can be represented by decimals. ... Hardware designers have settled on a binary representation of floating ... And BTW, your calculator is not, in general, more accurate than the ...
    (comp.lang.python)