Re: This calculation is just wrong / computer can't count!

On Sat, 6 Oct 2007 21:00:02 -0700, "David Ching"
<dc@xxxxxxxxxxxxxxxxxxxxxx> wrote:

I want to ask: can anyone recommend a library that produces results like a
calculator (I believe it is called fixed precision decimal)? I understand
that the floating point types in C++ are not meant for this, but what search
terms do I google to find a library that offers this functionality?

I had thought BCD (binary coded decimal) would be a good start, but there
isn't anything on sourceforge, for example that has this....

Thanks,
David


"fixed point math library"
.

Relevant Pages

  • Re: This calculation is just wrong / computer cant count!
    ... can anyone recommend a library that produces results like a ... calculator? ... isn't anything on sourceforge, ... David ...
    (microsoft.public.vc.mfc)
  • Re: invalid calculation on double data type
    ... Going back to the OP's issue with a calculator not showing the same problem ... as floating point, it does show a totally analogous problem if you do (4 / ... calculators cannot represent all fractions with total precision, ... You can't avoid the precision errors due to limited memory, ...
    (microsoft.public.vb.general.discussion)
  • Re: invalid calculation on double data type
    ... Going back to the OP's issue with a calculator not showing the same ... as floating point, it does show a totally analogous problem if you do (4 / ... (insufficient precision when limited to 5 ... You can't avoid the precision errors due to limited memory, ...
    (microsoft.public.vb.general.discussion)
  • Re: invalid calculation on double data type
    ... "Tony Proctor" skrev i ... Going back to the OP's issue with a calculator not showing the same problem ... as floating point, it does show a totally analogous problem if you do (4 / ... calculators cannot represent all fractions with total precision, ...
    (microsoft.public.vb.general.discussion)
  • Re: Lack of deep shadow detail in digital images
    ... >>> David J Taylor ... >>> Personnally I prefer single precision IEEE floating point ... > you have to come up with 128 MHz A/D converters. ...
    (rec.photo.digital)