Re: float vs. double?


"Robert Wong" <robertwong@xxxxxxxxxxx> wrote in message
news:OK%23gpmGJHHA.1252@xxxxxxxxxxxxxxxxxxxxxxx

I wasn't sure if it was a code generation problem. Also kind of weird is
depending on the variable type, is different precision being stored?

Of course - "double" is called "double" because it stores about twice as
many significant figures as "float".

FORTRAN is a bit more verbose "float" = "REAL" and "double" = "DOUBLE
PRECISION".

The same multplication is used.

answer = value * lsb; // debugger - 1004714.7 float
danswer = value * lsb; // debugger - 1004714.7080078125 double

It is also worth noting that multiplication essentially preserves the number
of significant figures - the result will be accurate to the number of
significant figures as the less accurate of the multiplicands. [And note
that integers are first converted to float or double when multiplied by
float or double.]

But lesson number 1, which I learned in FORTRAN IV in my first lesson (in
1968), is "never use single precision" as you're laying yourself open to
errors which can be mitigated just by using double precision. Since then I
don't think I have ever done any floating point computation in single
precision - and these days, as I say, (because of me, obviously, and
possibly the other David too <g>) Mr Intel designs his chips to work most
efficiently in double precision.

If you're going to be doing a lot of floating point computation, I'd
seriously recommend looking at a textbook or two on numerical maths. [I'd
recommend those on my shelves but they've probably been out of print for 30
or 40 years.]

Dave
--
David Webber
Author MOZART the music processor for Windows -
http://www.mozart.co.uk
For discussion/support see
http://www.mozart.co.uk/mzusers/mailinglist.htm




.

Relevant Pages

  • Re: How wrong can you be with your NPV and MIRR functions in EXCel
    ... "Harlan Grove" wrote in message ... I always build up those calculations using simple ... > almost always retain greater precision and accuracy than any ... rounded to three or four significant figures depending on the client) ...
    (microsoft.public.excel.worksheet.functions)
  • Re: Excel Bug - Excel Geeks Unite!!
    ... Will round all values to cell format precision. ... Rounding to a specified number of significant figures does not necessarily solve the problem (see my previous decimal example involving ... That situation was potentially frustrating to serious numerical analysts, since it meant that there was far more precision available than you were allowed to use. ... Given the nature of this issue, I would tend to distrust any current package that does not exhibit the normal effects of finite precision binary approximation, unless the reason is clearly documented. ...
    (microsoft.public.excel.misc)
  • Floating point errors in finite difference calcs.
    ... significant figures of the H's to cancel out if H is order 1. ... left for usual double precision real variables. ...
    (sci.math.num-analysis)
  • Precision vs. Accuracy and Significant Figures
    ... This brings up the topic of precision. ... There are formal rules around implementing significant figures. ... If Java's API includes support for rounding, ... number of significant figures needs consideration when computing the answer. ...
    (comp.lang.java.advocacy)
  • Re: about c to fortran
    ... > more than 16 significant figures of precision. ... Where d is the number of decimal digits you write and ... no deficiencies and the other way is to make it so complicated ...
    (comp.lang.fortran)
Loading