Re: Phantom decimal places

From: Allen Browne (AllenBrowne_at_SeeSig.Invalid)
Date: 10/14/04 

Date: Fri, 15 Oct 2004 00:52:13 +0800

These inaccuracies are inherent in floating point numbers.

Some numbers (e.g. 1/3) cannot be represented accurately as a decimal, as
they require an infinite number of decimal places. Your computer can provide
around 7 or 8 accurate places (for a Number of size Single) or double that
(for a Number of size Double).

If you do not need more than 4 decimal places, you can use a Currency type
instead. Currency is a fixed-point type, not a floating-point type, and so
it avoids the rounding problems.

If you need more places, consider using a Double rather than a single, and
rounding the results of your calculations using the Round() function so the
small errors are not aggregated. 

-- 
Allen Browne - Microsoft MVP.  Perth, Western Australia.
Tips for Access users - http://allenbrowne.com/tips.html
Reply to group, rather than allenbrowne at mvps dot org.
"Brian" <Brian@discussions.microsoft.com> wrote in message
news:F5002622-38A0-4F6F-9AE3-7B8B8B30838F@microsoft.com...
>I have field defined as number/single. When performing simple DLookups 
>an/or
> simple calculations on them and then displaying the result in text boxes, 
> it
> is often displayed with extra digits. For example, an entry that was
> calculated as 30 - 30.22 shows -1.7800007.
>
> The text box itself has decimal places set to Auto (which is correct,
> because it could vary) and no format, although General Number format 
> produces
> the same result. The result is also the same whether or not the format in 
> the
> table is blank or General Number.
>
> How do I get numbers to display the correct data without the ghost decimal
> places?

Relevant Pages

  • Re: 32-bit IEEE float multiplication
    ... > when I multiply a huge floating point value by a really ... displaying it with something like a printf "%f" format, ... Try using printf's "%g" format for output, ...
    (comp.lang.c)
  • Phantom decimal places
    ... simple calculations on them and then displaying the result in text boxes, ... because it could vary) and no format, ... How do I get numbers to display the correct data without the ghost decimal ...
    (microsoft.public.access.forms)
  • Re: converting float to ascii w/o printf
    ... Currently I am using sprintfto format this output into ... the rule here is never use floating point at all if it can be avoided. ... If you know what the range of values and required accuracy is, scaled integer arithmetic will be faster and more efficient, especially if you can arrange the data to allow shifts, rather than multiply and divide. ... Floating point on small machines is slow, inefficient, adds complexity and rounding errors everywhere it's used and you can usually get better control and predictable accuracy using other methods. ...
    (comp.arch.embedded)
  • Re: Price Data Formatting
    ... that the data format is NOT KNOWN during design time but is based on the format of the data loaded during runtime. ... In other words, perhaps you should be looking for the MAXIMUM number of decimal digits in the data items, ignoring "odd balls" that have a lot more than the average. ... The other thing we have asked that you haven't yet provided an answer to is *where are you displaying the resultant output*. ...
    (microsoft.public.vb.general.discussion)
  • Re: converting float to ascii w/o printf
    ... Currently I am using sprintfto format this output into ... this is using simple multiplies and ... check if you have the ftoa function available. ... Do you really need to store the information in floating point format? ...
    (comp.arch.embedded)
Loading