Re: Double data Type Problem

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

Great tutorial on IEEE representation. I always knew the basic concept of
how they were stored (i.e., +/- Mantissa * 2^Exp), but never the precise bit
representation. Now I do...for all that it'll affect my daily life. :P


Rob

"Rick Rothstein (MVP - VB)" <rickNOSPAMnews@xxxxxxxxxxxxxxxxx> wrote in
message news:%238eKh6RUHHA.4632@xxxxxxxxxxxxxxxxxxxxxxx

"joc" <joc@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message
news:E17B8644-64F5-4594-BBF6-DB0F268F69FE@xxxxxxxxxxxxxxxx
I do not need advice on expanding my understanding of binary
representation
of floating point numbers. We all know that decimal numbers such as 0.1,
0.01, 0.001 etc cannot be exactly represented as binary fractions. Hence,
the
result shown in original post in this thread.
That is why I will round numbers to the number of decimal places that is
required my application.

There is just one caution to using the Round function... it performs its
calculation using something known as Banker's Rounding... if the number
ends in exactly 5 and you want to round to the position in front of the 5,
it rounds numbers down if the number in front of the 5's position is even
and rounds up otherwise (that is, numbers ending in 5 that are being
rounded to the previous decimal position are rounded towards the even
number).

Run the following

Debug.Print Round(1.2315, 3)
Debug.Print Round(1.2325, 3)
Debug.Print Round(1.2335, 3)
Debug.Print Round(1.2345, 3)
Debug.Print Round(1.2355, 3)
Debug.Print Round(1.2365, 3)

and the following prints out

1.232
1.232
1.234
1.234
1.236
1.236

Notice, Round will never produce a value that ends in an odd number if the
place being rounded from ends in exactly a 5. Also beware of CInt, Clng
and *almost* any other VB function that performs rounding as they too use
Banker's Rounding. The one exception -- Format. In place of

Round(YourNumber, Precision)

you can use for the general case

Format$(YourNumber, "0." & String$(Precision, "0"))

which looks somewhat intimidating. But note the simpler format in specific
cases. For example,

Round(YourNumber, 2)

becomes

Format$(YourNumber, "0.00")

Although the documentation for it is not as prominent as it should be, you
can find out about Banker's Rounding in the Remarks section of the Type
Conversion Functions entry in VB's help files. (And while it seems to deal
only with rounding from one decimal place to an exact integer, it actually
deals with any number whose decimal part ends in 5 which is being rounded
up one decimal place.) Here is the quote from that section:

"When the fractional part is exactly 0.5, CInt and CLng always round it to
the nearest even number. For example, 0.5 rounds to 0, and 1.5 rounds to
2. CInt and CLng differ from the Fix and Int functions, which truncate,
rather than round, the fractional part of a number. Also, Fix and Int
always return a value of the same type as is passed in."


The following links give some relevant information on floating point.
Although they are not specific to VB, they are relavant. (First 2 links
are
for Excel, third is for Python).
http://support.microsoft.com/kb/214118
http://support.microsoft.com/kb/78113/
http://effbot.org/pyfaq/why-are-floating-point-calculations-so-inaccurate.htm

Here are a couple of more references to add to the mix...

INFO: Visual Basic and Arithmetic Precision
http://support.microsoft.com/default.aspx?scid=http://support.microsoft.com:80/support/kb/articles/Q279/7/55.ASP&NoWebContent=1

(Complete) Tutorial to Understand IEEE Floating-Point Errors
http://support.microsoft.com/default.aspx?scid=http://support.microsoft.com:80/support/kb/articles/Q42/9/80.ASP&NoWebContent=1


Rick




.

Relevant Pages

  • Re: sprintf rouding error
    ... "uncorrectness" with floating point conversion leads to the ... zero"), it doesn't round. ... So there is no error at all in any internal representation of $n+0.5, ... power of 2, e. ...
    (comp.lang.perl.misc)
  • Re: is int(round(val)) safe?
    ... > round to the nearest integer by using int)? ... Since round() returns an integer, ... here) floating point values can handle *integer* values ... The problem* with floating point is inaccurate representation ...
    (comp.lang.python)
  • Re: Double data Type Problem
    ... of floating point numbers. ... 0.001 etc cannot be exactly represented as binary fractions. ... There is just one caution to using the Round function... ... calculation using something known as Banker's Rounding... ...
    (microsoft.public.vb.general.discussion)
  • Re: This calculation is just wrong / computer cant count!
    ... floating point binary number with a 52-bit mantissa, you can only deal with roundoff ±1 ... The only rounding representation possible is in binary, ... you would just download the Floating Point Explorer and play with it you would see how ...
    (microsoft.public.vc.mfc)
  • Re: is int(round(val)) safe?
    ... >> I suspect it is fine, but wanted to be sure that weird floating point ... >> representation on some platform might make it unsafe there (i.e. get the ... >Since round() returns an integer, ... doesn't matter that it's to the right of the new LSB of 2.0**53 ). ...
    (comp.lang.python)