Re: addition of 0.7 and 0.1 is not 0.8 CDbl / CStr vbs problem

On Thu, 21 Jun 2007 10:01:46 -0500, Anthony Jones <Ant@xxxxxxxxxxxxxxxx> wrote:


<robin@xxxxxxxxx> wrote in message
news:1182436865.902509.220780@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Hello Group,

i have some vbs code here and would be happy if one of you could
describe whats wrong with it. i just would like to make addition of
0.1 and 0.7 but the result just surprises me everytime again.

Comparing floating point values should be done using a tolerance :-

If Abs(CDbl("0.7") + CDbl("0.1") - CDbl(0.8)) < 0.000001 Then
'Near enough
Else
'Not near enough
End If

Another option would be to use CCur instead of CDbl. I wasn't able to
reproduce the behavior described with 0.7 and 0.1, so I have substituted
numbers that would exceed the precision of the VBScript double:

Option Explicit

WScript.Echo CDbl(100000000000000) + CDbl(.1)
WScript.Echo CCur(100000000000000) + CCur(.1)

The Currency data type is fixed to four digits of precision and has a
range of -922,337,203,685,477.5808 to 922,337,203,685,477.5807.

--
Justin Piper
Bizco Technologies
http://www.bizco.com/
.

Loading