Re: Round off error
- From: JE McGimpsey <jemcgimpsey@xxxxxxxx>
- Date: Mon, 26 Feb 2007 10:51:59 -0700
Hmmm...
For me, I get zero in the third step, not a very small number. However
the principle is correct - rounding errors occur when you use floating
point math.
You could force the zero with
Public Sub Test2
Dim i As Long
Dim di as Double
For i = -2 to 2
di = i/10
debug.print i, i/10
Next i
End Sub
Ultimately, though, the only way to prevent all rounding errors is to
use only integer math. Most numbers can't be exactly represented in
binary, just as most numbers can't be exactly represented in decimal.
In article <55C6C587-ACF7-4BA2-920A-F1ED7C380FD6@xxxxxxxxxxxxx>,
Harvey <Harvey@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote:
Okay try to run this code in VBA and see the reults in immediate window..
sub test
Dim i as double
for i= - 0.2 to 0.2 step 0.1
debug.print i
next
End sub
you must get "0" in the third step but you get a very small number well
this is a very common and known round up error but I want to know is there
any quick fix or trick to eliminate this error in VBA code?
Thanks:)
- Prev by Date: Re: ComboBox with multiple criteria
- Next by Date: Re: Looping to insert photo
- Previous by thread: Re: Round off error
- Next by thread: RE: Round off error
- Index(es):
Relevant Pages
|
