Re: Field changed from Single to Double issue..
anonymous_at_discussions.microsoft.com
Date: 08/11/04
- Next message: Jeremy Nelson: "How to truncate the leading zeros"
- Previous message: Kumar H.D: "access conversation & installation"
- In reply to: Chris Mills: "Re: Field changed from Single to Double issue.."
- Next in thread: anonymous_at_discussions.microsoft.com: "Re: Field changed from Single to Double issue.."
- Reply: anonymous_at_discussions.microsoft.com: "Re: Field changed from Single to Double issue.."
- Messages sorted by: [ date ] [ thread ]
Date: Wed, 11 Aug 2004 06:23:42 -0700
If I look at the single field, it says "0.3", period,
not "0.30000001" or anything like that -- it was entered
by hand as 0.3 so there is no precision issue with it.
However, after the conversion from single to double, it
does say "0.3000001192" (regardless of the exact number of
zeroes after the 3). This just doesn't make sense unless
the single was also 0.3xxxxxxxxxxxx, imo, and it wasn't..
I really don't want to code anything as I've got like 200
fields in this situation and they are all doing this sort
of thing -- sigh!
>-----Original Message-----
>I think you're missing two zeros in the precision there...
>
>It's well-known that binary can only represent decimal to
it's level of
>precision.
>
>For a table field (single), 0.30000001 (6 zeros) seems to
run off the scale so
>get's truncated (or in your case vice-versa). But
actually a single can store
>much higher precision than that (13 zeros in the example)
according to tests
>on a "Dim s as Single". And both a single and double in
code present 0.3 as
>0.3, so it's confusing isn't it?
>-----
>Still, I s'pose you wanted a resolution...
>
>I think you've found a limitation of the way table
conversions work. It may be
>a "bug" but regardless it's a fact.
>
>You could maybe do your conversions via a Recordset. I
haven't tested that,
>just that conversions in code seem to work better.
>
>If it's critical (like for your accountant), see if one
of the PRECISE formats
>suits, like integer(multiplied) or currency.
>
>It's dangerous to use "=" in comparing floating point
numbers, which are
>unlikely to be to infinite precision (though I see in
code sample comparisons
>it says "near enough" at some point. You can use >=0.3 or
maybe
>Format(single,"blah-blah")=0.3 or some other rounding
truncation to control
>the precision.
>
>(tested A2000 and A2002)
>Chris
>
>"DN" <anonymous@discussions.microsoft.com> wrote in
message
>news:3b4f01c47f28$35729e60$a501280a@phx.gbl...
>> I have an Access 2000 DB (I think it's 2000, it's
>> whatever comes with Visual Studio 6.0).. Simply put, I
>> have a field that is of type "Single" that contains the
>> value 0.3. When I modify that field and make
>> it "Double", access automatically converts the values,
as
>> it should, but returns 0.300001192 in the field.. I
>> understand that there can be precision issues sometimes
>> in comparisons and such, but is this NORMAL for Access
to
>> do as well? Is there any simple work around or patch?
I
>> suppose I can write an SQL statement to take care of
>> this, but that isn't something I want to do as there are
>> many fields. . . Thanks!
>
>
>.
>
- Next message: Jeremy Nelson: "How to truncate the leading zeros"
- Previous message: Kumar H.D: "access conversation & installation"
- In reply to: Chris Mills: "Re: Field changed from Single to Double issue.."
- Next in thread: anonymous_at_discussions.microsoft.com: "Re: Field changed from Single to Double issue.."
- Reply: anonymous_at_discussions.microsoft.com: "Re: Field changed from Single to Double issue.."
- Messages sorted by: [ date ] [ thread ]
Relevant Pages
|
