Re: i know it's a floating-point imprecision...

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

From: Ian Boyd (ian.msnews010_at_avatopia.com)
Date: 01/26/05 

Date: Wed, 26 Jan 2005 16:23:39 -0500

i'm using float to store a fixed point (6 decimal places) exchange rate.

i would use decimal(9,6), but Delphi does not support fixed point (other
than 4 decimal places money fields) properly. Essentially it assmes that all
BCD fields are money.

So i was trying to go past Delphi's ADO wrapper and into ADO and it's
variants. And while checking how ADO handles floats and decimals, i
discovered this oddity.

"Adam Machanic" <amachanic@hotmail._removetoemail_.com> wrote in message
news:ekOdm%239AFHA.1292@TK2MSFTNGP10.phx.gbl...
> Looks like EM and ADO are both casting the value to some fixed precision
> format... You can almost certainly control that with ADO somehow (change
> the
> return type first), but with EM you probably have no choice... What are
> you
> using the floating point datatype for?
>
>
> --
> Adam Machanic
> SQL Server MVP
> http://www.sqljunkies.com/weblog/amachanic
> --
>
>
> "Ian Boyd" <ian.msnews010@avatopia.com> wrote in message
> news:%23OAGY49AFHA.3336@TK2MSFTNGP11.phx.gbl...
>> ...but why is it fixing my floating-point imprecision?
>>
>> Running the following query:
>> SELECT Value
>> FROM Values
>>
>> On the table
>> CREATE TABLE Values (Value float(53))
>>
>> In Query Analyzer:
>> Value
>> -----------------------------------------------------
>> 1.3899999999999999
>>
>> (1 row(s) affected)
>>
>>
>> In Enterprise Manager (Open table with query)
>> Value
>> 1.39
>>
>> In ADO, converting variant field to string:
>> 1.39
>>
>>
>> I understand that the value is stored in SQL Server using floating
>> points.
>> And that floating points cannot accuratly represent some/most numbers.
>> But how is it that QA is able to show me the imprecision inherient in
>> floating points; while EM and ADO can transparently "fix" it?
>>
>> What layer in the database connectivity sytem is able to "fix" my values?
>> Why am i not allowed to see the approximate values in EM and ADO?
>>
>>
>> Don't get me wrong, it's nice that ADO fixes it. I just want to know why
> and
>> how.
>>
>>
>
>

Relevant Pages