Re: i know it's a floating-point imprecision...

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

From: Steve Kass (skass_at_drew.edu)
Date: 01/26/05 

Date: Wed, 26 Jan 2005 14:50:25 -0500

Ian,

  EM and ADO don't "fix" the precision, really. Unlike Query Analyzer,
those tools display floating point data with less precision. Where Query
Analyzer displays the closest 17-digit decimal value to the stored float,
other tools will display the closest 15- or 16-digit decimal value. No
front
end that I'm aware of displays the exact decimal equivalent of stored
floats.

  You might store a float like pi() and see what each of these shows you.

Steve Kass
Drew University

Ian Boyd wrote:

>...but why is it fixing my floating-point imprecision?
>
>Running the following query:
> SELECT Value
> FROM Values
>
>On the table
> CREATE TABLE Values (Value float(53))
>
>In Query Analyzer:
> Value
> -----------------------------------------------------
> 1.3899999999999999
>
> (1 row(s) affected)
>
>
>In Enterprise Manager (Open table with query)
> Value
> 1.39
>
>In ADO, converting variant field to string:
> 1.39
>
>
>I understand that the value is stored in SQL Server using floating points.
>And that floating points cannot accuratly represent some/most numbers.
>But how is it that QA is able to show me the imprecision inherient in
>floating points; while EM and ADO can transparently "fix" it?
>
>What layer in the database connectivity sytem is able to "fix" my values?
>Why am i not allowed to see the approximate values in EM and ADO?
>
>
>Don't get me wrong, it's nice that ADO fixes it. I just want to know why and
>how.
>
>
>
>

Relevant Pages

  • Re: Sum Expression gives 1.45519152283669E-11 instead of 0 (zero)
    ... That is not due to Access itself, but to what is a floating point number. ... AssetHolding: Sum*CreditDebit*CCur) ... In the query indeed the AssetHolding is presented as 0 (or ... TransactionQuantity bought is equal to the sum of TransactionQuantity ...
    (microsoft.public.access.queries)
  • Re: weird problem
    ... I already told you that the comparison between an integer and a float ... And now a question about something else: why do you use floating ... use then to copy a float into a char *1. ... binary representation doesn't resemble a string like "123.46343" ...
    (comp.lang.c)
  • Re: Safer of 2 methods for very long doc
    ... I use wrapped objects so seldom that I have to think about how I've handled that when I have used them, and I guess, as you say, I've moved the anchor. ... paragraph and the picture move to the next page. ... in Word the text does not flow from after a 'floating' ... >I'd appreciate knowing how you "float" a table, ...
    (microsoft.public.word.docmanagement)
  • Re: float vs. double?
    ... There have been repeated myths that float is faster than double. ... In antique machines, the ... Note that the floating-point ALU of Intel chips supports an 80-bit floating point number, ... is different precision being stored? ...
    (microsoft.public.vc.mfc)
  • Re: Rounding of the double
    ... I still think you can do what you want using DWORDs and just assume the decimal position rather than using floats of doubles. ... I think you're going to have the same problem in any language as the floating point technology is not specific to C++ or unmanaged C++. ... /* Round a double or float to 'sig' places to the left or right of the decimal. ... As Joe says "there is no accurate representation of 0.80 in IEEE floating ...
    (microsoft.public.vc.mfc)