Re: Singles to Doubles

Here is my guess... and it is only a GUESS as I have never been one to "get under the hood" of VB. I'm guessing that VB calculates its Singles and Doubles using some kind of power series expansion (I'm guessing that is what is at the heart of FFT which I think is what is used)... for Singles, it stops iterating the series after X number of loops whereas for Doubles it continues iterating past X until Y loops have completed. This would explain why Doubles are more accurate than Singles (more iterations of the series expansion). Now, to continue my guessing, I think 15 (maybe 16) digits were calculated for the Single and stored; but, because VB knows it is a Single, it only reports 6 of 7 digits when asked. My further guess is that CDbl simply reveals the underlying 15 or 16 digits that were generated when the Single was calculated; hence, a not so accurate value. Now, for the CStr conversion, I think VB is grabbing only the 6 or 7 digits it would normally use for display (I'm thinking the same "text engine" is underneath the displayer as the string converter because, well, text is text, so why should they be different). These 6 or 7 digits are not burdened by the inaccurate excess digits that Single calculated, and so are more nearly equal to the original inputted value. Now, that nearly pure number is being fed into the CDbl series expansion engine (just like if you typed it in) producing a near perfect conversion. Anyway, that is my guess at what is going on.

Rick



"Karl E. Peterson" <karl@xxxxxxxx> wrote in message news:uldbmN%23xHHA.4652@xxxxxxxxxxxxxxxxxxxxxxx
Hi Bob --

What the heck? Seems silly to have to convert it to a String first, then
to a Double. But, it is more accurate. Anyone know of a less ugly way to
accurately replicate a Single with a Double, or this the best it gets?

Isn't it mainly that the Single isn't representing the value as accurately?

I guess, yeah. But how is it that a *String* seems to do so even more accurately? For discussion purposes, I modified your example just a bit:

Public Sub Main()
Dim s As Single
Dim d1 As Double
Dim d2 As Double
s = 0.3026!
d1 = s
d2 = CDbl(CStr(s))
Debug.Print s, d1, d2
Debug.Print s = d1, s = d2
Debug.Print s - d1, s - d2
End Sub

Which gives me these results:

0.3026 0.302599996328354 0.3026
True True
0 -3.67164609826887E-09

And, of course, the value I'm after *is* 0.3026 -- not the one that's off 0.00000000367... Seems like some sort of register garbage that I'm picking up in the conversion?

Thanks... Karl
--
.NET: It's About Trust!
http://vfred.mvps.org


.

Relevant Pages

  • Re: How to get decimal form of largest known prime?
    ... but I learned from it much about Python syntax. ... here code of a bigDecReprOfInt class as a kind of module and ... i.e. 4096 and not as I would expect it 80 decimal digits long ... # and strconversion for very large integers i. ...
    (comp.lang.python)
  • Re: Any progress yet? (was Re: Fast pi program?)
    ... conversion to base 26, ... Base 2^32 would /reduce/ the number of digits by a factor of about 1.7 ... numbers approximated with floating point is even worse.), the "pyramid ... it does cancel the performance gain if there was any. ...
    (comp.programming)
  • Re: Any progress yet? (was Re: Fast pi program?)
    ... conversion to base 26, ... you need to compute 2x the digits then convert to 26 (since the ... If you pack 4 elements and you use 8 primes ... numbers approximated with floating point is even worse.), the "pyramid ...
    (comp.programming)
  • Re: %x question
    ... %x is a conversion specifier that makes any of the printf ... the letters abcdef are used for x conversion and the letters ... fewer digits, it is expanded with leading zeros. ... The result of converting a zero value with a precision of zero is ...
    (comp.lang.c)
  • Re: Previous Year in a Date Field
    ... Jamie wrote: ... > I have a date field, the date format is ... Word is guessing about how to interpret what you typed, ... will be discarded) as 10/1/04 or you can enter the year as all four digits, ...
    (microsoft.public.word.docmanagement)
Loading