Re: Singles to Doubles
- From: "Karl E. Peterson" <karl@xxxxxxxx>
- Date: Mon, 16 Jul 2007 15:49:50 -0700
Hi All --
Choosing one response to hang off, here...
In the string domain the value is exact, and can be converted to a double using
all 53 bits of precision. So yes, you get the best result if you pass through
a domain where the value is exact.
This makes perfect sense.
In this case you could also have transitioned through a Currency value and
gotten the same good results.
And *lots* faster, too! I'm iterating some 4 million items at a time, and that
trims it down from 3.2s to 0.6s on each loop. But I see I only hold 4 decimal
places. Gonna have to confer with the client to see if that's acceptable. Using
the String method is definitely producing the closest replication of their data when
it goes to Double.
Thanks... Karl
--
..NET: It's About Trust!
http://vfred.mvps.org
.
- Follow-Ups:
- Re: Singles to Doubles
- From: Jim Mack
- Re: Singles to Doubles
- From: dpb
- Re: Singles to Doubles
- References:
- Singles to Doubles
- From: Karl E. Peterson
- Re: Singles to Doubles
- From: Bob Butler
- Re: Singles to Doubles
- From: Karl E. Peterson
- Re: Singles to Doubles
- From: Jim Mack
- Singles to Doubles
- Prev by Date: Re: Problems with Program on version 6.0
- Next by Date: Re: VB6 - why isnt a straight Win32 DLL offered as a project type?
- Previous by thread: Re: Singles to Doubles
- Next by thread: Re: Singles to Doubles
- Index(es):
Relevant Pages
|