Re: 15 Significant Digits Limitation a Mistake for Spatial Informa
- From: Jerry W. Lewis <post_a_reply@xxxxxxxxxxxxx>
- Date: Thu, 17 Apr 2008 03:39:05 -0700
The OP's question was less than specific. Your answer only applies to one of
the possible interpretations.
Hardware limitations would restrict to 8-byte IEEE double precision (DP,
though PC hardware that does not support 10-byte reals has not been made for
nearly 20 years).
However, DP does not restrict to 15 decimal digits. Most 16 digit numbers
can be represented in DP, so many packages can display more than 15 digits.
It takes 17 digits to uniquely determine a DP value, so most packages
(including Excel's VBA, but not Excel) will use 17 digits of the input value
to obtain the best binary representation.
Input and output precision are more tightly linked in Excel (and in VBA)
than in most packages, since in most packages the source code is not altered
on input. In the Excel model input becomes output, so some alteration on
input of the source is unavoidable if more than 15 figures are displayed
(e.g. 9007199254740993 has no DP representation, and so must change on entry
to either 9007199254740992 or 9007199254740994 if Excel were to support 16
figures on input and output. However 15 digit input means that you cannot
easily get full DP input precision, and 15 digit output means that you cannot
easily maintain DP precision in the face of subsequent editing.
Thus 15 digit I/O is a design decision, not a physical constraint. It is
not clear to me that MS adequately thought it through before making the
decisions (particularly since they decided differently for Excel and VBA),
but since the decisions were made, I doubt that MS will reconsider them.
Jerry
"Howard Kaikow" wrote:
"Matthew" <Matthew@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message.
news:98365604-FD37-43AC-BB50-2FC52C4072FB@xxxxxxxxxxxxxxxx
The IEEE Standard of limiting 15 significant digits of precision is aLatitude &
practical mistake for Microsoft to follow. Especially for marking
Longitude coordinates for geocodes and establishing and storing boundaries
for maps at zoomed levels. Take for instance, a latitude value of
39.729831646409565 in Denver Colorado. Excel only stores it as
39.7298316464095. The longitude of -105.02543449401855 is stored as
-105.025434494018. You loose two significant digits of accuracy for this
spacial information.
Can the decision be reevaluated for practical purposes?
No.
Due to hardware limitations, computers always have used very specic formats
for numeric data.
In general, there are integer (8-bit, 16-bit, 32-bit, 64-bit, etc. fixed
types), and floating-point types.
For integer types, you have signed and unsigned types, and ones complement
or twos complement format.
Each is subject to a range of values determined by the number of bits used
to represent the data type.
For floating-point, there used to be more diifferent formats han one could
easily count.
IEEE (actually I was a participant in the early meetings 30 years ago) did
us a favor by standardiszing the format.
If you need to do any calculations that exceed the ranges of these standard
data types, then you either have to try
Decimal data type or roll your own extended precision data types.
A place to start might be "The Definitive Guide To How Computers Do Math".
ISBN 0-471-73278-8.
Or a good numerical analysis book.
- References:
- 15 Significant Digits Limitation a Mistake for Spatial Information
- From: Matthew
- Re: 15 Significant Digits Limitation a Mistake for Spatial Information
- From: Howard Kaikow
- 15 Significant Digits Limitation a Mistake for Spatial Information
- Prev by Date: RE: Conditional Formating
- Next by Date: Re: Recovery file???
- Previous by thread: Re: 15 Significant Digits Limitation a Mistake for Spatial Information
- Next by thread: Re: 15 Significant Digits Limitation a Mistake for Spatial Information
- Index(es):
Relevant Pages
|
