Re: Strange rounding issue in formatted number field
- From: Bob Greenblatt <bob@xxxxxxxxxx>
- Date: Fri, 21 Mar 2008 13:52:00 -0400
On 3/21/08 8:39 AM, in article ee95931.-1@xxxxxxxxxxxxxxxxxxxxxxx,
"pattera5@xxxxxxxxxxxxxxxx" <pattera5@xxxxxxxxxxxxxxxx> wrote:
Version: 2008This is an artifact of the conversion of binary to decimal and vice versa in
Operating System: Mac OS X 10.5 (Leopard)
Processor: intel
I have a cell formatted as a number with 13 decimal places, I have a formula
in there which returns a bizarre result.
It can be easily replicated by putting the following in the cell
=4935.48-4904.68
The result I get is 30.7999999999993. This is simple subtraction, where is
this rounding issue coming from? any ideas?
accordance with industry accepted standards. This is just the way Excel (and
almost every other program) works. Here is a tutorial to give you much more
information than you ever wanted to know:
http://support.microsoft.com/kb/42980/en-us
--
Bob Greenblatt [MVP], Macintosh
bobgreenblattATmsnDOTcom
.
- References:
- Strange rounding issue in formatted number field
- From: pattera5
- Strange rounding issue in formatted number field
- Prev by Date: Re: Converting
- Next by Date: Can I view an older Excel file with Macros in Office 2008???
- Previous by thread: Re: Strange rounding issue in formatted number field
- Next by thread: can I add a percent symbol without multiplying by 100?
- Index(es):
Relevant Pages
|
