Working with floating point values
- From: "David Veeneman" <davidv@xxxxxxxxxx>
- Date: Tue, 9 Aug 2005 08:40:04 -0500
I'm working on a project that uses floating-point values (doubles), and I'm
being driven crazy by something pretty basic. I understand that it's in the
nature of floating-point calculations to produce values like
0.10000000000000003, when what I really want is 0.1. But is there any way to
eliminate that digit at the end? I've tried rounding, but that simply moves
the digit to the least significant position, such as 0.1000003.
Failing that, can anyone recommend any good online articles for dealing with
floating-point issues, beyond using Epsilon for zero comparisons? Thanks in
advance.
David Veeneman
Foresight Systems
.
- Follow-Ups:
- Re: Working with floating point values
- From: Red Forks
- Re: Working with floating point values
- From: David Veeneman
- RE: Working with floating point values
- From: KH
- Re: Working with floating point values
- From: ranjeet
- RE: Working with floating point values
- From: Brian Delahunty
- Re: Working with floating point values
- From: questions
- Re: Working with floating point values
- Prev by Date: Start Process with different user
- Next by Date: OriginAtMargins question
- Previous by thread: Start Process with different user
- Next by thread: Re: Working with floating point values
- Index(es):
Relevant Pages
|
Loading