Re: Floating point precision problem
- From: "Simon Trew" <ten.enagro@werts>
- Date: Wed, 12 Oct 2005 23:40:49 +0100
"David Webber" <dave@xxxxxxxxxxxxxxxxxxx> wrote in message
news:esVut90zFHA.1252@xxxxxxxxxxxxxxxxxxxxxxx
> If you are testing the result with something like
>
> if( result==98655827.0032 )
>
> and finding it false instead of true, then again you need to employ a
> numerical mathematician who will know that anyone who does that sort of
> thing with floating point should be shot, and will replace it by
>
> if( abs(result-98655827.0032) < some_appropriate_small_number )
Or, more generally, using a comparison function that uses a relative error
(tolerance) instead of an absolute one.
.
- Follow-Ups:
- Re: Floating point precision problem
- From: David Webber
- Re: Floating point precision problem
- Prev by Date: Re: _vsnwprintf Entry Point Not Found in Windows 2000
- Next by Date: Re: DEFINE_GUID problem
- Previous by thread: Re: Floating point precision problem
- Next by thread: Re: Floating point precision problem
- Index(es):
Relevant Pages
|