Re: This calculation is just wrong / computer can't count!

On Thu, 4 Oct 2007 22:44:14 +0100, "GT"
<ContactGT_remove_@xxxxxxxxxxx> wrote:

One of the results should be exactly 0, but the computer gives the
result -0.00000000007. Which is then displayed on the screen as -0.0. This
result is displayed on the screen (in an edit boxes) along with all the
positive numbers and is just wrong!

-0.0 = 0.0 for all intents and purposes. The only difference is the
sign bit in the internal representation.

You cannot perform floating point math on a computer and compare the
result to integer 0 or even a float 0. You must compare it to what
your application will allow to be close enough to zero to be
considered zero. Therefore your statements for comparing a floating
point value to zero to six figures should be the equivalent of:

If 0.000001 > n < -0.000001 then n = 0 where n is type double and the
bracketing values are acceptable in your application.

.

Relevant Pages

  • Re: rounding off double values
    ... while the second array is result of some calculation ... compare elements of both arrays .How do i do the rounding off? ... That's where floating points becomes scary, ... zero than far from zero, but this seems to measure whether the program ...
    (comp.lang.java.programmer)
  • Re: decimal values create inequality
    ... I grew up with the saying "never compare a floating number to zero", ...
    (microsoft.public.excel.worksheet.functions)
  • REPRESENT revisited
    ... Forth-94 has few floating point display functions [none in the ... If u is greater than zero the character string shall consist ... If u is zero the string shall consist of one digit representing ...
    (comp.lang.forth)
  • Re: Floating point errors in collision routines
    ... > Trying to converge a floating point calculation all the way to ... > zero is fundamentally futile. ... > Newer collision engines, like Gino's SOLID, get ... >> the plane.) ...
    (comp.lang.cpp)
  • Cfv: Floating point truncation
    ... RfD: Floating point truncation ... which, respectively, provide rounding to the nearest integer, and rounding to the nearest integer on the side of the number line towards negative infinity. ... ANS Forth does not provide a word for rounding towards zero, and returning a floating point representation of the resulting integer. ... Include a standard word, named "FTRUNC", to perform "truncation", i.e. rounding of a floating point number towards zero, returning a floating point representation of the resulting integer. ...
    (comp.lang.forth)