Re: ARGH! Access can't add up?!?

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

From: Pecanfan (pecanfan_at_no.spam)
Date: 03/18/05 

Date: Fri, 18 Mar 2005 12:48:14 -0000

> > =Int(([InvoiceTotalNET]*(([cbxVATRate].[Column](0))/100))*100)/100
> >
> > Now, you'd expect when InvoiceTotalNET = 108 and cbxVATRate = 17.5 that
> > the
> > above would come out as 126.90 but NO - for some reason it's coming out
at
> > 126.89!!! Same happens for 10800 (this comes out at 12689.99) BUT 1080
> > comes out at the correct value, 1269.00. What the... ?!?!
>
> Are you sure your formula's right? I get 18.9 when I do the above. To
get
> your figure, I have to do this:

Are you sure? That's copied and pasted straight out the form.

> =[InvoiceTotalNET] + [InvoiceTotalNET] * [cbxVATRate].[Column](0)] / 100
>
> The error you're seeing is called floating point error. Most versions of
> BASIC (including VBA) use floating point numbers to represent data and
> there's some rounding error.

Bugger. Cheers! :-)

Andy

Relevant Pages

  • Re: 32-bit IEEE float multiplication
    ... Andy wrote: ... > I don't know if this is the correct group to post this, ... Assuming 32-bit IEEE floating point numbers, ... their product should be the representation of the number that is closest ...
    (comp.lang.c)
  • Re: 32-bit IEEE float multiplication
    ... bikejog@hotmail.com (Andy) wrote: ... > I don't know if this is the correct group to post this, ... > when I multiply a huge floating point value by a really ... Could you post which compiler is used, and post source ...
    (comp.lang.c)
  • Re: Advice please: admin /i in ex2003
    ... Many thanks Arlo, ... Andy ... > There are a lot of these floating around. ...
    (microsoft.public.exchange.admin)
  • Re: Rounding error in Stdev function result.
    ... But the implementation of STDEV() could ... >the 80-bit FP registers. ... Rounding error elimiated without using 80-bit FPU ... While floating point addition and multiplication are ...
    (microsoft.public.excel.worksheet.functions)
  • Re: problem with float
    ... i found the value stored is not the exact value which i gave. ... The floating point numbers are a subset of the real numbers. ... Usually the subset is based on a binary representation and so most non ... expect that a rounding error won't be introduced during the conversion to ...
    (comp.lang.c)