Re: floating point precision

Tech Tip: Click here to run a free scan for Windows Errors and optimize PC performance

.... well this is from a select <calculation> into result cursor, so I guess
I'll have to use cast(<calculation> as B(?)) to ensure that the cursor uses
the data type I want. The docs aren't entirely clear about how B(decimals)
will affect accuracy.

"Rush Strong" <rpstrong@xxxxxxxxx> wrote in message
news:SN36j.29636$AX6.19697@xxxxxxxxxxx
Lew wrote:
On the more general subject of floating point precision ... see previous
post ... I guess that storing division results in an array would be as
accurate as vfp can get ... I mean as opposed as storing them in a
cursor. So I'm thinking of a routine that creates a cursor of lookup keys
and an array of long division results. The routine would seek the key and
return arrayelement[recno()]. Has anyone tried something like this or...
do I just need more sleep?


I think you probably need more sleep. Will an array give you greater
precision than a 'double' field type? I don't think so.

From the help file:

"When the Double data type is used in a table, the number of decimals
specified is for presentation purposes only. Visual FoxPro stores the real
value."

- Rush


.

Relevant Pages

  • Re: array to iterator
    ... protected Objectarray; ... protected int length; ... * This member is used to keep track of the cursor of the iterator. ...
    (comp.lang.java.programmer)
  • Re: How about this?
    ... When parsing your values into a multirow array (or cursor) you can also ... Then, when displaying the grid, you can change the backcolor of every ...
    (microsoft.public.fox.programmer.exchange)
  • Re: How do I write an array to a text file?
    ... CREATE CURSOR temp ... You now have the array, the table, and the textfile. ... record keys, and a few buttons to Save, Print, Fix, and Exit. ...
    (microsoft.public.fox.helpwanted)
  • Re: Best approach?
    ... INSERT INTO Searchwords FROM ARRAY cArray ... CREATE CURSOR inventory (id Int, descript Memo) ... INSERT INTO Inventory VALUES ...
    (microsoft.public.fox.programmer.exchange)
  • Re: Array Question.
    ... >placing txt boxes with a reference to the array e.g. arraythen ... >arrayand so on 25 lines of 12 on the report. ... >i thought using an array to create what i neede was the simplest. ... >i need to do is use a cursor like mentioned below and go with that. ...
    (microsoft.public.fox.programmer.exchange)