Newbie: Data storage and manipulation speed

Hi,

I am developing a small program that reads some data and then calculates
some parameters based on the data:

-----------------------------------
- Read data from txt file
- Store data (where?)

Year X Y

(i read and need to perform calculations on around (depending on user
selections) 1-3 million records)

- Perform calculations and store in appropriate tuple

Year X Y Result1 Result2 .......... Result20

- Filter/graph/display part of the results
-------------------------------------

Two questions:

1) No matter my choice of storage, i need a way for the program to "breath".
I dont care much about the time it takes to calculate.
I noticed that wether i use an Access (ADO) Table or an array the program
becomes almost irresponsive. (if i try to click on it, there is a msg at the
top saying Program is not responding") but then if i let it finish it's ok.
It doesnt hang. ** I AM using WithEvents **. Any other trick that i dont
know about ?

2) Its too long to explain the calculations that need to be performed but
SQL would be too hard and too complex I believe. Take my word on that! We
have to jump and read a whole bunch of different tuples (Year,X,Y) to
calculate the various Result(s) .
When I try to read and write records using MyRecordset.Move or
AbsolutePosition , I have the same response problem i mentioned above.

What would be my best choice for sacrificing space and gaining in
responsiveness and time ?
- Pagefile = Memory , RAM would soon be filled up , especially in older
256MB systems? This means pagefile acces ....
- Access Table through ADO manipulations.
-Binary txt file, with Get and Put ?

Your suggestions are greatly appreciated! I have been struggling with this
for a while now. Maybe i should go to RentACoder but I would really like to
solve it myself.

Thanx again!
steve



.

Relevant Pages

  • Re: Creating formulas for each items
    ... Microsoft Access is a relational database. ... If you need to store data, then do calculations, store the data in Access ... The value of the item water consumes is based on another type of formula ...
    (microsoft.public.access.forms)
  • Re: Spreadsheet calculations with offsets
    ... you should follow the answer style used by the person responding to you. ... If you would rather not do that, are you allowed to run a macro which could perform the calculations? ... And you didn't answer my question about how the plus sign is place in your Station numbers or how the "foot mark" is placed at the end of the Offset distance. ... like it in a cell format maybe something like this. ...
    (microsoft.public.excel)
  • Re: Creating formulas for each items
    ... Microsoft Access is a relational database. ... If you need to store data, then do calculations, store the data in Access ...
    (microsoft.public.access.forms)
  • Re: Formulas
    ... >> Access tables will only store data. ... >> You can perform calculations in Access Queries, Forms, and Reports. ... incorrectly designed database. ... Once again, this is not Excel. ...
    (microsoft.public.access.gettingstarted)
  • Re: Obscene words on a tee shirt...can anything be done
    ... Such considerations do not enter into Tony's calculations. ... Who is 'Tony'? ... I was responding to Fredie Smith. ...
    (uk.legal)