Re: Newbie: Data storage and manipulation speed
- From: "steve" <try@xxxxxxxx>
- Date: Wed, 17 May 2006 01:30:43 -0400
Thanx for your kind response !
See below for my feedback......
"Larry Serflaten" <serflaten@xxxxxxxxxxxxxx> wrote in message
news:OjUO1$UeGHA.1208@xxxxxxxxxxxxxxxxxxxxxxx
Hmmm. i'll try to play with the timing more. I mod every 10000 iterations,
"steve" <try@xxxxxxxx> wrote
I am developing a small program that reads some data and then calculates
some parameters based on the data:
-----------------------------------
- Read data from txt file
- Store data (where?)
In memory? (An obvious choice...)
1) No matter my choice of storage, i need a way for the program to
"breath".
I dont care much about the time it takes to calculate.
I noticed that wether i use an Access (ADO) Table or an array the
program
becomes almost irresponsive. (if i try to click on it, there is a msg at
the
top saying Program is not responding") but then if i let it finish it's
ok.
It doesnt hang. ** I AM using WithEvents **. Any other trick that i
dont
know about ?
You have to decide how to implement it, but during large computational
routines, plan to call DoEvents about 3-5 times per second. Too few and
you get that unresponsive feeling, too many and you increase the total
time
to compleation. 3-5 times per second is an optimal range. I gave a per
second value to give you an idea of how often, you'll have to decide when
and where DoEvents will need to be put to hit that range.
maybe i have to fine-tune it.
So i guess DoEvents is my only choice ?
What would be my best choice for sacrificing space and gaining in
responsiveness and time ?
That would be difficult to answer without knowing the nature of your data.
What do you mean by nature? Types ?
Well, like in the table of my original post, I have 3 X Int originally. And
then i add 20 x reals which are coming from calculations.
- Pagefile = Memory , RAM would soon be filled up , especially in older
256MB systems? This means pagefile acces ....
- Access Table through ADO manipulations.
-Binary txt file, with Get and Put ?
Can you process your data in chunks of suitable size? What is the desired
result of the complete job, etc... But, if memory is insuficient, you'll
have to
use files on disk, either you try to optimize that (first choice), or let
Windows
do the paging thing (not so desirable).
Chunks. I thought about it but like i said in the first post, i dont think
thats possible.
For every calculation iteration i have to "jump" to different places in the
original data (Year, X, Y).
SQL could possibly do it, but the statements would be horendous!!!
especially for me.
Is it true that using a binary file with Put/Get , is faster than looping
through a recordset in ADO ?
Thanx again!
But again, far more information is needed to even discuss possible
improvements.
LFS
.
- Follow-Ups:
- Re: Newbie: Data storage and manipulation speed
- From: Larry Serflaten
- Re: Newbie: Data storage and manipulation speed
- References:
- Newbie: Data storage and manipulation speed
- From: steve
- Re: Newbie: Data storage and manipulation speed
- From: Larry Serflaten
- Newbie: Data storage and manipulation speed
- Prev by Date: change extension of file
- Next by Date: how to add a reference?
- Previous by thread: Re: Newbie: Data storage and manipulation speed
- Next by thread: Re: Newbie: Data storage and manipulation speed
- Index(es):
Relevant Pages
|
