Re: Newbie: Data storage and manipulation speed


"steve" <try@xxxxxxxx> wrote

I am developing a small program that reads some data and then calculates
some parameters based on the data:

-----------------------------------
- Read data from txt file
- Store data (where?)

In memory? (An obvious choice...)

1) No matter my choice of storage, i need a way for the program to "breath".
I dont care much about the time it takes to calculate.
I noticed that wether i use an Access (ADO) Table or an array the program
becomes almost irresponsive. (if i try to click on it, there is a msg at the
top saying Program is not responding") but then if i let it finish it's ok.
It doesnt hang. ** I AM using WithEvents **. Any other trick that i dont
know about ?

You have to decide how to implement it, but during large computational
routines, plan to call DoEvents about 3-5 times per second. Too few and
you get that unresponsive feeling, too many and you increase the total time
to compleation. 3-5 times per second is an optimal range. I gave a per
second value to give you an idea of how often, you'll have to decide when
and where DoEvents will need to be put to hit that range.


What would be my best choice for sacrificing space and gaining in
responsiveness and time ?

That would be difficult to answer without knowing the nature of your data.

- Pagefile = Memory , RAM would soon be filled up , especially in older
256MB systems? This means pagefile acces ....
- Access Table through ADO manipulations.
-Binary txt file, with Get and Put ?

Can you process your data in chunks of suitable size? What is the desired
result of the complete job, etc... But, if memory is insuficient, you'll have to
use files on disk, either you try to optimize that (first choice), or let Windows
do the paging thing (not so desirable).

But again, far more information is needed to even discuss possible improvements.

LFS


.

Relevant Pages

  • Re: Not Responding errors
    ... memory in the hopes that it would easily solve my problems. ... had the "not responding" would have been eliminated. ... using a programme like Autoruns. ... Look in Norton for ways to suspend scanning on Start Up to test it's ...
    (microsoft.public.windowsxp.help_and_support)
  • Re: fc5 hangs
    ... responding extremely slowly. ... quite responsive (even when a second ssh session was still sluggish) but sometimes ... of free memory, but I freed up some memory by killing a few processes and the problem ... home directory again, but the system was still sluggish. ...
    (Fedora)
  • Re: Not Responding errors
    ... using a programme like Autoruns. ... Look in Norton for ways to suspend scanning on Start Up to test it's ... direct correlation between the start up process and the Not Responding ... problem is insufficient RAM memory. ...
    (microsoft.public.windowsxp.help_and_support)
  • Re: hacker challenge - traverse a list
    ... Omemory, and what I call O'memory. ... I am happy to concede that there are plenty of people in this newsgroup who ... programming with interest and enthusiasm. ... meant "the group of people in this newsgroup responding to Harter". ...
    (comp.programming)
  • Re: fc5 hangs
    ... responding extremely slowly. ... Since no process seems to be hogging memory or swap. ... quite responsive (even when a second ssh session was still sluggish) but sometimes ... home directory again, but the system was still sluggish. ...
    (Fedora)
Loading