Re: File Searching, how to speed it up?

Better still perhaps: a service that either uses a watcher to update your saved file list, or just scans periodically and does the update, so that when your app is executed, it's already current, or has a short list of changed files to scan.

--Bob

Bob Grommes wrote:
This may or may not be a good trade-off, but possibly moving a lot of this logic to a Windows service that continually maintains a (preferably in-memory) listing and provides an API for your app to access that data, is something to consider.

Also, no personal experience here, but a file system watcher might be able to notify your app (or service) when something changes in a particular directory, thus avoiding a complete re-scan.

--Bob

janderson wrote:
Hi,

I have a win32 C# app that needs to recursively search for a particular file type really fast. It always searches the same place for these files.

I'm using Directory.GetDirectories and Directory.GetFiles to do it currently. Because I know the location is the same every time I build a cache of all the files and directories. If a directory changes I update its contents on startup of the app.

Unfortunately when I do a last-date-of-modification check on a directory it only reports on changes of files that are directly under it. So I have to check every single directly rather then simply checking the parent directory.

While I get a significant improvement when none of the directories change (its much faster to load a file with all the entries in it then to do a directory search). In particular the first time the app loads up and has to search every file (around 100,000 and growing) which is painfully slow (2 or 3 minutes to start up the app).

What techniques/APIs can I use to speed up file searching?

Is there a way to more directly look at the raw data that make up the file-tables?

Cheers
.

Relevant Pages

  • Re: Slow start up times...
    ... I'm afraid I really don't know, especially since it runs fine the first time ... Try testing the Normal template, for Word at least--see the link on this ... After the app loads, it then hangs for a good 10 secs. ...
    (microsoft.public.mac.office)
  • RE: File Monitoring
    ... > Or Am I missing some basic infos? ... One of the key goals of our app was to ... watcher at any given time, the session based watcher made this more ... The underlying session that forms the core of the ...
    (perl.beginners)
  • Re: File searching is really slow, how do I speed it up?
    ... This service can always run in the background and use the file system watcher to be triggered on new/changed files even if the other app is not running. ... If a directory changes I update its contents on startup of the app. ... In particular the first time the app loads up and has to search every file which is painfully slow. ...
    (microsoft.public.dotnet.framework)
  • File searching is really slow, how do I speed it up?
    ... If a directory changes I update its contents on startup of the app. ... In particular the first time the app loads up and has to search every file which is painfully slow. ...
    (microsoft.public.dotnet.framework)
  • File Searching, how to speed it up?
    ... If a directory changes I update its contents on startup of the app. ... In particular the first time the app loads up and has to search every file which is painfully slow. ...
    (microsoft.public.dotnet.framework.performance)
Loading