Re: Large directories... DirectoryInfo or Win32 API?

From: Justin Rogers (Justin_at_games4dotnet.com)
Date: 09/29/04 

Date: Wed, 29 Sep 2004 14:41:22 -0700

To add, I've tested Whidbey and it appears to get a bit better. Use your
solution for now, in the case that it gets faster later ;-) 

-- 
Justin Rogers
DigiTec Web Consultants, LLC.
Blog: http://weblogs.asp.net/justin_rogers
"Justin Rogers" <Justin@games4dotnet.com> wrote in message 
news:eHvIsBmpEHA.648@tk2msftngp13.phx.gbl...
> Internally they are doing the Win32 API calls for you. However, they are
> also allocating a LOT of array space to do it. Specifically they are making
> an extra copy, which probably isn't necessary, and there is a possible
> resource leak in that particular method that they aren't accounting for.
>
> That includes both DirectoryInfo and Directory versions of GetFiles... If
> you've found a method that works for you and it is extremely fast, then
> perhaps stick with that. There are a number of refactorings in their code
> that are going to take a great deal more time than doing it yourself in a
> more direct manner.
>
>
> -- 
> Justin Rogers
> DigiTec Web Consultants, LLC.
> Blog: http://weblogs.asp.net/justin_rogers
>
>
> "Mike G." <MikeG@discussions.microsoft.com> wrote in message 
> news:5513454F-017C-49F7-A1CD-541A73945737@microsoft.com...
>> Hi All,
>>
>> I have a situation where I have several directories each containing hundreds
>> of thousands of files. I have written a program which looks at each file in a
>> directory, moves it to a subdirectory of a more manageable size, and updates
>> the file location in our database. I initially used DirectoryInfo.GetFiles()
>> to populate a FileInfo array to loop through. This turned out to be so slow
>> on these large directories that files were being added there faster than they
>> could be moved. It appears that when GetFiles is called it loads the entire
>> directory into memory before proceding to the next line.
>>
>> To solve this problem I put the Win32 API functions FindFirstFile,
>> FindNextFile, and FindClose in a wrapper class and created a struct to hold
>> the FileData. This works much faster and is an acceptable solution. However,
>> I would like to know, is there a way of doing this natively in .NET without
>> directly accessing the Win32 API? I would really like to stay in the world of
>> managed code if possible.
>>
>> Thanks,
>>
>> Mike G.
>
>

Relevant Pages

  • Re: Large directories... DirectoryInfo or Win32 API?
    ... "Justin Rogers" wrote: ... > That includes both DirectoryInfo and Directory versions of GetFiles... ... >> managed code if possible. ... >> Mike G. ...
    (microsoft.public.dotnet.framework.performance)
  • Re: Large directories... DirectoryInfo or Win32 API?
    ... Internally they are doing the Win32 API calls for you. ... also allocating a LOT of array space to do it. ... I would really like to stay in the world of> managed code if possible. ... > Mike G. ...
    (microsoft.public.dotnet.framework.performance)
  • Large directories... DirectoryInfo or Win32 API?
    ... It appears that when GetFiles is called it loads the entire ... directory into memory before proceding to the next line. ... To solve this problem I put the Win32 API functions FindFirstFile, ... managed code if possible. ...
    (microsoft.public.dotnet.framework.performance)
  • Re: Getting HWND of Console (.net 2.0)
    ... > You can use Win32 API ... >> functions to control window position. ... The console class doesn't provide ... >> Mike Ober. ...
    (microsoft.public.dotnet.framework)
  • RE: Managed OS?
    ... As far as I can understand, Longhorn will be built using Win32 API. ... Windows OS functionality is vast and some of it could be accessed using ... that will be built with managed code. ...
    (Security-Basics)