Re: Scanning hard drive
- From: "Stephany Young" <noone@localhost>
- Date: Mon, 22 Oct 2007 18:31:25 +1300
It's Stephany!
You asked 'Any chance of seeing your code???'
I replied 'It was your code (that I ran)'.
The DriveData class did not add any significant overhead.
I used 10 for Age1Days and 20 for Age2Days.
Here are my results for my main dev machine (Vista).
Drive size1 size2 size3 objects elapsed time objects per second
-----------------------------------------------------------------------------------------------
C:\ 7,406,454,214 1,166,404,747 45,787,333,348 167,342 1:57.0308662 1429.8962780812093
D:\ 395,375,683 582,398,194 31,460,898,868 59,563 0:11.8571418 5023.3859900368232
E:\ 168 0 87,079,722,334 36,356 0:04.6710676 7783.2313966083470
-----------------------------------------------------------------------------------------------
7,801,830,065 1,748,802,941 164,328,154,550 263,461 2:13.5590756 1972.6177260244529
Grand Total: 173,878,787,556 (161.937240 GB)
C:\ drive is the system drive and has the paging file so those 2 aspects go someway to explaining the slower rate.
Quite acceptable!
"Rotsey" <malcolm_smith@xxxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message news:e6ZVRVGFIHA.1188@xxxxxxxxxxxxxxxxxxxxxxx
Sorry Stephen what do you mean?
"Stephany Young" <noone@localhost> wrote in message news:%23r4NtPGFIHA.748@xxxxxxxxxxxxxxxxxxxxxxxIt was YOUR code!
"Rotsey" <malcolm_smith@xxxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message news:%236KhIkFFIHA.2004@xxxxxxxxxxxxxxxxxxxxxxxSorry about that , but it just a class with the generic list and a string drive property
All I am doing is displaying some of the info when ScanFolder returns
But that is after the scan completes of course which is not the problem.
Any chance of seeing your code???
public class DriveData
{
private List<PieFileInfo> mList = new List<PieFileInfo>();
public List<PieFileInfo> DataList
{
get { return mList; }
set { mList = value; }
}
private string mDrive;
public string Drive
{
get { return mDrive; }
set { mDrive = value; }
}
}
"Stephany Young" <noone@localhost> wrote in message news:%23CeybYFFIHA.5360@xxxxxxxxxxxxxxxxxxxxxxxApprox. 13 seconds on a 250GB disk with 30,000+ directories and 1,000,000+ files.
What is DriveData? (You omitted to post it).
What do you do with the content of DriveData when the call to ScanFolder(...) returns?
"Rotsey" <malcolm_smith@xxxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message news:eiuJvfEFIHA.4752@xxxxxxxxxxxxxxxxxxxxxxxOK. Here is the class the scans the drive.
Can you see any issues here?
Bela, would you be able to post your code that scans the drive.
I am scanning 56GB in about 10 minutes.
public struct PieFileInfo
{
public int RowID;
public PieInfoType Type;
public string Name;
public string FullName;
public double Filesize;
public DateTime LastAccess;
public DateTime Created;
public string Extension;
public DateTime LastWrite;
public int ParentFolderRowID;
public string Drive;
public double FileAge1;
public double FileAge2;
public double FileAge3;
public bool IsRoot;
}
public class ScanDrive
{
private DriveData mList = new DriveData();
public DriveData DataList
{
get { return mList; }
set { mList = value; }
}
private string mDrive;
private int mNewFolderID = 0;
public ScanDrive(string drivefolder)
{
mDrive = drivefolder;
mList.Drive = drivefolder;
}
public bool ScanFolder(string folder, int folderID, ref double psize1, ref double psize2, ref double psize3)
{
DirectoryInfo objDir = new DirectoryInfo(folder);
int newFolderID;
double size1 = 0;
double size2 = 0;
double size3 = 0;
System.DateTime fileDate;
try
{
newFolderID = WriteRow(objDir, folderID);
if (newFolderID == 0)
{
return false;
}
foreach (DirectoryInfo dir in objDir.GetDirectories())
{
if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref psize3))
{
return true;
}
size1 += psize1;
size2 += psize2;
size3 += psize3;
}
foreach (FileInfo fle in objDir.GetFiles())
{
WriteRow(fle, newFolderID);
fileDate = GetAgeDate(fle);
if (fileDate >= DateTime.Today.AddDays(Settings.Instance.SettingsData.Age1Days * -1))
size1 += fle.Length;
else if (fileDate >= DateTime.Today.AddDays(Settings.Instance.SettingsData.Age2Days * -1))
size2 += fle.Length;
else
size3 += fle.Length;
}
UpdateFolderRow(newFolderID, size1, size2, size3);
psize1 = size1;
psize2 = size2;
psize3 = size3;
return true;
}
catch (Exception)
{
return true;
}
}
private DateTime GetAgeDate(FileInfo f)
{
if (f.CreationTime >= DateTime.Today.AddDays(Settings.Instance.SettingsData.Age1Days * -1))
return f.CreationTime;
else if (f.LastWriteTime >= DateTime.Today.AddDays(Settings.Instance.SettingsData.Age2Days * -1))
return f.LastWriteTime;
else
return f.LastAccessTime;
}
private int WriteRow(DirectoryInfo dir, int folderID)
{
PieFileInfo info = new PieFileInfo();
if (mNewFolderID == 0)
info.IsRoot = true;
mNewFolderID++;
info.RowID = mNewFolderID;
info.Type = PieInfoType.Folder;
info.FullName = dir.FullName;
info.Name = dir.Name;
info.LastAccess = dir.LastAccessTime;
info.LastWrite = dir.LastWriteTime;
info.Created = dir.CreationTime;
info.ParentFolderRowID = folderID;
info.Drive = mDrive;
mList.DataList.Add(info);
return mNewFolderID;
}
private int WriteRow(FileInfo file, int folderID)
{
PieFileInfo info = new PieFileInfo();
info.Type = PieInfoType.File;
info.Name = file.Name;
info.FullName = file.FullName;
info.Filesize = file.Length;
info.Extension = file.Extension;
info.LastAccess = file.LastAccessTime;
info.LastWrite = file.LastWriteTime;
info.Created = file.CreationTime;
info.ParentFolderRowID = folderID;
info.Drive = mDrive;
if (file.LastAccessTime >= DateTime.Today.AddDays(-30))
info.FileAge1 = file.Length;
else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))
info.FileAge2 = file.Length;
else
info.FileAge3 = file.Length;
mList.DataList.Add(info);
return folderID;
}
private void UpdateFolderRow(int folderID, double size1, double size2, double size3)
{
int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)
{
if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)
return true;
else
return false;
});
PieFileInfo info = mList.DataList[index];
info.FileAge1 += size1;
info.FileAge2 += size2;
info.FileAge3 += size3;
info.Filesize += size1 + size2 + size3;
mList.DataList[index] = info;
}
}
"Bela Istok" <bela_i@xxxxxxxxxxx> wrote in message news:ED0DEFAE-1EB6-4DEF-96F8-CE11CAB9B316@xxxxxxxxxxxxxxxxWhat do you scan? I do a directory and File scan in my 100GB (64GB of data) hard drive and took 24 seconds. (Scanned every directory and every file in the drive for info).
PD: for reference 170,187 files and 21,899 folders.
Regards,
Bela Istok
"Rotsey" <malcolm_smith@xxxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message news:errJ9I6EIHA.1208@xxxxxxxxxxxxxxxxxxxxxxxHi,
I am writing an app that scans hard drives and logs info
about every fine on the drive.
The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.
I wanted it to be quicker.
So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.
I am wondering if I can improve on this even more??
Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?
rotsey
.
- References:
- Scanning hard drive
- From: Rotsey
- Re: Scanning hard drive
- From: Rotsey
- Re: Scanning hard drive
- From: Stephany Young
- Re: Scanning hard drive
- From: Rotsey
- Re: Scanning hard drive
- From: Stephany Young
- Re: Scanning hard drive
- From: Rotsey
- Scanning hard drive
- Prev by Date: Re: Weird design decision in LinQ To SQL
- Next by Date: Re: Scanning hard drive
- Previous by thread: Re: Scanning hard drive
- Next by thread: Re: Scanning hard drive
- Index(es):
Relevant Pages
|
