Re: Copying large quantity of files taking forever
- From: "Pegasus \(MVP\)" <I.can@xxxxxxxxxx>
- Date: Thu, 9 Oct 2008 22:32:37 +0200
The utility xxcopy.exe is known for baulking at large copy jobs. Perhaps
there is a similar but much higher limit for robocopy. Which version do you
use? It should be at least Version XP010.
If this is a limitation of robocopy.exe then you may be able to walk around
it by splitting up the job into many smaller jobs, using a batch file. I
would try this:
@echo off
for %%a in (0 1 .. 9 a b .. z) do robocopy /s /.. "D:\SourceFolder"
"E:\DestFolder" %%a*.*
robocopy /s /XO "D:\SourceFolder" "E:\DestFolder" *.*
The second line will copy all files starting with 0 .. 9 and a .. z. You
must expand the dots to include the full alphabet - the batch file won't do
it for you! The third line will scoop up any file that was not already
copied by the first line. If the third line slows down to a crawl after a
while then you have to expand the collection of characters in Line 2 to
include all valid ASCII characters. Use charmap.exe to identify them!
"SwitchKat" <SwitchKat@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message
news:561323C4-1B08-4034-9BB6-63C4089824C6@xxxxxxxxxxxxxxxx
The files are spread out through many directories. I originally thought
this
might be an issue too since it also takes a long time to open any of the
folders.
"Pegasus (MVP)" wrote:
"SwitchKat" <SwitchKat@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message
news:EFEF754F-402C-438E-9654-0ABC30409346@xxxxxxxxxxxxxxxx
Hi, I wanted to throw this question out because after losing over a
week
of
productivity, I'm not sure where else to turn.
Here's my issue. We're trying to restore a file repository for a
project
that has just under 2 million files. We've tried restoring from tape,
copying from another Windows server and even moving from another
Windows
server. The problem we run into in every case is that the file copy to
the
new system takes off fast and we can see file transfer rates start out
copying 1000's of files per second, but over time it always slows down
to
a
crawl and we end up where we are now. We are currently transferring
one
file
every few minutes.
All the files are fairly small (<10MB), gigabit network (although we
see
the
same performance if we attach the drive locally and just try a copy),
and
all
are running Windows Server 2000 or 2003 server. The target is always
running
2003. We've been monitoring resources, and at this point, both servers
involved show flatlines for their resources being used. My hunch is
that
it
has something to do with a data block size issue or the file table
(maybe
fragmentation?), but I'm not sure. Any ideas?
You don't say where these files reside but if they reside in a single
directory then this is your answer. If you want to keep your machine
running
at top speed, don't keep more than 10,000 entries in a single folder. 2
million is two hundred times too many.
.
- Follow-Ups:
- Re: Copying large quantity of files taking forever
- From: SwitchKat
- Re: Copying large quantity of files taking forever
- References:
- Copying large quantity of files taking forever
- From: SwitchKat
- Re: Copying large quantity of files taking forever
- From: Pegasus \(MVP\)
- Re: Copying large quantity of files taking forever
- From: SwitchKat
- Copying large quantity of files taking forever
- Prev by Date: Re: File Server Transfer - Help
- Next by Date: Re: Profile question server2003/terminal server 2008
- Previous by thread: Re: Copying large quantity of files taking forever
- Next by thread: Re: Copying large quantity of files taking forever
- Index(es):
Relevant Pages
|
