RE: EMERGENCY! Offline Files Disappearing / Vanishing

I greatly appreciate everyone's input here, but I'm afraid I am no closer to
solving this

(maddening) mystery.

Looking at each contributor's advice, one by one:

Dave Nickason:
-------------

Dave - one thing you said in your first paragraph:

"XP will limit the number of files to the size specified in Windows Explorer
Tools ->
Folder Options -> Offline Files."

I believe this is untrue. This refers to 'temporary' caching, and not to
files manually or administratively cached. See:
http://support.microsoft.com/?kbid=904676

Also, when (My Documents) folder redirection is administratively set in SBS
2003, it WILL make all sub-folders available offline.

On user error: I have considered this, but I have ruled it out. The user has
an excellent understanding of Offline Files, and is ultra-careful and well
instructed! He is certainly not deleting all his folders. Additionally, I
have enabled the Group Policy "Remove 'Make Available Offline' " so he is
unable to pick and choose what to synchronize - everything in My Docs is set
to be available offline, and he cannot change that.

Quotas, free space, disk errors - all ruled out.

The one thing I haven't been able to look at this time (because he is away)
are the client event logs...something may lie there. I'm really UN-hopeful at
this stage though.

Lanwench:
--------

You helped me before with something - thanks. Point taken on Offline Files.
I'm starting to feel the same way.

My client *seems* to be taking this okay. However, he wants to "talk about
it" when he gets back. He will want answers though, and will want to be
CERTAIN that this never happens again. At this stage, short of hosing the
entire server and every machine - literally building the domain from scratch
- which I may do - I can't think of a way around this one, and nobody seems
to have an answer.

I think I will take your advice and give SecondCopy a second look. Also, I'm
thinking about SyncToy in the XP PowerToys kit. There is a guy from Microsoft
- Jonathan Hardwick - who has a blog dedicated to Offline Files
(http://blogs.msdn.com/jonathanh/archive/category/6957.aspx), and I got the
recommendation from there.

kj:
--

There are no related entries in the server event log, because he is never
connected to the server when this happens! This always occurs when he is away
from the office, and not connected to the server. As I noted above though, I
have not yet seen the event logs on the laptop itself.

Ethan:
-----

We can rule out Window Washer. There are no other unusual third-party apps
on the machine, just the standard faire of Office 2003, Acrobat Reader, etc.
Thanks for your suggestion though - you never know with these things -
anything is possible!

So What Now?
-----------

Well, that's where I'm at. Perhaps the event logs on the laptop itself will
shed some light. Perhaps not.

As I mentioned in my first post, the one thing at the "back of my mind" that
I have doubts about, is the fact that "Mary" at one point had access to his
User share on the server. This access has since been removed and permissions
replaced back to normal, and also all the machines have been re-imaged.
However, knowing how finicky Windows Permissions can be sometimes, I'm
wondering if *something* is holding over somehow? I really don't think so
though...everything looks exactly as it should.

I'm now grasping at straws, and wondering if it's a particular file?
Something corrupted somewhere? I'm really at my wits end.

The one thing I pretty much know will fix it, will be a total reinstall of
the server from scratch (along with all the workstations). That is a
monumental task, and a pretty brute-force "fix" even for a small office such
as this. I may be left with no choice though...


"Macker" wrote:

I never believed the Offline Files horror stories I would read on forums from
other system admins - until now. (Apologies for not taking "you" seriously).
I may have lost a client today...

(Please excuse the length of this post, but I want to try and give as much
information as possible)

Here is the infrastructure/scenario:

- SBS 2003
- My Documents Redirection enabled
- My Documents made "Available Offline"
- Size Limit on server set to 5GB (rather than default 1GB limit)
- Client laptop: Dell Latitude, XP Pro SP2

Here is a basic overview of what is happening:

- STARTING POINT (before any problems):
- The client's My Documents are fully synched and available offline - no
problems.
- The client has no problems for two years - his My Documents are always
available offline.
- Everyone is happy.
- The size of his "My Documents" before the problems started, is about 4.2GB

- PROBLEMS:
- One evening (6 months ago) I get a call: The client is offline, and the
most of the contents of his My Documents are GONE.
- I say "most", because about ten files (out of 2,000+) are available. They
are the files he most recently worked on. They are available at the end of
the tree, eg:

My Documents\School Files\School 2005\Science Class\ScienceProject1.doc

AND

My Documents\School Files\School 2006\Civics Class\CivicsProject1.doc

However, in the "Science Class" or "Civics Class" folders, there should be
hundreds of files, not just the ScienceProject1.doc and CivicsProject1.doc
files. Additionally, there should be (lots of) files in the folders above
these. eg: the "School 2006" folder should also have hundreds of files in it,
as well as other folders. They are all missing.

ATTEMPTED FIXES:

FIRST TIME:

The first time this happened (at 11PM at night, when he had a flight at 6AM):

- I drove to his house, picked up his laptop, and drove to his office where
the server lives.
- I first of all looked on the server, and sure enough his "missing" files
were all there under his appropriate user folder on the server.
- Next, I hooked up the laptop, and logged onto the network as the user.
- I DID NOT SYNCHRONIZE (YET)
- I opened up his My Documents, and sure enough, there were his files - all
WITHOUT the blue synchronization arrows next to them. The only ones that DID
have the arrows, were the ones mentioned above (ScienceProject1.doc,
CivicsProject1.doc, etc.)
- I then performed a full synchronization. This took about three hours, over
a 100mbps network. It appeared to have to rebuilt the offline cache from
scratch.
- Once the synchronization was complete, I did another synchronization "just
to be sure." This time it ran like a normal sync, and took about a minute.
- I then powered down the laptop, disconnected the network cable, and
rebooted in offline mode. All the files were now there - everything seemed
fine.
- I returned the laptop to the client (at about 4AM).

SECOND TIME:

My client worked away merrily for about a week, then disaster struck again:
Exact same symptoms - only about the most recent 7 to 10 files he had been
working on, were available. Again, the tree structure above the 7 to 10 files
was intact, but there was nothing else in the folders.

This time (about 1PM in the afternoon), he was again about to leave for the
weekend at 8PM.

- I rendevouzed with him at his office.
- I synched up the laptop as before.
- THEN -
- I COPIED the contents of his My Documents onto his Desktop.
- I DELETED the contents of his My Documents
- I did another sync (of his now empty My Documents)
- I deleted his Offline Cache (Folder Options, on the Offline Files tab,
press CTRL+SHIFT, and then click Delete Files).
- I did another sync.
- THEN -
- I COPIED the (previously copied) My Documents contents, from his desktop,
back to his real My Documents.
- I performed another full sync.
- I then did another sync (just to be sure), and rebooted.
- Everything was still there. I synched again.
- I shut down, rebooted in offline mode.
- All the files were there. "This is fixed!" I thought.

THIRD TIME:

Everything was okay for about a month. Then lightning struck a third time.
Again, the exact same problem - only a few files available.

This time I really went overboard: It is a small office - three users only.
(I have a client with FIFTY users, and have never had these problems).

His assistant (let's call her "Mary") has a workstation. She sometimes needs
access to his files, so sometimes she logs onto her workstation as him (let's
call him "Joe"). There is also a laptop which Mary uses, and sometimes she
logs on as Joe on the laptop too. I figured that perhaps there was some
conflict between the three machines. I therefore decided to re-image (wipe)
ALL THE MACHINES and start afresh.

- I first of all made a systemwide backup.
- For added backup, I then burned several DVDs with each user's My Documents
on them.
- I deleted the contents of the User's folders on the server.
- I re-imaged all the machines.
- I deleted the computer accounts of the old machines (using the proper SBS
tools).
- I joined the "newly minted" computers to the domain (using the proper SBS
wizards).
- I logged on each machine as Administrator on the domain, and added a few
extra apps that were not on the image.
- I then logged on as the respective user who would be using that machine.
- I copied the contents of the User's My Docs DVD to their desktop first...
- Then I COPIED the contents of the desktop folder to their My Documents
(thereby repropogating their empty/clean User folder on the server).
- I then synched up each machine.
- THIS ALL TOOK A-G-E-S BY THE WAY!!!
- I then went offline with Joe's laptop, checked the files were there, etc.
- Everything was fine.
- That was back in January.

FAST FORWARD SEVEN MONTHS...

FOURTH TIME:

I really thought that was the end of it. There have been no problems since
January. To my horror, I received a call from my client on Monday of this
week (today is Wednesday). MISSING FILES AGAIN!

Same problem - exactly.

I thought there may be some "salvation," as he said he had been "moving a
lot of files around." My guess was that he had just moved some files around
while in the office, and then forgot to synchronize at the end. He had
basically moved older folders into a new "Past Years" folder in his My
Documents. Nothing wrong with that - you should be able to do that! I guessed
he hadn't synchronized.

So, back over to his house, and back down to the office...

- Hooked up the laptop
- Logged onto the network as Joe.
- Synched files - took a good while (half-hour on his new 1000 mbps network)
- Synched again (just to be sure)
- Shut down.
- Rebooted in offline mode.
- All the files were there.
- Returned the laptop to him.

FIFTH TIME (TODAY):

- I get a call from Joe at 6AM. I am asleep. He is at the airport in San
Francisco. (We are based in San Francisco).
- HIS FILES ARE GONE AGAIN!
- He is flying (via Chicago) to a small town in the East Coast for business
- He needs his files.
- I may lose Joe as a client as a result of this.


SUMMARY / ADDITIONAL THOUGHTS (POSSIBLE OTHER REASONS):

This is clearly unacceptable. This is business "make or break" stuff.

I have a single rule for myself and my employees: I don't care if you go out
and get drunk every night. I don't care if you prefer to work naked at your
desk. I don't care if you masturbate for an hour each day in the bathroom at
work. I don't care if you lose your temper and take a hammer and chisel to a
computer (as long as it's backed up). I really don't care what you do, as
long as...

*** YOU NEVER EVER EVER E-V-E-R LOSE A CLIENT'S DATA - NEVER. ***

You will be fired immediately. We can work on your personal problems. We can
replace broken equipment. We can get you in a recovery program. We can buy
new computers. WE CANNOT BUY NEW DATA. Therefore it must never happen.

POSSIBILITIES:

1) The one common thread in all of this, is that this ALWAYS seems to happen
when Joe is AWAY from the office: He has all of his files offline and working
fine. Then, suddenly, they disappear. There is no VPN involved. He is not
connecting to the network remotely (except for Outlook via RPC over HTTPS).

2) THIS MAY BE IMPORTANT: There was a time, that Mary had direct access to
Joe's User folder on the server. He would ocassionally need her to go in and
modify a file. However, when I reimaged all the machines (see "Third Time"),
I explictly removed her permissions from his folder and restored the folder
to it's default state - and replaced permissions with the proper ones. Maybe
that has something to do with it, but I don't see how? It has been working
for seven months without a hitch.

CAN ANYBODY HELP ME?!!! Thank you.
.

Relevant Pages

  • Re: EMERGENCY! Offline Files Disappearing / Vanishing
    ... I would not even consider an undertaking like reinstalling a server as ... Folder Options -> Offline Files." ... have not yet seen the event logs on the laptop itself. ...
    (microsoft.public.windows.server.sbs)
  • Re: EMERGENCY! Offline Files Disappearing / Vanishing
    ... Both as the admin and the user, Folder Redirection, Desktop sync, ... I don't use offline files, but I do redirect My Documents. ... stored on a server that will be critical if my network or server ... The one common event is THIS LAPTOP. ...
    (microsoft.public.windows.server.sbs)
  • Re: EMERGENCY! Offline Files Disappearing / Vanishing
    ... PeerSync to synchronize data to his laptop. ... it always happend when being OFFLINE, so it does not seem to have anything ... were all there under his appropriate user folder on the server. ... WITHOUT the blue synchronization arrows next to them. ...
    (microsoft.public.windows.server.sbs)
  • Re: EMERGENCY! Offline Files Disappearing / Vanishing
    ... My Documents made "Available Offline" ... Client laptop: ... files were all there under his appropriate user folder on the server. ...
    (microsoft.public.windows.server.sbs)
  • RE: EMERGENCY! Offline Files Disappearing / Vanishing
    ... it always happend when being OFFLINE, so it does not seem to have anything to ... I increased the harddisk of the laptop because it only had about 10 GB free ... were all there under his appropriate user folder on the server. ... WITHOUT the blue synchronization arrows next to them. ...
    (microsoft.public.windows.server.sbs)