Re: After manually undoing a recent MS Update, my server is a mess


Susan (and anyone masochistic enough to stick with this thread this far),

I have got to share the latest on this situation with you:

I e-mail my Tech Support guy at Microsoft and (naively again), gave him the
information to login to my server remotely to look around--hoping he could
figure out why my internet connection was dropping every 2 or 3 minutes and
my user's workstations were locking up for 30 seconds or so... (my contention
being that it was a problem from the server restore we had done)

He logs in for quite a while and decides to run the Fix My Network wizard.
It claims to have fixed a couple problems (like assigning a static IPV6
address), but couldn't fix Exchange. Now he can't even RDP into my server
because it knocks him out after about 30 seconds, then reconnects, and
repeats.

So, he tells me to replace my Switch (having just replaced my Router to one
that does IPV6). But suddenly my users were working fine again. Now, I can't
access my server remotely and things are still haywire.

If Microsoft's Tech Support team can't find/resolve the problems it's not a
good sign. Should I cut my losses now and re-install? What would you do?

Thanks again for all your advice,

jase

"Susan Bradley" wrote:

Keep in mind that I have seen OEM drivers up there so given that you've
blown that image away, I can't say for certain that the 'critical' was
100% pushed by Microsoft.

Never install drivers from Microsoft update. It's a rule I live by and
haven't regretted it.

When I tried to use a USB for external storage of the shadow copies, it
too did not like that and would spontaneously boot the box at random
events.

External USB is no replacement for local drives.

jase wrote:
Susan,

Glad to hear you aren't a betting gal. LOL I show my naivety here when I
say that I showed great diligence in my blind application of any MS suggested
resolution to "Critical Errors" on my Small Business Server Monitor Report. I
still find it hard to believe what qualifies as a "critical error" in MS
Terms.

Just to clarify: When I got the first BSOD, I booted from the orginal CD
(which did not have access to the dynamic volume that my user files were
stored on) and chose to only restore the System files. I figured this would
replace the corrupt or incompatible driver.

During the restore, the server rebooted without confirmation. I assumed that
was normal until the BSOD appeared again. So, I am still not sure if the
Server restarted during the restore because of error or because it's supposed
to. However, the restore session did not end in BSOD (because it restarted).

Thanks to you, I found another potential problem on my network. I am
temporarily using an old router as a gateway. This router is not IPv6
compatible. I will remedy this situation today.

I think the big issue at hand is that it just not wise to try to mirror an
external USB drive with an internal SATA drive and use that mirror to store
your user documents. Unfortunately, I have to find a way to move all my user
files to another disk before I can reclaim the two dynamic disk, which is
going to be another pain.

Thanks again for your help, I always learn alot from your posts.

Jase

"Susan Bradley" wrote:

If I were a betting gal, I'd say one of your updates was one of two things:

1. a true driver update (which I never ever ever ever update a driver
from Microsoft update). I've been burned too many times, so I just
don't do it.

2. Win2k8 sp2 which 'might' have driver updates.

But the fact that you got a bsod during the restore process does not
bode well for me. Confirm you did right?

jase wrote:
Susan,

Encouraged by your statement that "[I] didn't get to the underlying cause
though :-( ", I decided to give it one more try. I think I have figured out
the problem, but was hoping you could give me some input on this theory:

BACKGROUND:
Before installing this server, my users were accustomed to storing all of
their files on what they call the "Cases" drive (in a folder labeled with
their names). The "Cases" drive was actually a shared folder on an external
SATA drive (connected by USB to some computer in the workgroup).

When I installed the server, I followed the instructions explicitly and
waited until everything was up and running to make any big changes to the
public shares.

I connected the "Cases" drive to the server and installed an identical drive
to SATA0 inside the server (separate from the two Raid 1 volumes that
contained my OS, Exchange, Admin and Data volumes). I mirrored the two drives
(internal and external) to create a virtual drive (U:\) in which I created a
"users" directory.

I used the SBS2008 Folder Redirection Wizard to redirect everyone's My
Documents folder to the U:\%username% directory. The change happened quickly
because nobody used My Documents to store any files--everything they did was
stored on the "Cases" drive.

Next, I copied all of the user's files from their beloved Cases folder to
their appropriate user folder. I created a public share named "Cases" that
mapped to U:\users\ ... and nobody knew any different. My thought was that by
doing this, if I ever had a problem with the server, I could snag the
external drive and connect it to any workstation immediately for emergency
access to the files (since it was a mirrored volume).

Fortunately, this all worked exceptionally well with one small exception:
Backups. For some reason the SBS2008 Backup software did not want to find the
U: volume while executing. You could assign the drive to be backed up, but
every time it would error out claiming it could not find the volume. I was
never sure if I could safely ignore that message or not and since 98% of the
files used by my office were on the U: drive, I decided I needed to fix it.

During one update, my mirror apparently broke because a new volume appeared
with the same label, but different drive letter. The internal SATA drive was
now U:/ and though it had a lot of the same content as the previous U:/
drive, it was not an exact mirror like I had expected it would be. This was
the final clue that told me something had to be done now.

I setup a D:\Users directory and created a public share for the directory
called "users2" as a temporary measure to redirect user folders. I used the
Folder Redirect Wizard again to move users' folders to the new
"D:\users\%username%" directory and waited for the policy to apply.

Two days later, after an update, I get the BSOD. I think perhaps the problem
was that the update contained something that caused corruption in the drivers
used to read a dynamic volume. This would explain why I could get 3/4 of the
way through the boot process before it went BSOD.

My first reaction, of course, was to restore to a date prior to the update.
I chose to only restore the System since I felt like it was a driver issue,
nothing more. of course, this didn't help because the Restore process worked
for an hour or two and then suddenly restarted the server. I thought maybe it
was done, but BSOD appeared in the same place after restart.

So, proud of my idea to mirror the external drive, I snatched the drive from
the server, went to an unused workstation to mount the drive, share it up and
tell me users to get to their "cases" drive on PC03 instead of the server.
Brilliant, right?
Wrong...Windows XP didn't have the necessary drivers to mount the broken
half-mirror dynamic drive and I was told that because the server was 64 bit,
the 32 bit drivers probably wouldn't even help.

That's when I contacted Dell, then Microsoft and the MS tech helped me to
use the recovery console to manually replace about 10 files embedded deep in
Windows\System32.....

After 4.5 hours, the server finally booted and I was able to get to my
files. The U: volume (previously the mirrored volume) was mounted the I:
volume (the external half of the mirror with the out of date copies of U:/
files) , but the Cases were fine.

For the next couple of days, things appeared to go quite well. That is, of
course, until the Group Policy for Folder Redirection (which I had assigned
on the date that I used for restoral) tried to update the workstations by
telling them to redirect their documents to a share that didn't exist
(because of the restore).

Just tonight I found 3 basic errors on my workstations stating that the
Group Policy could not be applied or that a "service downgrade attack" had
been prevented or that the default storage path for user documents was not
available on-line and the workstation was no longer connected to the network.

Some workstations were fine (if they didn't have very many files to
redirect), but that didn't matter because the server was so slow trying to
resolve all of the incompatable Group Policies, missing shares, broken mirror
volume, etc.

The temporary fix I applied was to delete the scheduled task for Group
Policy deployment (to stop the server from telling the workstations to
redirect to a share that didn't exist) and remove all Folder Redirection
policies (allowing this new policy to propogate quickly). The I restarted
each workstation 3 times (to assure that the policy was applied) and checked
any other drive mappings and/or shortcuts.

This appears to have solved several of the problems, but I am not sure that
it is over. I am just not sure about the state of my server since the OS has
been pieced together in a frankensteinish mixup of backup states and
incompatible settings and the Best Practices tools tell me that everything is
GREAT and that the only error I have can be ignored...

(sorry about the long post, I just figured that perhaps it could help
someone else facing a similar dilemma)

Does this sound reasonable? What should I do from here and what advice do
you have about making sure that something like this never happens again?

Thanks for your patience, your encouragement, and your dedication--I really
appreciate it.

jase



"Susan Bradley" wrote:

Also flash the bios and the nic drivers up to the latest and greatest on
that server before you deploy it.

Susan Bradley wrote:
You didn't get to the underlying cause though :-(

Did you install Win2k8 sp2?
Did you install Drivers?

If you didn't do any of those two things, it could be nothing at all to
do with updating but a hardware/driver/subsystem issue.

You should have exported out the psts to make sure you didn't lose data.

http://www.forensit.com/domain-migration.html
Use that to move the profiles around.

jase wrote:
Susan,

Thanks a lot for all the time you have invested in helping me resolve
this issue.

After evaluating my options, I have decided to wipe the server and
re-install from scratch. It may not be the best method, but it will
undoubtedly take less time and provide more peace of mind than hunting
for some obscure setting I didn't know about.

I am also going to look into a third party backup system that does
bit-wise backups instead of the integrated backup that comes with
SBS2008...it has failed me twice (both times because of some obscure
technicality) and like Bush said, "Fool me once, shame on you, the
fool can't get fooled again": http://www.youtube.com/watch?v=eKgPY1adc0A

If you have any advice about re-installing an SBS2008 server on the
same hardware with around 8 users that only use the file-sharing and
e-mail functions (not power users, really) with focus on reliability
and availability...I'd be glad to hear it.

You have already spent more time on this issue than I intended and
thank you again for all your help.
Best wishes,

Jase

"Susan Bradley" wrote:

Sorry to take so long to get back to you, but I was
checking/comparing to a good box. DTC should be running and is
dependent on RPC.

"About 3/4 of the way through the restore, the
server restarted and did the exact same thing."

This either feels like a bad driver or something in the server is
going bad.

Your best bet would be to hang in there with Microsoft support and
have them keep looking at this.

My only other non drastic suggestion would be to ensure that TOE/RSS
is disabled on the nic cards and to ask you if Windows 2008 SP2 was
installed, and finally what antivirus you are using and if you'd
consider (temporarily) removing it from the server.

jase wrote:
Susan, thanks again for your help.

If IPv6 disabled on that server nic?
No, it's enabled.

Can you see if there is a .dmp file somewhere on the box?
Unfortunately, the debug settings were configured so no .dmp would
be created. I just changed it to Kernel, but that's not gonna help
us right this minute, is it? Sorry about that.

Also, here is the ipconfig/all from one of my workstations.

Windows IP Configuration
Host Name . . . . . . . . . . . . : PC01
Primary Dns Suffix . . . . . . . : ldwolfe.lan
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : ldwolfe.lan
ldwolfe.lan
Ethernet adapter Local Area Connection:
Connection-specific DNS Suffix . : ldwolfe.lan
Description . . . . . . . . . . . : Broadcom 440x 10/100
Integrated Controller
Physical Address. . . . . . . . . : 00-1A-A0-04-5D-41
Dhcp Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
IP Address. . . . . . . . . . . . : 10.10.10.101
Subnet Mask . . . . . . . . . . . : 255.255.255.0
IP Address. . . . . . . . . . . . : fe80::21a:a0ff:fe04:5d41%4
Default Gateway . . . . . . . . . : 10.10.10.254
DHCP Server . . . . . . . . . . . : 10.10.10.1
DNS Servers . . . . . . . . . . . : 10.10.10.1
fec0:0:0:ffff::1%1
fec0:0:0:ffff::2%1
fec0:0:0:ffff::3%1
Primary WINS Server . . . . . . . : 10.10.10.1
Lease Obtained. . . . . . . . . . : Friday, August 07, 2009
11:54:01 AM
Lease Expires . . . . . . . . . . : Monday, January 18, 2038
11:14:07 PM

Tunnel adapter Teredo Tunneling Pseudo-Interface:
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Teredo Tunneling
Pseudo-Interface
Physical Address. . . . . . . . . : FF-FF-FF-FF-FF-FF-FF-FF
Dhcp Enabled. . . . . . . . . . . : No
IP Address. . . . . . . . . . . . : fe80::ffff:ffff:fffd%5
Default Gateway . . . . . . . . . :
.

Relevant Pages

  • Re: Exchange 2000 Server Very Slow
    ... leak or you are having performance issues with your Disk drives. ... utilities on drives that contain Exchange databases. ... Remove 3rd party software from the server one at a time. ... > That symptom is the exact symptom to a badmail folder issue. ...
    (microsoft.public.exchange2000.general)
  • Re: backup solution for sbs?
    ... Restoring Your Server ... you can restore to the same computer or you can restore to new hardware. ... Before restoring your server, disconnect any external disk drives, such ... restore your server from the latest successful backup. ...
    (microsoft.public.windows.server.sbs)
  • Re: Backup Solutions
    ... I'm after suggestions as to go about backing up our server files on a small ... on alternate drives on alternate nights. ... the Yosemite backup software that came with the tape drive and use ntbackup. ... a like-to-like hardware restore ...
    (microsoft.public.win2000.advanced_server)
  • Re: Disaster Recovery
    ... We take a Server class DC and restore it to a PC and it ... When you are finished with the installation of the server create a folder on ... files with what you have in the backup folder BEFORE REBOOTING THE ...
    (microsoft.public.cert.exam.mcse)
  • Re: Can SQLDMO do this for me and how?
    ... > files on the SQL Server machine. ... >> an app in, say, vb .net, you can easily get drive, file, folder info and ... What I want to do is get a list of drives, files, and folders ... >>>like Enterprise Manger does when you are doing a backup or restore from ...
    (microsoft.public.sqlserver.programming)