Re: Exchange 2003 Private Store Corruption

I can't tell you with 100% certainty that yes, it will mount after your
reboot, but it seems like the odds are good.

You might want to have the phone number for Microsoft PSS handy just in case
things do go pear-shaped.

http://support.microsoft.com/?LN=en-us&scid=gp%3Ben-us%3Bofferprophone&x=11&y=14

--
Ben Winzenz
Exchange MVP
MessageOne
Read my blog!
http://winzenz.blogspot.com
http://feeds.feedburner.com/winzenz (RSS Feed)


"Corbin O'Reilly" <corbinoreilly@xxxxxxxxxxxxx> wrote in message
news:R8etg.41333$gv2.41141@xxxxxxxxxxxxxxxxxxxxxxxxx
Thanks Ben. I will follow your advice. Since the stores are currently
mounted do you think they will mount properly after I reboot?

"Ben Winzenz [Exchange MVP]" <ben_winzenz@nospamdotmessageonedotcom> wrote
in message news:%23stxcRfpGHA.4368@xxxxxxxxxxxxxxxxxxxxxxx
:-(

Yeah - look at the articles I posted in my last reply. -1018's are never
fun. I think you're on the right track with creating a new mailbox
store.

Once you've pulled the faulty RAM, create the new mailbox store and move
everyone that you can to that new store. I'd recommend that on the first
pass, leave the defaults - this will make it so that only the users with
problems will end up being skipped. Once you've moved everyone else,
then you can start working on the problem mailboxes.

Once you have everyone else moved, dismount the problem mailbox store,
and create a copy of it. Then, remount it, and do the move mailbox
again, this time telling it to skip corrupt items. You may have to try
the move a few times, bumping up the amount of corrupt items it can
encounter.

Depending on how much data is lost, you can try to either restore from
backup to the Recovery Storage Group, and recover lost mail that way, or
you can use something like Quest's Recovery manager to do the same thing.

You could also at that point look at using eseutil against the copy to
try and fix it, then mount it to the Recovery Storage Group.

--
Ben Winzenz
Exchange MVP
MessageOne
Read my blog!
http://winzenz.blogspot.com
http://feeds.feedburner.com/winzenz (RSS Feed)


"Corbin O'Reilly" <corbinoreilly@xxxxxxxxxxxxx> wrote in message
news:F2dtg.8317$IF2.1593@xxxxxxxxxxxxxxxxxxxxxxxxx
Hi Ben. Here is the error I am getting. Please advise. Thanks.
Information Store (2856) First Storage Group: The database page read
from the file "E:\NewMailboxStore\NewMailbox Store (GTMAIL).edb" at
offset 5251796992 (0x0000000139081000) (database page 1282176
(0x139080)) for 4096 (0x00001000) bytes failed verification due to a
page checksum mismatch. The expected checksum was 1873244556488189379
(0x19ff19ffcee9a5c3) and the actual checksum was 5178919806808169513
(0x47df38200d091c29). The read operation will fail with error -1018
(0xfffffc06). If this condition persists then please restore the
database from a previous backup. This problem is likely due to faulty
hardware. Please contact your hardware vendor for further assistance
diagnosing the problem.

For more information, click
http://www.microsoft.com/contentredirect.asp.

For more information, see Help and Support Center at


"Ben Winzenz [Exchange MVP]" <ben_winzenz@nospamdotmessageonedotcom>
wrote in message news:u$BCCMdpGHA.524@xxxxxxxxxxxxxxxxxxxxxxx
It could be that it is just DIMM 4 that is the problem. Do you have to
have DIMM's installed in pairs in this server? If not, I'd just pull
out DIMM 4.

Also, can you post the entire contents of the error event? Does it
mention something to the effect of

Information Store (<PID>) The database page read from the file "<path
to edb file>" at offset <offset> for <value> bytes failed verification
due to a page checksum mismatch. The expected checksum was <checksum>
and the actual checksum was <checksum>. The read operation will fail
with error <error code> (<error code>). If this condition persists then
please restore the database from a previous backup.

and is the error code referenced a -1018 error? If so, then that *is*
a database corruption issue, not just a corrupt mailbox. Even if that
is the case, I still wouldn't recommend using eseutil or isinteg -
those are really last resort utilities. Moving mailboxes is one
option. Restoring from backup should be another option you should look
at, or restoring from backup to the Recovery Storage Group.

You might also look at the storage group properties and see if you have
the option "Zero out deleted database pages" option checked, and if so,
you might consider unchecking it.

Also, do you have any Exchange 2003 SP's installed? If not, once you
get this fixed, you should really make sure you are up to date. SP1
includes an additional error correcting code that is designed to fix
many of the -1018 errors that are as a result of a flipped bit.

Here are some KB articles that talk about 474 errors and -1018 errors.
http://support.microsoft.com/default.aspx?scid=kb;en-us;810411
http://support.microsoft.com/kb/327334/
http://support.microsoft.com/default.aspx?scid=kb;en-us;867626
http://support.microsoft.com/default.aspx?scid=kb;en-us;314917
http://support.microsoft.com/default.aspx?scid=kb;en-us;812531 -
webcast on understanding and resolving -1018 errors.

In your case, it really would appear that faulty memory is the
underlying cause.

--
Ben Winzenz
Exchange MVP
MessageOne
Read my blog!
http://winzenz.blogspot.com
http://feeds.feedburner.com/winzenz (RSS Feed)


"Corbin O'Reilly" <corbinoreilly@xxxxxxxxxxxxx> wrote in message
news:uj9tg.89246$qd2.10626@xxxxxxxxxxxxxxxxxxxxxxxxx
Thanks for the info Ben. Yes they are mounted now so hopefully you are
right and they will mount properly after the reboot. The HP Insight
Online Diagnostic Utility says "Correctable Memory Threshold Exceeded"
for DIMM 4 so it does look like the memory is faulty. Have you seen
474 errors before? In your experience what leads to a 474 error? The
description points to hardware. I am exporting mailboxes to PST files
right now. Did you see Mitch's previous response. I am just trying to
get the server back up and running without the faulty memory
installed. My plan is to create a new store, move as many uncorrupted
mailboxes as I can to it, and then try to repair the old stores.

"Ben Winzenz [Exchange MVP]" <ben_winzenz@nospamdotmessageonedotcom>
wrote in message news:%23iIUsWcpGHA.3600@xxxxxxxxxxxxxxxxxxxxxxx
Inline.

--
Ben Winzenz
Exchange MVP
MessageOne
Read my blog!
http://winzenz.blogspot.com
http://feeds.feedburner.com/winzenz (RSS Feed)


"Corbin O'Reilly" <corbinoreilly@xxxxxxxxxxxxx> wrote in message
news:OX7tg.8250$IF2.4659@xxxxxxxxxxxxxxxxxxxxxxxxx
Hello everyone. I need some help. We have an Exchange 2003
Enterprise Edition Server running Windows 2003 Enterprise Edition R2
server. The machine is a HP DL380 G4 server with 2 GB of memory and
has been online for about two months. The only thing running on this
server is Exchange 2003 and Trend-Micro AntiVirus for Exchange. The
server came standard with 2 - 512 MB HP-brand DIMMs. We purchased
additional memory from CDW. They are 2 -512 MB EDGE-brand DIMMs and
are supposed to be compatible. On the first day that we powered up
the server the server said there was an issue with DIMM 4, the
EDGE-brand chip. After a reboot the error went away. About three
weeks ago I noticed that the server had not completed its online
backup properly and two of our employees were having Calendar
issues. I checked the Event Viewer and it had tons of 474 errors
which says that there is probably a hardware problem. I created a
new store on the server and moved all mailboxes to the new store.
The two mailboxes with the corrupted Calendars would not move so I
left them in the corrupted store until I got back from vacation.
While on vacation for two weeks I checked the NTBACKUP logs daily
and the online backups were completing properly. Everything was fine
for almost the entire time I was gone. The day before I got back
from vacation I got calls from 5 employees saying they were having
strange Exchange problems. Some were not getting new e-mails (we use
cached mode), others had Inboxes that were empty, etc. I checked the
NTBACKUP log and sure enough the new store I created is now
corrupted too with a 474 error. My theory is that the EDGE-brand
memory chip is acting up and corrupting the store. What I would like
to do is shutdown the server and remove the 2 - 512 MB EDGE-brand
memory DIMMs from server, and boot it back up. I have a couple of
questions:

1. The server is partitioned into a RAID 1 OS C-Drive and a RAID 6
Exchange Data E-Drive. The server currently has 2 GB of memory and
the page file is located on the C-Drive and is set to Initial
: 3072 and Maximum: 6144. Since I will be removing the 2 - 512 MB
EDGE-brand DIMMs the new total memory size will be 1024 MB (1GB).
Should I adjust the page file to Initial: 1536 and Maximum 3072 in
Windows 2003 BEFORE or AFTER I remove the memory?

I don't think it much matters whether you do it before or after,
though, except that if you do it afterwards, you will have to reboot
a second time.


2. Will the server perform OK with 1 GB of memory for a while? We
have about 75 mailboxes and a few public folders.

More than likely, it will be just fine. It doesn't sound like your
server is heavily loaded


3. Since both Private Info stores are corrupted will they mount
properly when the server is rebooted?

Are they mounted right now? If so, then once you reboot, they should
be able to mount. Dealing with corrupt mailboxes is somewhat
different than dealing with an actual corrupt database. Corrupt
databases are not mountable. Databases with corrupt mailboxes
usually are mountable, but the mailbox itself may or not be able to
be fixed. I'd consider using Exmerge to export whatever you can from
the corrupt mailboxes. If they can be moved, great. If not, then the
mailboxes should be deleted and re-create on the new store.


I am trying desperately to avoid being dead in the water tonight. My
concern is that the stores will not come up and we will be totally
down. I am getting more complaints from different employees so the
problem seems to be spreading and getting worse. I would appreciate
your expertise on this problem. Thanks.

As an additional note, have you run any server diagnostics to test
the new memory? What is your plan if removing the memory doesn't
curtail the problem? It's really better to have a true diagnosis
that the memory is faulty before you pull it out. HP should have
some diagnostics that you can run to test the memory. I'd do that
first before you pull it out.
















.

Relevant Pages

  • Re: Exchange 2003 Private Store Corruption
    ... I think you're on the right track with creating a new mailbox store. ... Moving mailboxes is one option. ... Online Diagnostic Utility says "Correctable Memory Threshold Exceeded" ... get the server back up and running without the faulty memory installed. ...
    (microsoft.public.exchange.admin)
  • Re: no mailboxs in private foulder
    ... limited experince with exchange. ... Ping the exchange server ... Right click the mailbox store, is the first option "mount store" or is it ... Where are you seeing "Mailboxes and stuff"? ...
    (microsoft.public.exchange.admin)
  • Re: Exchange 2003 Private Store Corruption
    ... Information Store First Storage Group: The database page read from ... Online Diagnostic Utility says "Correctable Memory Threshold Exceeded" ... the server back up and running without the faulty memory installed. ...
    (microsoft.public.exchange.admin)
  • Re: public folders are not all in sync
    ... Exchange 2003 server about the calendar connector. ... then just dismount the public store. ... Network Administrator mailboxes are supposed to be moved. ... I don't have very many folders so I ...
    (microsoft.public.exchange.admin)
  • Re: Exchange 2003 Private Store Corruption
    ... Diagnostic Utility says "Correctable Memory Threshold Exceeded" for DIMM 4 ... Edition Server running Windows 2003 Enterprise Edition R2 server. ... I created a new store on ... Dealing with corrupt mailboxes is somewhat different than ...
    (microsoft.public.exchange.admin)