Re: IIS State Analysis

Tech-Archive recommends: Fix windows errors by optimizing your registry

From: Pete Conlan (anonymous_at_discussions.microsoft.com)
Date: 03/23/04


Date: Tue, 23 Mar 2004 06:21:52 -0800

Thanks Pat.

>-----Original Message-----
>The OLEDB threads are utility threads (they handle
things like cache
>scavenging, etc.). 38 & 39 are utility threads for
handling calls into the
>DTC process.
>
>Thread 30 is a 'heartbeat' thread to make sure things
are still working.
>
>Overall, this is a process that has done some work but
is currently idle.
>
>
>Pat
>
>"Peter Conlan" <anonymous@discussions.microsoft.com>
wrote in message
>news:11cfe01c41052$41459b60$a101280a@phx.gbl...
>> I would appreciate some help with an IIS State
analysis.
>> We have a 3 server web cluster, and the following was
>> taken on one of the servers after it was suspended from
>> the cluster. Often when we do this, there will be web
>> connections (looking at Web Service\Current
Connections)
>> that do not go away. Looking at the log file, it
appears
>> that some of the threads are still doing things and are
>> in WAIT states. In particular:
>>
>> -Threads 28 and 29 look like they are doing something
>> with OLEDB. We use an SQL server database. Is there a
>> symbol file we could get that would give us more info?
>>
>> - Threads 30 is doing some "PingThread" function. What
>> is that?
>>
>> - Thread 33,53, 54,55 look like they are doing the same
>> thing, but don't know what it is.
>>
>> - Threads 38 and 39 look like they are
using "Distributed
>> Transaction". What could that be?
>>
>>
>> Any help would be appreciated.
>>
>> Opened log file 'E:\Scripts\output\IISState-1736.log'
>>
>> ***********************
>> Starting new log output
>> IISState version 3.3.1
>>
>> Mon Mar 22 14:22:15 2004
>>
>> OS = Windows 2000
>> Executable: dllhost.exe
>> PID = 1736
>>
>> Note: Thread times are formatted as HH:MM:SS.ms
>>
>> ***********************
>>
>>
>>
>>
>> Thread ID: 0
>> System Thread ID: 6c0
>> Kernel Time: 0:0:0.15
>> User Time: 0:0:0.0
>> Thread Status: Thread is in a WAIT state.
>> Thread Type: Other
>> # ChildEBP RetAddr
>> 00 0006fd28 7c599e8e ntdll!NtWaitForSingleObject+0xb
>> 01 0006fd50 7c57b3d5 KERNEL32!
WaitForSingleObjectEx+0x71
>> 02 0006fd60 77aaa483 KERNEL32!WaitForSingleObject+0xf
>> 03 0006fd80 77aa9c91 ole32!
>>
CSurrogateProcessActivator::WaitForSurrogateTimeout+0x4f
>> 04 0006fd9c 01001230 ole32!CoRegisterSurrogateEx+0x169
>> 05 0006ff24 010014c6 dllhost!WinMain+0xb0
>> 06 0006ffc0 7c5987e7 dllhost!WinMainCRTStartup+0x156
>> 07 0006fff0 00000000 KERNEL32!BaseProcessStart+0x3d
>>
>>
>>
>> Thread ID: 4
>> System Thread ID: 814
>> Kernel Time: 0:0:0.0
>> User Time: 0:0:0.0
>> Thread Type: Possible ASP page. Possible DCOM activity
>> Executing Page: Unable to locate ASP page
>>
>> Remote call is either to a MTA object or object not
>> initialized. Also, possible utility thread.
>> DCOM call being made to Process ID: 1696
>> Waiting on thread id: ffffffff
>>
>> # ChildEBP RetAddr
>> 00 00cffb68 77d4ec77 ntdll!NtRequestWaitReplyPort+0xb
>> 01 00cffb94 77d3a2c7 RPCRT4!
LRPC_CCALL::SendReceive+0x11e
>> 02 00cffba0 77b23b2c RPCRT4!I_RpcSendReceive+0x2c
>> 03 00cffbc0 77b239f7 ole32!ThreadSendReceive+0xef
>> 04 00cffbd8 77b20aa5 ole32!
>> CRpcChannelBuffer::SwitchAptAndDispatchCall+0x14a
>> 05 00cffc18 77b23870 ole32!
>> CRpcChannelBuffer::SendReceive2+0x96
>> 06 00cffc28 77a6c767 ole32!
>> CRpcChannelBuffer::SendReceive+0x11
>> 07 00cffc88 77ab6ac3 ole32!
CAptRpcChnl::SendReceive+0xa9
>> 08 00cffce0 77d90328 ole32!
CCtxComChnl::SendReceive+0x124
>> 09 00cffcfc 77d92b3f RPCRT4!NdrProxySendReceive+0x4c
>> 0a 00cfff44 77d95f85 RPCRT4!NdrClientCall2+0x4f5
>> 0b 00cfff60 77d77fcb RPCRT4!ObjectStublessClient+0x76
>> 0c 00cfff70 787f372e RPCRT4!ObjectStubless+0xf
>> 0d 00cfffb4 7c57b382 COMSVCS!
>> CEventDispatcher::GetEventServerInfoThread+0x10e
>> 0e 00cfffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 12
>> System Thread ID: 838
>> Kernel Time: 0:0:47.203
>> User Time: 0:1:1.859
>> Thread Type: Possible ASP page. Possible DCOM activity
>> Executing Page: Unable to locate ASP page
>>
>> No remote call being made
>> # ChildEBP RetAddr
>> 00 0134feb8 7c585323 ntdll!NtRemoveIoCompletion+0xb
>> 01 0134fee4 77d809da KERNEL32!
>> GetQueuedCompletionStatus+0x27
>> 02 0134ff20 77d50ede RPCRT4!COMMON_ProcessCalls+0x9e
>> 03 0134ff74 77d50d17 RPCRT4!
>> LOADABLE_TRANSPORT::ProcessIOEvents+0x99
>> 04 0134ff78 77d39a00 RPCRT4!ProcessIOEventsWrapper+0x9
>> 05 0134ffa8 77d41c6c RPCRT4!
BaseCachedThreadRoutine+0x4f
>> 06 0134ffb4 7c57b382 RPCRT4!ThreadStartRoutine+0x18
>> 07 0134ffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 13
>> System Thread ID: 840
>> Kernel Time: 0:0:59.375
>> User Time: 0:1:14.734
>> Thread Type: Possible ASP page. Possible DCOM activity
>> Executing Page: Unable to locate ASP page
>>
>> No remote call being made
>> # ChildEBP RetAddr
>> 00 0162feb8 7c585323 ntdll!NtRemoveIoCompletion+0xb
>> 01 0162fee4 77d809da KERNEL32!
>> GetQueuedCompletionStatus+0x27
>> 02 0162ff20 77d50ede RPCRT4!COMMON_ProcessCalls+0x9e
>> 03 0162ff74 77d50d17 RPCRT4!
>> LOADABLE_TRANSPORT::ProcessIOEvents+0x99
>> 04 0162ff78 77d39ad0 RPCRT4!ProcessIOEventsWrapper+0x9
>> 05 0162ffa8 77d41c6c RPCRT4!
BaseCachedThreadRoutine+0x11f
>> 06 0162ffb4 7c57b382 RPCRT4!ThreadStartRoutine+0x18
>> 07 0162ffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 14
>> System Thread ID: 844
>> Kernel Time: 0:0:0.0
>> User Time: 0:0:0.0
>> Thread Type: ASP
>> Executing Page: Unable to locate ASP page
>>
>> # ChildEBP RetAddr
>> 00 0214fe70 7c59a059 ntdll!ZwWaitForMultipleObjects+0xb
>> 01 0214fec0 77e1e9fb KERNEL32!
>> WaitForMultipleObjectsEx+0xea
>> 02 0214ff1c 77e1ea48 USER32!
>> MsgWaitForMultipleObjectsEx+0x153
>> 03 0214ff38 74a01e69 USER32!
MsgWaitForMultipleObjects+0x1d
>> 04 0214ff7c 78008454 asp!
CMTACallbackThread::Thread+0x42
>> 05 0214ffb4 7c57b382 MSVCRT!_endthread+0xc6
>> 06 0214ffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 15
>> System Thread ID: 84c
>> Kernel Time: 0:0:0.0
>> User Time: 0:0:0.15
>> *** WARNING: Unable to verify checksum for
>> *** ERROR: Symbol file could not be found. Defaulted
to
>> export symbols for -
>> Thread Type: PDM (Debugger) Thread.
>> # ChildEBP RetAddr
>> 00 021cfddc 7c59a059 ntdll!ZwWaitForMultipleObjects+0xb
>> 01 021cfe2c 77e1e9fb KERNEL32!
>> WaitForMultipleObjectsEx+0xea
>> 02 021cfe88 77e1ea48 USER32!
>> MsgWaitForMultipleObjectsEx+0x153
>> 03 021cfea4 4a00886c USER32!
MsgWaitForMultipleObjects+0x1d
>> 04 021cff88 4a008a85 pdm+0x886c
>> 05 021cffb0 4a008a09 pdm+0x8a85
>> 06 021cffb4 7c57b382 pdm+0x8a09
>> 07 021cffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>>
>> Thread ID: 28
>> System Thread ID: 884
>> Kernel Time: 0:6:52.203
>> User Time: 0:13:3.859
>> *** ERROR: Symbol file could not be found. Defaulted
to
>> export symbols for C:\Program Files\Common
>> Files\System\OLE DB\oledb32.dll -
>> Thread Status: Thread is in a WAIT state.
>> Thread Type: Other
>> # ChildEBP RetAddr
>> 00 02e7ff5c 7c599e8e ntdll!NtWaitForSingleObject+0xb
>> 01 02e7ff84 7c57b3d5 KERNEL32!
WaitForSingleObjectEx+0x71
>> 02 02e7ff94 1f93cf88 KERNEL32!WaitForSingleObject+0xf
>> WARNING: Stack unwind information not available.
>> Following frames may be wrong.
>> 03 02e7ffb4 7c57b382 oledb32!DllGetClassObject+0xa470
>> 04 02e7ffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 29
>> System Thread ID: 888
>> Kernel Time: 0:10:29.671
>> User Time: 0:7:2.296
>> Thread Status: Thread is in a WAIT state.
>> Thread Type: Other
>> # ChildEBP RetAddr
>> 00 02ebff58 7c599e8e ntdll!NtWaitForSingleObject+0xb
>> 01 02ebff80 7c57b3d5 KERNEL32!
WaitForSingleObjectEx+0x71
>> 02 02ebff90 1f93d152 KERNEL32!WaitForSingleObject+0xf
>> WARNING: Stack unwind information not available.
>> Following frames may be wrong.
>> 03 02ebffb4 7c57b382 oledb32!DllGetClassObject+0xa63a
>> 04 02ebffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 30
>> System Thread ID: 88c
>> Kernel Time: 0:0:0.15
>> User Time: 0:0:0.62
>> Thread Status: Thread is in a WAIT state.
>> Thread Type: Idle ASP thread
>> # ChildEBP RetAddr
>> 00 02effd54 7c599e8e ntdll!NtWaitForSingleObject+0xb
>> 01 02effd7c 7c57b3d5 KERNEL32!
WaitForSingleObjectEx+0x71
>> 02 02effd8c 7878d425 KERNEL32!WaitForSingleObject+0xf
>> 03 02efffb4 7c57b382 COMSVCS!PingThread+0xf5
>> 04 02efffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 31
>> System Thread ID: 890
>> Kernel Time: 0:0:0.0
>> User Time: 0:0:0.0
>> Thread Type: Other
>> # ChildEBP RetAddr
>> 00 0308ff20 77f8c35a ntdll!NtRemoveIoCompletion+0xb
>> 01 0308ffb4 7c57b382 ntdll!RtlpWorkerThread+0x6b
>> 02 0308ffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 32
>> System Thread ID: 4f0
>> Kernel Time: 0:0:0.0
>> User Time: 0:0:0.0
>> Thread Type: Other
>> # ChildEBP RetAddr
>> 00 060bfea4 7c59a059 ntdll!ZwWaitForMultipleObjects+0xb
>> 01 060bfef4 7c599f6c KERNEL32!
>> WaitForMultipleObjectsEx+0xea
>> 02 060bff0c 743d78d7 KERNEL32!
WaitForMultipleObjects+0x17
>> 03 060bff54 743d7a3d browscap!
CMonitor::DoMonitoring+0xf9
>> 04 060bff7c 780085bc browscap!CMonitor::ThreadFunc+0x26
>> 05 060bffb4 7c57b382 MSVCRT!_endthreadex+0xc1
>> 06 060bffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 33
>> System Thread ID: 894
>> Kernel Time: 0:0:32.484
>> User Time: 0:0:42.140
>> Thread Type: Possible ASP page. Possible DCOM activity
>> Executing Page: Unable to locate ASP page
>>
>> No remote call being made
>> # ChildEBP RetAddr
>> 00 1276feb8 7c585323 ntdll!NtRemoveIoCompletion+0xb
>> 01 1276fee4 77d809da KERNEL32!
>> GetQueuedCompletionStatus+0x27
>> 02 1276ff20 77d50ede RPCRT4!COMMON_ProcessCalls+0x9e
>> 03 1276ff74 77d50d17 RPCRT4!
>> LOADABLE_TRANSPORT::ProcessIOEvents+0x99
>> 04 1276ff78 77d39a00 RPCRT4!ProcessIOEventsWrapper+0x9
>> 05 1276ffa8 77d41c6c RPCRT4!
BaseCachedThreadRoutine+0x4f
>> 06 1276ffb4 7c57b382 RPCRT4!ThreadStartRoutine+0x18
>> 07 1276ffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>>
>> Thread ID: 38
>> System Thread ID: a20
>> Kernel Time: 0:0:0.46
>> User Time: 0:0:0.15
>> Thread Type: Distributed Transaction. Checking for
>> running ASP page
>> Executing Page: Unable to locate ASP page
>>
>> # ChildEBP RetAddr
>> 00 00a5ff50 7c59a0ed ntdll!NtDelayExecution+0xb
>> 01 00a5ff70 7c59a0b8 KERNEL32!SleepEx+0x32
>> 02 00a5ff7c 6dfcead3 KERNEL32!Sleep+0xb
>> 03 00a5ffb0 6dfcea2b MSDTCPRX!
>> CConnectionManager::TimerProcForNonGuardedUser+0xa3
>> 04 00a5ffb4 7c57b382 MSDTCPRX!
>> CConnectionManager::TimerProc+0x1b
>> 05 00a5ffc0 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 39
>> System Thread ID: a24
>> Kernel Time: 0:0:0.0
>> User Time: 0:0:0.0
>> Thread Status: Thread is in a WAIT state.
>> Thread Type: Distributed Transaction. Checking for
>> running ASP page
>> Executing Page: Unable to locate ASP page
>>
>> # ChildEBP RetAddr
>> 00 0362fd38 7c599e8e ntdll!NtWaitForSingleObject+0xb
>> 01 0362fd60 7c57b3d5 KERNEL32!
WaitForSingleObjectEx+0x71
>> 02 0362fd70 6dfd81a7 KERNEL32!WaitForSingleObject+0xf
>> 03 0362ff90 6dfd767f MSDTCPRX!
>> CSessionObject::MaintainItInUpState+0x35a
>> 04 0362ffb0 6dfd73fd MSDTCPRX!
>> CSessionObject::MaintainIt+0x243
>> 05 0362ffb4 7c57b382 MSDTCPRX!
>> CSessionObject::MaintainSession+0xd
>> 06 0362ffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>> Thread ID: 45
>> System Thread ID: c80
>> Kernel Time: 0:0:0.93
>> User Time: 0:0:0.31
>> *** ERROR: Symbol file could not be found. Defaulted
to
>> export symbols for -
>> Thread Type: Other
>> # ChildEBP RetAddr
>> 00 054afe3c 7c59a059 ntdll!ZwWaitForMultipleObjects+0xb
>> 01 054afe8c 77e1e9fb KERNEL32!
>> WaitForMultipleObjectsEx+0xea
>> 02 054afee8 77e1ea48 USER32!
>> MsgWaitForMultipleObjectsEx+0x153
>> 03 054aff04 1e4d9282 USER32!
MsgWaitForMultipleObjects+0x1d
>> WARNING: Stack unwind information not available.
>> Following frames may be wrong.
>> 04 054aff74 1e4d9204 msolap80!DllRegisterServer+0xd56f
>> 05 054aff7c 78008454 msolap80!DllRegisterServer+0xd4f1
>> 06 054affb4 7c57b382 MSVCRT!_endthread+0xc6
>> 07 054affec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 53
>> System Thread ID: 101c
>> Kernel Time: 0:1:0.171
>> User Time: 0:1:33.390
>> Thread Type: Possible ASP page. Possible DCOM activity
>> Executing Page: Unable to locate ASP page
>>
>> No remote call being made
>> # ChildEBP RetAddr
>> 00 24f0fe24 77d574d4 ntdll!NtReplyWaitReceivePortEx+0xb
>> 01 24f0ff74 77d56d9e RPCRT4!
>> LRPC_ADDRESS::ReceiveLotsaCalls+0x74
>> 02 24f0ff78 77d39a00 RPCRT4!RecvLotsaCallsWrapper+0x9
>> 03 24f0ffa8 77d41c6c RPCRT4!
BaseCachedThreadRoutine+0x4f
>> 04 24f0ffb4 7c57b382 RPCRT4!ThreadStartRoutine+0x18
>> 05 24f0ffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 54
>> System Thread ID: 100c
>> Kernel Time: 0:1:3.765
>> User Time: 0:1:35.859
>> Thread Type: Possible ASP page. Possible DCOM activity
>> Executing Page: Unable to locate ASP page
>>
>> No remote call being made
>> # ChildEBP RetAddr
>> 00 04e8fe24 77d574d4 ntdll!NtReplyWaitReceivePortEx+0xb
>> 01 04e8ff74 77d56d9e RPCRT4!
>> LRPC_ADDRESS::ReceiveLotsaCalls+0x74
>> 02 04e8ff78 77d39a00 RPCRT4!RecvLotsaCallsWrapper+0x9
>> 03 04e8ffa8 77d41c6c RPCRT4!
BaseCachedThreadRoutine+0x4f
>> 04 04e8ffb4 7c57b382 RPCRT4!ThreadStartRoutine+0x18
>> 05 04e8ffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 55
>> System Thread ID: e1c
>> Kernel Time: 0:0:31.562
>> User Time: 0:0:50.62
>> Thread Type: Possible ASP page. Possible DCOM activity
>> Executing Page: Unable to locate ASP page
>>
>> No remote call being made
>> # ChildEBP RetAddr
>> 00 0402fe24 77d574d4 ntdll!NtReplyWaitReceivePortEx+0xb
>> 01 0402ff74 77d56d9e RPCRT4!
>> LRPC_ADDRESS::ReceiveLotsaCalls+0x74
>> 02 0402ff78 77d39a00 RPCRT4!RecvLotsaCallsWrapper+0x9
>> 03 0402ffa8 77d41c6c RPCRT4!
BaseCachedThreadRoutine+0x4f
>> 04 0402ffb4 7c57b382 RPCRT4!ThreadStartRoutine+0x18
>> 05 0402ffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 56
>> System Thread ID: f0c
>> Kernel Time: 0:0:0.0
>> User Time: 0:0:0.0
>> Thread Type: Other
>> # ChildEBP RetAddr
>> 00 0088ff5c 7c59a0ed ntdll!NtDelayExecution+0xb
>> 01 0088ff7c 7c59a0b8 KERNEL32!SleepEx+0x32
>> 02 0088ff88 77ab85fc KERNEL32!Sleep+0xb
>> 03 0088ff90 77ab46fb ole32!
>> CROIDTable::WorkerThreadLoop+0xc
>> 04 0088ffa8 77ab4653 ole32!CRpcThread::WorkerLoop+0x22
>> 05 0088ffb4 7c57b382 ole32!
>> CRpcThreadCache::RpcWorkerThreadEntry+0x1a
>> 06 0088ffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>>
>> Thread ID: 57
>> System Thread ID: fa0
>> Kernel Time: 0:0:5.125
>> User Time: 0:0:8.296
>> Thread Type: Possible ASP page. Possible DCOM activity
>> Executing Page: Unable to locate ASP page
>>
>> No remote call being made
>> # ChildEBP RetAddr
>> 00 03fcfe24 77d574d4 ntdll!NtReplyWaitReceivePortEx+0xb
>> 01 03fcff74 77d56d9e RPCRT4!
>> LRPC_ADDRESS::ReceiveLotsaCalls+0x74
>> 02 03fcff78 77d39a00 RPCRT4!RecvLotsaCallsWrapper+0x9
>> 03 03fcffa8 77d41c6c RPCRT4!
BaseCachedThreadRoutine+0x4f
>> 04 03fcffb4 7c57b382 RPCRT4!ThreadStartRoutine+0x18
>> 05 03fcffec 00000000 KERNEL32!BaseThreadStart+0x52
>>
>>
>>
>> *****
>>
>> Dump name is formatted as: PID-Timestamp.dmp
>>
>> Creating E:\Scripts\output\1736-1079983340.dmp - mini
>> user dump
>>
>> *****
>>
>> Closing open log file E:\Scripts\output\IISState-
1736.log
>>
>
>
>.
>



Relevant Pages

  • Re: IIState log
    ... >System Thread ID: f9c ... >Thread Type: Other ... > # ChildEBP RetAddr ... >No remote call being made ...
    (microsoft.public.inetserver.iis)
  • Re: *Painful* iis admin service stopping every few minutes
    ... > The thread that was executing is completely corrupted. ... >> # ChildEBP RetAddr ... >> No remote call being made ... >> lock owner. ...
    (microsoft.public.inetserver.iis)
  • Re: IISState Log Help
    ... and upgraded to W2K SP4 and IIS 5.0 and the problem stayed. ... >> # ChildEBP RetAddr ... >> No remote call being made ... >> frames may be wrong. ...
    (microsoft.public.inetserver.iis)
  • Re: IISState analysis - Thread waiting on PID 0?
    ... >> We often have requests get hung and then the ASP queue ... >> # ChildEBP RetAddr ... >> Remote call is either to a MTA object or object not ... Following frames may ...
    (microsoft.public.inetserver.iis)