Re: ASP Thread Scheduling Anomalies

Tech-Archive recommends: Fix windows errors by optimizing your registry

From: Pat [MSFT] (patfilot_at_online.microsoft.com)
Date: 03/03/05 

Date: Wed, 2 Mar 2005 21:11:47 -0800

This is the only thread executing:

Thread ID: 21
System Thread ID: 928
Kernel Time: 0:0:2.78
User Time: 0:0:5.921
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\WINDOWS\system32\MSVBVM60.DLL -
Thread Type: ASP
Executing Page:

 # ChildEBP RetAddr
00 0264dec0 02ca3367 msado15!CRsetField::SetupDataAccessor+0x29d
01 0264dee4 02ca2604 msado15!CStdCollection::LookupItem+0x14e
02 0264def0 02ca7eb2 msado15!CStdComObjectRoot::InternalAddRef+0x10
03 0264df20 02ca279c msado15!CRsetFields::get_Item+0x3e
04 0264df3c 110b7b6d msado15!ATL::CComObject<CCommand>::AddRef+0xc
05 0264df48 660d9a27 ACExec!ACServices::LoadObject+0x2a3d
WARNING: Stack unwind information not available. Following frames may be
wrong.
06 0264e34c 040ccd98 MSVBVM60!_vbaFreeObj
07 00000000 00000000 0x40ccd98

I can't say why or even if this is a real problem. The thread has only used
5 seconds of time. On the good news side...No XML to be found. You can try
isolating the ACExec.dll to a separate COM+ server package and see if the
problem follows or not. Also, if you build the dll with symbols and place
the symbol file in the same directory with the dll, IISState will pick it up
and give you line number and .cls file names.

Pat

"Tony Proctor" <tony_proctor@aimtechnology_NoMoreSPAM_.com> wrote in message
news:OuQjH91HFHA.580@TK2MSFTNGP15.phx.gbl...
> Thanks for the response Pat
>
> I can confirm that there is no apartment-threading issue, or cross-thread
> marshalling. These are issue we were aware of before the application was
> designed.
>
> The piece about the 4-thread increments is interesting. It explains a few
> of
> the observations we've made. Also, it makes pretty obvious sense to try
> and
> use threads FIFO to take advantage of processor caches, and even minimise
> paging of TLS. Hence, I've always been a bit suspicious of the scheduling
> characteristics we've so far taken for granted (i.e. round-robin across
> all
> threads).
>
> I've attached an IISState log from a previous failure run involving
> multiple
> ASP Sessions (my recent test case involved just a single Session). I did
> check the log myself but it appeared OK to the "untrained eye". At the
> time
> of the log, only one ASP thread was being used.
>
> Only a few moments ago, we uncovered some more clues. The one ingredient
> that explained the difference between my real server and my cut-down
> server
> was a WithEvents on an MSXML2.Document variable in a VB class. Without it,
> there is no acquired thread affinity when the DOMs are cleared down, and
> only 2 ASP threads are ever used for my single Session (requests evenly
> oscillating between them). With the keyword, all 20 ASP threads are used
> until the DOMs are cleared down on one thread. The other threads are then
> no
> longer used. The handle count is also significantly greater in this
> failure
> case (500 as opposed to 280 in the working case).
>
> Although this points the finger back at MSXML, could you suggest a
> criterion
> that would cause ASP threads to be avoided like this Pat? In the
> multi-Session case, we eventually end up with many requests queued up and
> only one ASP thread handling them all.
>
> Tony Proctor
>
> "Pat [MSFT]" <patfilot@online.microsoft.com> wrote in message
> news:#pTLsN1HFHA.3536@TK2MSFTNGP14.phx.gbl...
>> ASP.dll does not directly schedule threads (i.e. they are not fibres).
> ASP
>> established a thread pool and passes requests to the thread pool. The OS
>> schedules threads.
>>
>> The thread scheduler will try to reuse an idle thread that was recently
> run
>> over a thread that has not run in a while (i.e. FIFO), but this is not
>> guaranteed. This is an attempt to preserve the L1/L2 cache and improve
>> performance. You cannot guarantee which thread will service a given ASP
>> page. Also, it is important to differentiate if a single thread is
>> executing the entire ASP page or if a different ASP page is starting the
>> executing and then marshalling over to the other thread b/c of a COM
>> apartment issue. One other point is that the thread pool size will grow
> on
>> demand up to the configured maximum threshold. I think by default it
>> only
>> creates 4 threads. So you have to create some degree of parallel stress
> to
>> even get it to create the 20 threads you are looking for.
>>
>> I would recommend that you get an iisstate log of the problem occurring
> and
>> post it. We can take a look and see if we can identify the cause of the
>> serialization.
>>
>>
>> Pat
>>
>>
>>
>> "Tony Proctor" <tony_proctor@aimtechnology_NoMoreSPAM_.com> wrote in
> message
>> news:%237sljSzHFHA.2476@TK2MSFTNGP12.phx.gbl...
>> > Thanks Brian. No, that's not the reason. The application is
> multi-threaded
>> > and well-behaved until one thread decides to clear something down
>> >
>> > Tony Proctor
>> >
>> > "Brian Staff" <brianstaff@[NoSpam]cox.net> wrote in message
>> > news:sMjVd.43600$xt.1828@fed1read07...
>> >> Tony,
>> >>
>> >> This is from the IIS help files...maybe you have server-side debugging
>> >> turned ON?
>> >>
>> >> Debugging Flags
>> >>
>> >> Enable ASP Server-Side Script Debugging
>> >>
>> >> Select this check box to enable the Web server to enter the Microsoft
>> > Script
>> >> Debugger while processing ASP pages. You can then use the debugger to
>> >> examine your scripts.
>> >>
>> >> Enabling ASP server-side debugging causes ASP to run in
>> >> single-threaded
>> >> mode. For performance reasons, ASP debugging is not recommended in a
>> >> production environment.
>> >>
>> >> Brian
>> >>
>> >> "Tony Proctor" <tony_proctor@aimtechnology_NoMoreSPAM_.com> wrote in
>> > message
>> >> news:u%23aa$TyHFHA.1476@TK2MSFTNGP09.phx.gbl...
>> >> > Hi everyone
>> >> >
>> >> > We're experiencing some serious anomalies with the scheduling of ASP
>> >> > threads. I'd be interested to hear if anyone knows what algorithm is
>> > used
>> >> > (e.g. simple round-robin, or something more sophisticated), and what
>> >> > situations might perturb it. Even a hint as to what would be
> considered
>> >> > normal scheduling might help.
>> >> >
>> >> > The root of our problem is that we observed a normally well-behaved
> web
>> >> > application suddenly limit itself to a single ASP thread. The
>> > application
>> >> > consists mainly of a VB6 DLL instantiated via an ASP page. Note that
>> >> > the
>> >> > application does not store STA objects in Session/Application, and
>> >> > normally
>> >> > exhibits no thread affinities at all. The previous post asking for
>> > advice
>> >> > about this is at:
>> >> >
>> >
> http://www.google.ie/groups?safe=off&as_umsgid=uAkYyaaHFHA.588@TK2MSFTNGP15.phx.gbl&lr=lang_en&hl=en.
>> >> >
>> >> > As that previous post says, we eventually found that the switch of
> all
>> >> > incoming requests (irrespective of their Session ID) to a single ASP
>> >> > thread
>> >> > occurred when one thread just happened to destroy a Collection of
> MSXML
>> >> > DOM
>> >> > objects. Hah!, we thought, a bug in MSXML 3.0, probably connected
> with
>> > its
>> >> > new free-threaded capability (which we don't use). We then created a
>> > much
>> >> > cut-down version of our server application with a view to reporting
> it
>> > to
>> >> > Microsoft. Unfortunately, the cut-down version didn't show the same
>> >> > results.
>> >> > Interestingly, though, its ASP thread scheduling was markedly
> different
>> >> > from
>> >> > that in our real application.
>> >> >
>> >> > The problem has only been observed under IIS 6, and so it could well
> be
>> > an
>> >> > IIS/ASP difference rather than the fault of MSXML. Running the same
>> >> > simple,
>> >> > single-client (with an auto-refreshed HTML page at 1 sec intervals)
>> >> > test
>> >> > against our 2 servers (the real one and the cut-down one), we
>> >> > observe
>> > the
>> >> > following strange thread behaviour. Any explanations here would be
> very
>> >> > welcome:
>> >> >
>> >> > IIS 6 (configured with 20 ASP threads):
>> >> > Real server:
>> >> > All 20 threads used in perfect round-robin fashion, until
>> >> > one
>> >> > destroys
>> >> > a Collection of MSXML DOM objects. After that, all
> incoming
>> >> > requests
>> >> > are handed to this one thread (sometimes to a pair of
>> >> > threads
>> >> > if
>> >> > they
>> >> > happen to have cleared their Collections down at the same
>> > time)
>> >> > Cut-down server:
>> >> > One thread handles the initial request. All subsequent
>> > requests
>> >> > oscillate
>> >> > evenly between just 2 ASP threads
>> >> >
>> >> > IIS 5 (configured with 20 ASP threads):
>> >> > Real server:
>> >> > All 20 threads are used. However, the first 8 are
> scheduled
>> >> > sequentially.
>> >> > After that it gets more and more random which ASP thread
> is
>> >> > used. No
>> >> > affinity every arises though.
>> >> > Cut-down server:
>> >> > Only 8 ASP threads are ever used, and these are scheduled
> in
>> >> > perfect
>> >> > round-robin fashion
>> >> >
>> >> > I can't explain these observations as the 2 servers are handling the
>> > same
>> >> > requests. There seems to be more involved than simple round-robin
>> >> > scheduling. Hence, I though if I understood the algorithm better, I
>> > might
>> >> > have a clue as to the initial problem we encountered (which is now
>> >> > being
>> >> > considered serious)
>> >> >
>> >> > Tony Proctor
>> >> >
>> >> >
>> >>
>> >>
>> >
>> >
>>
>>
>
>
>

Quantcast