Re: Reprise: Pended IOCtrl + LoadLibrary == Deadlock?

>Ok, I rebuilt everything the way I was before so that I could reproduce
>the bug again. I then tried all possible compinations of the following
>three things:

>1) running my app with and without handle tracking
>2) running msdev.exe with and without handle tracking
>3) launching msdev.exe from the shell and from within windbg.

The interesting configuration here would be:
-1- devenv.exe and your application both with handle-tracing ON
-2- attach cdb/ntsd/windbg to devenv.exe after it started and before you run
your application
-3- attach a non invasive cdb/ntsd/windbg to the application spawned by
devenv.exe
after you've gotten axceptions in either devenv.exe or your application

>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>run windbg
>load (from within windbg) msdev.exe(av-)
>load (from within msdev) ionwizrd.exe(av-)
>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>no lockup.

>But, strangely I get a C0000008 in the process of loading
>msdev. Output of windbg is:
// snip

This looks to me the initial breakpoint when the process starts-up.
You should simply "g" past that, or start cdb/ntsd/windbg with the
"-g -G" command line options.

>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>run msdev.exe(av-)
>load (from within msdev) ionwizrd.exe(av+)
>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>msdev loads fine. When I hit F5 to run my app the following occurs:
>exception c0000008 (this I get via a dialog box from msdev)
>stack is (note this is before my apps CRT init):

>NTDLL! 77f75e2f()
>7ffe0304()
>MSVCRT! 77c2b4b1()
>NTDLL! 77f5b42c()
>NTDLL! 77f56771()
>NTDLL! 77f649a8()
>00020000()

If this is really a c0000008 exception, then this is a starting point.
Since devenv.exe is inadequate to perfrom system debugging,
when your target process is in this configuration, attach
a non invasive debugger, use the symsrv.dll capability of cdb/ntsd/windbg
to get a good stack, identify the handle being bad, and issue `!htrace
<handlevalue>`,
to see where it does come from.

The other configurations are variations of the problems you
have been seeing all the way throgh, with the complication added by
the devenv.exe debugger that it makes it hard to debug an OS-triggered
invalid-handle exception.

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm


"Tom Udale" <email@xxxxxxxxxxx> wrote in message
news:AJGdnWDHyoerlYbeRVn-hg@xxxxxxxxxxxxxx
> Dear Ivan,
>
>> > You mention below turning on handle tracing in devenv.exe. After
> poking
>> > around for a bit I have realized that this will not work for me as I
> am
>> > running DevStu 6, not .NET.
>> >
>> handle tracing is an OS concept, and it does not depend on the target
>> application
>> or on the debugger. Try this:
>> c:\>gflags /v IMAGE /enable devenv.exe HANDLE_CHECKS
>> c:\>gflags /v IMAGE /enable TestApp.exe HANDLE_CHECKS
>
> I see, I did not understand completely what you were asking before. I
> now get it (and see that using gflags is the same as using appverifier
> insofar as the -v option is concerned. It obviously has other nice
> features as well).
>
>
>
>> and then run devenv.exe under cdb/ntsd/windbg, and, when you have
>> the original probelm in the debugger, attach a non invasive debugger
>> to the target application spawned by devenv.exe (cdb -pv -p <PID>)
>> and execute !htrace in both devenv.exe and the target app.
>
> Ok, I did this and have included more discussion and my results at the
> bottom of this post.
>
>
>
>> > When I run the app (F5) I get fault traps that I had never seen
> before
>> > (presumably generated by AV).
>>
>> You should prompthly investigate those new AVs, because they may show
>> that the root cause of the problem is yet to be found
>
> You are a taskmaster! :) I have included this case in the results
> below.
>
>
>> > If I run the app either directly from the shell or via windbug
>> > _on_the_target_ (not from my km debugging host I have been
> mentioning
>> > before) the app does not fail and naturally AV and !htrace confirm
> that
>> > nothing odd is happening. The handle that the driver ends up with
> is
>> > used several times for other purposes before the driver gets the
> value,
>> > but they are all perfectly symetrical open/close pairs and no events
> on
>> > that handle value occur after the driver is opened.
>> >
>> You should pursue the root cause from the scenario when the app
>> is launched from devenv.exe. It's most likely an interaction problem
> with
>> devenv.exe
>
>
> It is looking that way, which is some ways is a relief as my customers
> will not be doing this.
>
>
>
>
> *******************************************************
> More discussion
> *******************************************************
> Ok, I rebuilt everything the way I was before so that I could reproduce
> the bug again. I then tried all possible compinations of the following
> three things:
>
> 1) running my app with and without handle tracking
> 2) running msdev.exe with and without handle tracking
> 3) launching msdev.exe from the shell and from within windbg.
>
> The bottom line in that the only way I can reproduce the crash is when
> all the handle tracking is off and I am running msdev from the shell -
> i.e. the way I was doing things when I first saw this bug 3 weeks ago.
>
> All other cases either result in not being able to run msdev or my app
> at all, or with no troubles.
>
> Unfortunately I cannot tell if this is a bug in my app revealed by
> msdev, or a bug in msdev revealed by my app. I am actually leaning
> towards the latter as I am seeing a number of handle errors in msdev
> itself both when it starts up and when it is trying to load my app. I
> am also seeing errors in my apps CRT init (which is completely out of my
> hands).
>
> I am still going to try Alexanders suggestion of trying to read the
> startupinfo in my app when I first get control and see if that is in any
> way telling. That may give me enough information to sort the mystery
> once and for all.
>
> One happy upshot of everything is that I have discovered that I can run
> my app in its original build configuration without seeing the bug if I
> load msdev from within windbg. I can even have handle tracking enabled
> for both my app and msdev. So while I cannot debug the problem at the
> very moment, if it is infact an issue with my app (rather than with
> msdev), it may well show up again later in a different manifestation.
> If it does, I will at least be in a position to bring more tools to bear
> upon it.
>
> In the meantime, I can move forward on this project. I have been
> working on this bug for nearly 3 weeks now and am sorely behind
> schedule. I simply cannot spend any more time on it in its current
> state (well after trying Alexander's suggestion of course).
>
>
> Best regards,
>
> Tom
>
>
>
>
> *******************************************************
> Results of my testing
> *******************************************************
>
> av+ means appverifier/gflags handle tracking only enabled.
> av- means appverifier/gflags disabled
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> run msdev.exe(av-)
> load (from within msdev) ionwizrd.exe(av-)
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> lockup occurs.
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> run windbg
> load (from within windbg) msdev.exe(av-)
> load (from within msdev) ionwizrd.exe(av-)
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> no lockup.
>
> But, strangely I get a C0000008 in the process of loading
> msdev. Output of windbg is:
> Microsoft (R) Windows Debugger Version 6.5.0003.7
> Copyright (c) Microsoft Corporation. All rights reserved.
>
> CommandLine: I:\devstu\MSDev98\Bin\MSDEV.EXE
> Symbol search path is: <..snip..>
> Executable search path is: <..snip..>
> ModLoad: 00400000 00443000 msdevP.exe
> ModLoad: 77f50000 77ff7000 ntdll.dll
> ModLoad: 77e60000 77f46000 D:\WINDOWS\system32\kernel32.dll
> ModLoad: 73dd0000 73ec2000 D:\WINDOWS\System32\MFC42.DLL
> ModLoad: 77c10000 77c63000 D:\WINDOWS\system32\MSVCRT.dll
> ModLoad: 7e090000 7e0d1000 D:\WINDOWS\system32\GDI32.dll
> ModLoad: 77d40000 77dcc000 D:\WINDOWS\system32\USER32.dll
> ModLoad: 77dd0000 77e5d000 D:\WINDOWS\system32\ADVAPI32.dll
> ModLoad: 78000000 78087000 D:\WINDOWS\system32\RPCRT4.dll
> ModLoad: 50000000 50146000 I:\devstu\MSDev98\Bin\devshl.dll
> ModLoad: 771b0000 772d4000 D:\WINDOWS\system32\ole32.dll
> ModLoad: 77120000 771ab000 D:\WINDOWS\system32\OLEAUT32.dll
> ModLoad: 773d0000 77bca000 D:\WINDOWS\system32\SHELL32.dll
> ModLoad: 70a70000 70ad4000 D:\WINDOWS\system32\SHLWAPI.dll
> ModLoad: 77340000 773cb000 D:\WINDOWS\system32\COMCTL32.dll
> (6a4.6a8): Break instruction exception - code 80000003 (first chance)
> eax=00251ea4 ebx=7ffdf000 ecx=00000003 edx=77f51308 esi=00251ea4
> edi=00251f18
> eip=77f75a58 esp=0013fb38 ebp=0013fc2c iopl=0 nv up ei pl nz na
> pe nc
> cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000
> efl=00000202
> ntdll!DbgBreakPoint:
> 77f75a58 cc int 3
>
> The stack is:
>
> Args to Child
> 7ffde000 7ffdf000 00000000 ntdll!DbgBreakPoint (FPO: [0,0,0])
> 0013fd30 77e60000 0013fce8 ntdll!LdrpInitializeProcess+0xdf3 (FPO:
> [5,82,3])
> 0013fd30 77f50000 00000000 ntdll!LdrpInitialize+0x186 (FPO: [Non-Fpo])
> 00000000 00000000 00000000 ntdll!KiUserApcDispatcher+0x7
>
> Other than this, there are no warnings or errors.
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> run msdev.exe(av-)
> load (from within msdev) ionwizrd.exe(av+)
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> msdev loads fine. When I hit F5 to run my app the following occurs:
>
> exception c0000008 (this I get via a dialog box from msdev)
>
> stack is (note this is before my apps CRT init):
>
> NTDLL! 77f75e2f()
> 7ffe0304()
> MSVCRT! 77c2b4b1()
> NTDLL! 77f5b42c()
> NTDLL! 77f56771()
> NTDLL! 77f649a8()
> 00020000()
>
> Hit F5, say do not pass on to app being debugged.
>
> exception c0000008 (now we are at my apps CRT init. This looks very
> similar to the
> stack when I lock up loading the monoatd4 dll later)
>
> NTDLL! 77f75e2f()
> 7ffe0304()
> _ioinit() line 250 + 60 bytes
> WinMainCRTStartup() line 169
> KERNEL32! 77e814c7()
>
>
> Hit F5, say do not pass on to app being debugged.
>
> exception c0000008 (now we loading one of my plugins. Again in CRT init
> and again this looks very similar to the
> stack when I lock up loading the monoatd4 dll later)
>
> NTDLL! 77f75e2f()
> 7ffe0304()
> _ioinit() line 250 + 60 bytes
> _CRT_INIT(void * 0x018e0000, unsigned long 0x00000001, void *
> 0x00000000) line 127
> _DllMainCRTStartup(void * 0x018e0000, unsigned long 0x00000001, void *
> 0x00000000) line 231 + 17 bytes
> NTDLL! 77f5b42c()
> NTDLL! 77f56771()
> NTDLL! 77f6151e()
> NTDLL! 77f570e0()
> KERNEL32! 77e7d901()
> KERNEL32! 77e7d95e()
> SystemDLL::SystemDLL(const STRINGc & {...}, int 0x00000001) line 96 + 18
> bytes
> DynamicLibInst::Create(const STRINGc & {...}, int 0x00000001) line 222 +
> 35 bytes
>
> This continues for every DLL that is loaded. I gave up after about 10.
>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> run windbg
> load (from within windbg) msdev.exe(av-);
> load (from within msdev) ionwizrd.exe(av+)
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> Same break on the load of msdev as before. No other troubles.
>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> load msdev.exe(av+)
> load (from within msdev) ionwizrd.exe(av+)
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> The app terimates with a message saying "The application failed to
> initialize properly (0xc0000008).
> Click on OK to terminate the application." If I click "OK" I get the
> same message a
> second time and then indeed the app terminates.
>
> I presume this is the same c0000008 that I am seeing when I run
> msdev.exe from within windbg.
>
>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> run windbg
> load (from within windbg) msdev.exe(av+)
> load (from within msdev) ionwizrd.exe(av+)
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> Same break on the load of msdev as before. No other troubles.
>
>
>
>
>
>
>
>
>


.

Relevant Pages

  • Re: Reprise: Pended IOCtrl + LoadLibrary == Deadlock?
    ... > and execute!htrace in both devenv.exe and the target app. ... launching msdev.exe from the shell and from within windbg. ... All other cases either result in not being able to run msdev or my app ... itself both when it starts up and when it is trying to load my app. ...
    (microsoft.public.development.device.drivers)
  • Re: Reprise: Pended IOCtrl + LoadLibrary == Deadlock?
    ... I cannot run msdev with tracing on. ... > load ionwizrd.exe ... > second time and then indeed the app terminates. ... > when your target process is in this configuration, ...
    (microsoft.public.development.device.drivers)
  • Re: Reprise: Pended IOCtrl + LoadLibrary == Deadlock?
    ... Yes it looks like your deadlock is also caused by this shell issue ... > ran my app from msdev. ... > to load with C0000008 error. ...
    (microsoft.public.development.device.drivers)
  • Re: Reprise: Pended IOCtrl + LoadLibrary == Deadlock?
    ... The original trouble was a deadlock in my app that turned out to be ... Ivan suggested that I gflag both my app and msdev to HANDLE_CHECKS. ... It was observed that msdev would not load at all in this state - fails ... used windbg to load msdev in an attempt to "debug the debugger" the ...
    (microsoft.public.development.device.drivers)
  • Re: Mobile Development for WM 5 with one main exe controlling mult
    ... I'm trying to convince them with what you said about the load strategy, ... it could be due to the fact that all the loading are done ... App size has nothing ... split the whole program from one exe to multiple. ...
    (microsoft.public.pocketpc.developer)
Loading