Re: increasing handles | large VM consumption | windows service on Win2k server

Tech Tip: Click here to run a free scan for Windows Errors and optimize PC performance

From: Levi Wilson (lewilson_at_purdue.edu)
Date: 04/14/04 

Date: Wed, 14 Apr 2004 08:38:14 -0500

In my call to CoInitializeSecurity() I use EOAC_NONE,
RPC_C_AUTHN_LEVEL_NONE, and RPC_C_IMP_LEVEL_IMPERSONATE. This is just
enough for my process to connect (IWbemLocator::ConnectServer()) to my
machines. Once a connection is established, I make a call to
CoQueryProxyBlanket and query the pAuthnSvc and the pAuthnLevel values to
see what kind of security settings my IWbemServices proxy needs to make WMI
calls to that particular machine, and then I call CoSetProxyBlanket. I've
ran Compuware Bounds Checker on my service and did have pseudo leaks in the
wbemprox and other dlls that you mentioned, but other than that it didn't
list anything other than that if I recall. I have since re-ghosted the
machine, so I'll need to run that again to see if anything else is coming
up, but does that explain why things run fine as a console application? I
ran the consol app overnight just as I did the service the previous night,
when I came in this morning the handles remained stable at 209 (which is ~
what it starts out at) wheras yesterday morning the service had a handles
value of ~10000. The VM size of the console app is 201,436 K, which might
be a little high, but still runs 100x more stable than the service... i'll
look into the umdh.exe and see if that comes up with anything, but in the
meantime, any other suggestions? I really appreciate all your help, Ivan.
This problem's been completely baffling me, so thanks for helping me address
it.

"Ivan Brugiolo [MSFT]" <ivanbrug@online.microsoft.com> wrote in message
news:eQ1NotaIEHA.2376@TK2MSFTNGP12.phx.gbl...
> The problem is not easy in general.
> For example, if you are using EOAC_SECURE_REFS on your service,
> and WinMmgt is indicating instances to your process, and there is
different
> identity
> between the time the interface was marshaled and when it's used,
> then your interface will be released by RPCSS way after you are done with
> your async query.
> There are a handful of other cases that would be quite long to enumerate.
> They involve different apartments, different identities,
> cross process and cross machine boundaries.
> Just to make sure this is the problem, if you let the service sit around
> for few minutes after is has finished it's own activity, does the service
> recover it's memory usage ?
> There are generic leak tracking tools, like UMDH.exe.
> You will be finding (pseudo) leaks in, maybe, rpcrt4, wbemprox, fastprox,
> wbemcomn,
> but, you may be able to guess which interface pointers are really leaked.
>
> --
> This posting is provided "AS IS" with no warranties, and confers no
rights.
> Use of any included script samples are subject to the terms specified at
> http://www.microsoft.com/info/cpyright.htm
>
>
> "Levi Wilson" <lewilson@purdue.edu> wrote in message
> news:OUOCrsZIEHA.3968@TK2MSFTNGP12.phx.gbl...
> > Ivan, I really appreciate all of your help, and i'm sorry if i'm being a
> > pain...
> >
> > What do you mean un-authenticated calls that perform cleanup? I must be
> > missing something here, but what you said sounds like what is happening,
> > obviously some resources aren't being cleaned up, it's just that I was
> under
> > the impression that releasing my interface pointers was enough for the
> > cleanup of these resources. I mean, it's the exact same thread that's
> doing
> > the same work running under a windows service as it is when it runs as a
> > consol application, so i'm not making the connection here. Do you have
> any
> > suggestions or techniques for me to track down this issue, or to track
> down
> > what I need to do differently to ensure that these resources are being
> > released? Please let me know any information would be useful for you to
> > help with this, and i'm sorry if I don't give enough information.
Thanks,
> > again, in advance for any help!
> >
> >
> > "Ivan Brugiolo [MSFT]" <ivanbrug@online.microsoft.com> wrote in message
> > news:%23RJ4InXIEHA.3432@tk2msftngp13.phx.gbl...
> > > One example:
> > > you process may not allow un-authenticated calls to come through.
> > > Certain un-authenticated calls may be used to perform cleanup.
> > > If this does not happen the regular way, then your code may have to
wait
> > > until
> > > RPCSS runs the distributed garbage collection.
> > > This way the resources are "leaked" till they are reclaimed.
> > >
> > > --
> > > This posting is provided "AS IS" with no warranties, and confers no
> > rights.
> > > Use of any included script samples are subject to the terms specified
at
> > > http://www.microsoft.com/info/cpyright.htm
> > >
> > >
> > > "Levi Wilson" <lewilson@purdue.edu> wrote in message
> > > news:u9Oe$hVIEHA.3848@tk2msftngp13.phx.gbl...
> > > > sorry, I do keep adding pieces, and here's anothere one :-D It runs
> > under
> > > a
> > > > domain administrative account, not as local system when it's running
> as
> > a
> > > > service. When you say COM initialization, are you talking about
> > > > CoInitializeSecurity() and CoInitializeEx()? My DCOM permissions
are
> > set
> > > up
> > > > dynamically depending on the OS of the machine that i'm connecting
to
> > with
> > > > calls to CoQueryProxyBlanket() and CoSetProxyBlanket(), so I don't
> > > > understand the differences when it's running under svchost.exe as
> > opposed
> > > to
> > > > explorer.exe...any ideas on where I could start to look/what to look
> > for?
> > > >
> > > > "Ivan Brugiolo [MSFT]" <ivanbrug@online.microsoft.com> wrote in
> message
> > > > news:eAnXIlNIEHA.3240@TK2MSFTNGP12.phx.gbl...
> > > > > You keep adding pieces.
> > > > > If your application does not leak interface pointers if compiled
as
> > > > console
> > > > > and run from your account,
> > > > > but maybe it does when it's a serivce (run by localsystem ?)
> > > > > then you should go after the differences in COM initialization,
> > account
> > > > and
> > > > > DCOM permission.
> > > > >
> > > > > --
> > > > > This posting is provided "AS IS" with no warranties, and confers
no
> > > > rights.
> > > > > Use of any included script samples are subject to the terms
> specified
> > at
> > > > > http://www.microsoft.com/info/cpyright.htm
> > > > >
> > > > >
> > > > > "Levi Wilson" <lewilson@purdue.edu> wrote in message
> > > > > news:#o6nTWMIEHA.828@TK2MSFTNGP12.phx.gbl...
> > > > > > Sorry, I wasn't descriptive enough...the #define CONSOLE_APP is
> just
> > a
> > > > > > custom define statement in my code so that when it is defined,
it
> > > > compiles
> > > > > > my program as a console app (how it originated before I changed
it
> > to
> > > be
> > > > a
> > > > > > windows service), but if it's not defined it compiles it as a
> > windows
> > > > > > service.
> > > > > >
> > > > > > Example:
> > > > > >
> > > > > > void main()
> > > > > > {
> > > > > > #ifdef CONSOLE_SERVICE
> > > > > > ServiceInitialization(0,NULL,NULL);
> > > > > >
> > > > > > cout << "Type '@' to quit: " << endl;
> > > > > > char ch;
> > > > > > cin >> ch;
> > > > > > while(ch!='@')
> > > > > > {
> > > > > > cin >> ch;
> > > > > > }
> > > > > >
> > > > > > SetEvent(g_hQuit);
> > > > > >
> > > > > > WaitForSingleObject(g_hThread,INFINITE);
> > > > > > #else
> > > > > > SERVICE_TABLE_ENTRY DispatchTable[] = {
> > > > > > {"MyService",
> > > MyServiceStart},
> > > > > > {NULL, NULL}
> > > > > > };
> > > > > > if(!StartServiceCtrlDispatcher(DispatchTable))
> > > > > > {
> > > > > > SvcDebugOut(" [MY_SERVICE] StartSvcCtrlDispatcher error =
> %d\n",
> > > > > > GetLastError());
> > > > > > }
> > > > > > #endif
> > > > > > }
> > > > > >
> > > > > > All my ServiceInitialization() function does is creates the
thread
> > and
> > > > the
> > > > > > events that my program waits for...
> > > > > >
> > > > > > See what I mean? I still use the exact same initialization, it
> just
> > > > > changes
> > > > > > how it starts up...well, and also that it runs under
Explorer.exe
> > > rather
> > > > > > than svchost.exe or whatever. Any ideas? Thanks again for your
> > quick
> > > > > > response...fastest I've ever experienced in a news group, nice
> work!
> > > > > >
> > > > > > "Ivan Brugiolo [MSFT]" <ivanbrug@online.microsoft.com> wrote in
> > > message
> > > > > > news:u7rfgNMIEHA.3276@tk2msftngp13.phx.gbl...
> > > > > > > I'm not familair with the #define CONSOLE_APP pre-processor
> > setting,
> > > > > > > (which development environment does it apply to ?
> > > > > > > what does it change in linker settings ?
> > > > > > > what does it change in OLE32 initialization settings ?).
> > > > > > > It looks like a cros-apartment problem caused by different COM
> > > > > > > initialization settings.
> > > > > > > This may cause un-necessary cross-apartment marshaling,
> > > > > > > and extra "mediators" interface pointers to be kept around.
> > > > > > >
> > > > > > > --
> > > > > > > This posting is provided "AS IS" with no warranties, and
confers
> > no
> > > > > > rights.
> > > > > > > Use of any included script samples are subject to the terms
> > > specified
> > > > at
> > > > > > > http://www.microsoft.com/info/cpyright.htm
> > > > > > >
> > > > > > >
> > > > > > > "Levi Wilson" <lewilson@purdue.edu> wrote in message
> > > > > > > news:uSlnsuLIEHA.2836@TK2MSFTNGP11.phx.gbl...
> > > > > > > > Ivan, thank you very much for your quick response.
> > > > > > > >
> > > > > > > > my interface pointers to these classes (IWbemServices,
> > > > > > > IEnumWbemClassObject,
> > > > > > > > etc.) reside in classes for the different types of
information
> > > that
> > > > I
> > > > > > want
> > > > > > > > to gather, all with a generic base class that has the
methods
> to
> > > > > > enumerate
> > > > > > > > and gather the information that I want. The interface
> pointers
> > > are
> > > > > > > released
> > > > > > > > accordingly in these classes, and i've checked all of this
> with
> > > > debug
> > > > > > > > strings etc. Another funny thing that I REALLY don't
> understand
> > > is
> > > > > that
> > > > > > > > when I #define CONSOLE_APP in my application to make it a
> > console
> > > > app
> > > > > > > > (instead of starting the dispatch table, etc. it just spawns
> my
> > > > worker
> > > > > > > > thread) everything seems to work fine. I don't see a large
> > > increase
> > > > > in
> > > > > > > > either. Anyone have any clues on as to why this is
happening?
> > > > > Thanks,
> > > > > > > > again, for any information that you can provide.
> > > > > > > >
> > > > > > > >
> > > > > > > > "Ivan Brugiolo [MSFT]" <ivanbrug@online.microsoft.com> wrote
> in
> > > > > message
> > > > > > > > news:eSiIfULIEHA.2164@TK2MSFTNGP12.phx.gbl...
> > > > > > > > > Summary counters like "Virtual Memory" and "Handles" are
way
> > too
> > > > > borad
> > > > > > > to
> > > > > > > > > give anything actionable. RPC connection across machines
are
> > > > "seen"
> > > > > by
> > > > > > > the
> > > > > > > > > programmers
> > > > > > > > > are interface pointers, like IWbemServices,
> > > IEnumWbemClassObject,
> > > > > > > > > that are given back to the programmer.
> > > > > > > > > The responsability to close them is normally up to the
> > > programmer,
> > > > > > > > > by releasing the interface pointers when expected.
> > > > > > > > > On average RPCSS is able to engage a garbage collection of
> > IPIDs
> > > > > > > > > on your process (for example, when the remote client dies
> and
> > > you
> > > > > are
> > > > > > in
> > > > > > > a
> > > > > > > > > server,
> > > > > > > > > RPCSS will reclaim the stub resources minutes after the
> client
> > > > death
> > > > > > was
> > > > > > > > > detected).
> > > > > > > > > If you are using the scripting interfaces, this is further
> > > > > complicated
> > > > > > > by
> > > > > > > > > the wbemdisp layer
> > > > > > > > > and the scripting engine.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > This posting is provided "AS IS" with no warranties, and
> > confers
> > > > no
> > > > > > > > rights.
> > > > > > > > > Use of any included script samples are subject to the
terms
> > > > > specified
> > > > > > at
> > > > > > > > > http://www.microsoft.com/info/cpyright.htm
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > "Levi Wilson" <lewilson@direcway.com> wrote in message
> > > > > > > > > news:b8119366.0404120915.6281db83@posting.google.com...
> > > > > > > > > > i have a windows service that runs on a windows 2000
> server
> > > > (sp3).
> > > > > > It
> > > > > > > > > > basically has a list of computers that I want to obtain
> some
> > > > > > > > > > information from periodically. when running, I observe
the
> > > > process
> > > > > > > > > > with SysInternals process explorer and see that the
> handles
> > > > count
> > > > > > as
> > > > > > > > > > well as the Virtual memory size seems to increase every
> time
> > i
> > > > > check
> > > > > > > > > > the information on a PC. This seems to only happen on
> > Windows
> > > > > 2000
> > > > > > > > > > server. Has anyone experienced anything similar to
this?
> I
> > > > > forgot
> > > > > > to
> > > > > > > > > > mention, that when I look in process explorer, 90% of
the
> > > > handles
> > > > > > that
> > > > > > > > > > are open seem to either be file handles with the
following
> > > > names:
> > > > > > > > > >
> > > > > > > > > > \device\tcp
> > > > > > > > > > \device\afd\endpoint
> > > > > > > > > >
> > > > > > > > > > Is WMI not closing the RPC handles it opens when it
> gathers
> > > > remote
> > > > > > > > > > information from the computers? What is happening? Any
> > help
> > > is
> > > > > > > > > > greatly appreciated, and please let me know if more
> > > information
> > > > > > needs
> > > > > > > > > > to be provided.
> > > > > > > > > >
> > > > > > > > > > Levi
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

Relevant Pages