Re: Understand Working Set : Memory Usage
From: hector (nospam_at_nospam.com)
Date: 01/25/05
- Next message: Gabriel Bogdan: "Re: Mail application compiled in Multithread while dll in Singlethread"
- Previous message: greg: "Mail application compiled in Multithread while dll in Singlethread"
- In reply to: Pavel Lebedinsky: "Re: Understand Working Set : Memory Usage"
- Messages sorted by: [ date ] [ thread ]
Date: Tue, 25 Jan 2005 10:42:26 -0500
Thanks Pavel,
This is turning out to be a more complex issue than expected.
Still looking into it but I now need to back track to find out more about
"RPC worker threads."
Here is the skinny for your interest:
We have a RPC client/server framework where RPC clients serve as hosting
servers to the outside world and on the backend also talk to our RPC server
for session management.
sessions --> host/RPC client ---> RPC protocol --> RPC Server
Two RPC protocols are supported: Local and TCP/IP RPC protocols.
If the client/server are on the same machine, a local RPC bind is used.
If the client and server are on remote machines (LAN, WAN), then a TCP/IP
RPC bind is used.
What I found was that RPC client connections via TCP/IP RPC protocol are not
releasing the binding socket connections from what seems to be the RPC
worker thread pool.
So if a session is started and there isn't a RPC worker thread already
started, it seems RPC creates a new socket connection for the bind.
In a high volume environment, where concurrent connections come in at the
same time, then what I am seeing is the a build up of these unreleased
socket connections.
Such is not the case in a LOCAL RPC bind.
So I need to now to study if this is a design bug in our 10 year old
multi-threaded RPC client/server system or something related to the new
windows OS starting with Windows 2000. I'm surprise this never came up
before so I'm going to be busy trying to figure this one out :-)
If any RPC expert, MVP or even Microsoft network engineer are lurking
around, I would be great to get some confirmation here or input regarding
this RPC TCP/IP protocol behavior.
Thanks
--- Hector Santos WINSERVER "Wildcat! Interactive Net Server" http://www.santronics.com "Pavel Lebedinsky" <m_pll at hotmail com> wrote in message news:uzB#4LgAFHA.3368@TK2MSFTNGP10.phx.gbl... > "hector" wrote: > > > I am pretty confident I don't have a leak. The memory Working Set is > > what I am seeing increasing over the time and for some customers, > > they claim they need to restart the system at some future time. > > You should get more information from these customers. Just the fact > that the working set is large (how large?) is not enough to make any > conclusions. > > Have them collect perfmon counters such as > > Process\Private Bytes > Process\Virtual Bytes > Process\Working Set > Process\Page Faults/sec > Memory\Pages/sec > Memory\Available MBytes > > > So I am trying to understand why this is happening. > > What is happening? The working set increase? It might be perfectly > normal - if you have lots of available physical memory, why not use it? > > This is why I asked for more info - a growing working set by itself is > not always a problem (especially if we're talking about an increase > of just a few megabytes, like you mentioned in another message). > It's things like excessive paging that you should worry about. > > > Based on my reading, memory fragmentation seems to be a main > > or key reason for this behavior. > > Poor locality (which is sometimes caused by fragmentation at > various levels in the memory management stack) can be one of the > reasons for a large working set. But most commonly it's caused > by leaks or just allocating too much stuff at once. > > You can create a memory dump of the process and then > analyze it in WinDbg using the !address and !heap commands. > This should allow you to rule out or confirm theories like > heap/VM fragmentation. > > > I'm exploring SetProcessWorkingSetSize(h,-1,-1) to force the OS release of > > this working set overhead. I need to see if this is a clean, feasible, > > correct approach for production ware in lieu of a total revamp and/or > > replacement in memory management. > > No it's not a correct approach. The OS will trim your working set > automatically when available physical memory gets low. By doing > it yourself you are not actually freeing or releasing any memory, > you just end up causing more page faults than necessary. The pages > that you touch after calling SetProcessWorkingSetSize(-1, -1) > have to be page-faulted back into your working set. If the system is > not under memory pressure you are just doing unnecessary work. > And even if available memory is low, the OS is likely to do > a better job at trimming your working set because it has access > to information such as whether a given page has been recently > used or not etc. > >
- Next message: Gabriel Bogdan: "Re: Mail application compiled in Multithread while dll in Singlethread"
- Previous message: greg: "Mail application compiled in Multithread while dll in Singlethread"
- In reply to: Pavel Lebedinsky: "Re: Understand Working Set : Memory Usage"
- Messages sorted by: [ date ] [ thread ]
Relevant Pages
|
