Re: Garbage Collection Issues in long-standing services

Thanks for the reply.

I would agree that I must be holding on to some references, I will try to
isolate. If the GC was really being aggressive, then why after everything
came back to steady state did the memory in use reduce by half? (800MB -
350MB)

I did run some tests on the classes that would give me the most problems and
it appeared that the GC was smart enough to solve the issue. Here is a brief
synopsis:

TCPServer - Has a hashtable holding a class that encapsulates the client
socket (CS), including a refrence to a class that handles the application
processing (AP). The first thing AP does when it handles an OnConnect is
store a reference to CS so that it can use it to send data back to the client.

State Object (SO) - This holds a reference to CS and a pointer to a 4K byte
array. This is passed on the Async Send / Receive Calls.

Connection Termination - After the connection is closed (abnormally or
normally), the CS is removed from the hashtable. I've done test to see that
once this is performed that the GC is smart enough to deal with the circular
reference between CS and AP, so I don't explicitly set each reference to null.

Outstanding Async Send / Receive - Depending how the connection is
terminated, there is a chance that there can be an Outstanding Async Send and
Receive. I store a reference to the most current IAsyncResult for both the
Send and Receive in the CS object. When closing, I check to see if these are
still outstanding and if so, I retrieve the SO object and set the CS
reference to null. I'm not setting the buffer to null and I probably should.
I don't expect that this condition is normal, but I do my best at handling
it. You can't just stop an asyncronous request which is unfortunate because
I'm sure there is a structure being stored in the OS to encapsulate each
(e.g. same structure that holds the state object).

I will continue to try and isolate. Any ideas based on the above?

"Willy Denoyette [MVP]" wrote:

> Did you ever checked the GC performance counters (using perfmon) to check
> whether this is true? The GC is more aggressive than you imagine.
> Just check the Gen0 1 and 2 performance counters and you will see that the
> collector runs, your problem is that you are holding references to objects
> (probably large objects) which you never release, so there is little or
> nothing to collect.
> By starting another process that allocates memory, your service working set
> gets trimmed by the OS, that all that happens.
>
> Willy.
>
> "Larry Herbinaux" <LarryHerbinaux@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in
> message news:72FFED84-F740-456E-A986-0BB25C18ED39@xxxxxxxxxxxxxxxx
> > I'm having issues with garbage collection with my long-standing service
> > process. If you could review and point me in the right direction it would
> > be
> > of great help. If there are any helpful documents that you could point me
> > to
> > help me control the GC, then that would be great also.
> >
> > The .Net GC does not cleanup memory of our service process unless it is
> > forced to by another process that hogs memory.
> > · GC Algorithm - This is an issue because if the GC is not forced into
> > doing
> > this, it does not aggressively cleanup until the amount of physical memory
> > available is very small. I understand why it doesn’t want to force
> > cleanup
> > due to processor efficiency, but it forces applications into conditions
> > that
> > are not acceptable. It would be nice to be able to hint an upper limit
> > for
> > an application that helps the GC be more aggressive when required.
> > · Race Condition – The GC Algorithm causes race conditions because the
> > GC is
> > not coordinated with our application and our application throws
> > OutOfMemoryExceptions. We have very good exception handling that guards
> > against unhandled exceptions in the main thread and thread pool threads.
> > The
> > problem is that we use memory to log these issues so the handlers are
> > probably throwing another OutOfMemoryException. We can handle this, but
> > the
> > point is the OutOfMemoryException cause a transaction to fail.
> > · Force GC To Collect - I wrote a Memory Hogger application which when
> > run
> > will reduce the amount of memory used by the service application from
> > 800MB
> > to 3MB, so this proves that the GC will cleanup the memory when it truly
> > needs it. I noticed that Memory Usage displayed in the Task Manager
> > Process
> > tab did not add up to the total amount of memory in use, so this means
> > that
> > the inactive applications probably moved their heap to swap. One concern
> > here was that when the Memory Hogger application was terminated, our
> > service
> > application reclaimed half of its memory and we were not processing
> > transactions. Maybe GC just moved it to swap.
> > · GC.Collect – Using this is not recommended within the application.
> > Even
> > when this is used, it doesn’t make the GC any more aggressive, so I
> > agree,
> > there is no reason to use it. It would be nice to have the ability to
> > make
> > the GC more aggressive.
> > · CLR Profiler - I’ve used the CLR Profiler to determine what memory is
> > not
> > being collected. Mostly string and byte arrays. Our service handles TCP
> > Connections asynchronously and we reuse the same byte array when receiving
> > data for the next asynchronous read. We store a reference to the current
> > IAsyncResult for both the asynchronous send and receive requests. The
> > state
> > object holds the byte array which we are currently not setting to null.
> > There isn’t a way of canceling the async requests, so we might have to
> > explicitly set this to null. I will try this to see if makes a
> > difference.
> > As for the strings, I’m not sure where the problem is here.
> >
>
>
>
.

Relevant Pages

  • [RFC] page replacement requirements
    ... Submitting too much I/O at once can kill latency and even lead to deadlocks when bounce buffers are involved. ... Must be able to deal with multiple memory zones efficiently. ... When on completion of the write to their backing-store the reference bit is still unset a callback is invoked to place them so that they are immediate candidates for reclaim again. ... For traditional page replacement algorithms this is not a big issue since we just implement per zone page replacement; ...
    (Linux-Kernel)
  • Re: Java or C++?
    ... >> CTips wrote: ... GC can only kick in when the last reference has ... If f was already bound to a lot of memory when it enters do-it then ... > more than the extra memory leaked because of sloppy manual deallocation]. ...
    (comp.programming)
  • Re: 4-way Opteron vs. Xeon-IBM X3 architecture
    ... >>>The point here is that the issue concerns both speed AND capacity. ... >As to the reference, the message header points right back in this thread, ... >>>As you may suspect, I read plenty about memory systems, and I would ... >>>from the enthusiast market and assumed that it would work in the server ...
    (comp.sys.ibm.pc.hardware.chips)
  • Re: A re-announce on GCs defects
    ... It's bad for CPU/Resource intensive but memory cheap objects. ... There may be more than one strong references and you don't know when and where to call Dispose. ... Instead of calling the destructor you call Dispose if the reference counter is 0. ... Note GC in java and C# is not really an addictive as someone would argue since there is no way to do real memory management like delete obj in C++. ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Personal Folder not closing properly
    ... What are the steps to determine if a 3rd party program is holding a reference to outlook in memory? ...
    (microsoft.public.outlook)