Re: Memory Leak Experts!!!!

From: Alvin Bruney [MVP] (vapor)
Date: 06/06/04 

Date: Sun, 6 Jun 2004 11:54:13 -0500

>However, when you will test all
>this on simple examples you won't see any benefit. GC is able to process
>efficiently maybe 20K references per cycle - maybe less, maybe more
what is your source for this? Microsoft's benchmarks (that I have looked at)
show only processing as it relates to generations.

> Trimming working set then causes simple swapping of full memory to disk.
> However as soon as app is restored all this memory is swapped back. With
> related disk trashing and slow-down.
I used to believe this. In fact, i used to argue it as well (search the
*csharp newsgroups circa 2003). I've since changed my mind. I remain
unconvinced that trimming the working set has any effect.

>char c = <somechar>;
> return c;
>
> is more difficult for GC than simple
>
> return <somechar>;
Garbage collection does not concern itself with this case. These are thread
stack allocations. Deallocations occur automatically at the end of scope.
The scope is controlled by the calling routine which receives the return
call. 

-- 
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27***
"AlexS" <salexru2000NO@SPAMsympaticoPLEASE.ca> wrote in message 
news:ORg7Wb8SEHA.3548@TK2MSFTNGP09.phx.gbl...
>I would say it is a good approach.
> DataSet Dispose for example doesn't touch tables and rows. When you see 
> this
> code you might assume that GC will know that it should go through fields 
> and
> free references in due time. However, what works in normal circumstances
> might not work under stress.
>
> There are several simple recommendations, like if object implements
> IDisposable, Dispose should be called in code (explicitly) or object
> reference should be wrapped into using block. If references are passed
> between disconnected code, like in asynchronous events or threads, they
> should be freed by using null assignment. Don't keep references longer 
> than
> they are needed. And some more variations. However, when you will test all
> this on simple examples you won't see any benefit. GC is able to process
> efficiently maybe 20K references per cycle - maybe less, maybe more. I 
> don't
> know any official benchmarks for this. As soon as some of these hangers 
> will
> survive GC(0) and even worse GC(1) they tend to stay in memory. GC just
> doesn't know if they are free or not yet. Then relocations do make things
> worse.
>
> Now, in my experience, as soon you get in situation when application has
> bands of free memory interspersed with allocations GC becomes less 
> effective
> and needs some help. Note, that I am not talking about freeing memory. You
> can free references, which helps GC to free memory.
>
> Trimming working set then causes simple swapping of full memory to disk.
> However as soon as app is restored all this memory is swapped back. With
> related disk trashing and slow-down.
>
> I don't know exact figures when GC becomes ineffective, they are surely 
> very
> dependent on app itself. Statistically most of apps are fairly unstressful
> and GC can cope with them. Apps like your sample are kicking it out of
> balance.
>
> It's interesting that code like
>
> char c = <somechar>;
> return c;
>
> is more difficult for GC than simple
>
> return <somechar>;
>
> But only in heavy load situations, when code is called 000' of times per
> second. Can anybody confirm / negate this with figures? When you run this 
> in
> Button.Click you won't see any difference.
>
> Problem with arguments like "Calling dispose or
>> implementing the dispose pattern on managed code is strongly discouraged
>> because it directly interfers with the garbage collector" for me is that
> they tend to convey some negative sense that you do you don't know exactly
> what without any helping details. I prefer to base my conclusion on real
> experiments, which demonstrate repeatable behaviors. Generally speaking we
> have to be on low cholesterol. But then we might have problems with
> testosterone or progesterone. What is better? GC is supposed to be
> efficient, but real behavior does not confirm this in all situations.
>
> So, to sum up. If you don't have problems - don't try to make things 
> better
> than they are. Better is enemy of good. But if you have problems - use
> profiler and available tools to find out what is exactly wrong - relocated
> references, excessive allocations or Dispose, which doesn't dispose. 
> That's
> why I said that test sample, even if illustrative, is not worth the 
> effort.
>
> HTH
> Alex
>
> "Alvin Bruney [MVP]" <vapor at steaming post office> wrote in message
> news:%23SAOy05SEHA.3720@TK2MSFTNGP10.phx.gbl...
>> >I have a look at reflector and it looks to be a
>> > good solution.
>>
>> No, it's not a good solution.
>>
>> Implementing Idisposable is reserved for classes which wrap unmanaged
>> resources like bitmaps, database connections and file handles. A dataset
> and
>> its pool of associated objects is managed code. Calling dispose or
>> implementing the dispose pattern on managed code is strongly discouraged
>> because it directly interfers with the garbage collector.
>>
>> The implicit assumption in calling dispose is that the programmer knows
> when
>> the object is no longer in use. Well, that cannot be accurate all of the
>> time especially with datasets/datatables which are remotable,
> serializeable
>> objects. If you feel strongly or have a specific need to empty the 
>> dataset
>> wrapped by your class, provide a cleanup method which sets the dataset to
>> null.
>>
>> -- 
>> Regards,
>> Alvin Bruney
>> [ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
>> Got tidbits? Get it here... http://tinyurl.com/27***
>> "Mendonca" <mendo@cso.com> wrote in message
>> news:OCiWq54SEHA.3768@TK2MSFTNGP11.phx.gbl...
>> >I am wrapping a DataSet in my data structure, should i apply the
>> >IDisposable
>> > to my class and and call dataset.clear from there? Would that kill the
>> > tables inside the dataset? I have a look at reflector and it looks to 
>> > be
> a
>> > good solution.
>> >
>> > thanks
>> > Mendoca
>> >
>> > "Alvin Bruney [MVP]" <vapor at steaming post office> wrote in message
>> > news:u9$bso3SEHA.1768@TK2MSFTNGP10.phx.gbl...
>> >> >That's why you must dispose tables yourself -
>> >> what is your basis for this?
>> >>
>> >> -- 
>> >> Regards,
>> >> Alvin Bruney
>> >> [ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
>> >> Got tidbits? Get it here... http://tinyurl.com/27***
>> >> "AlexS" <salexru2000NO@SPAMsympaticoPLEASE.ca> wrote in message
>> >> news:uemDh$2SEHA.2944@tk2msftngp13.phx.gbl...
>> >> > Not sure if it was really interesting. But
>> >> >
>> >> > I've seen similar behavior with other apps, hence my conclusion. You
>> > just
>> >> > confirmed that. Once again - would be nice to try this on
>> >> > multiprocessor
>> >> > machine. On single processor one GC won't be able to cope properly
> with
>> >> > such
>> >> > number of created references.
>> >> >
>> >> > About working set and GC.Collect. Working set won't help in this
> case.
>> > GC
>> >> > might a bit, but not really. Collect might slow down things but not
>> >> > significantly, because collection is running with lesser priority
> than
>> >> > main
>> >> > process.
>> >> >
>> >> > If you really want to clean up things in real app, take it into
>> >> > profiler
>> >> > and
>> >> > experiment with explicit = null; assignments for references, which
> are
>> > not
>> >> > used anymore, and Dispose calls for internal objects. This test
> sample
>> >> > just
>> >> > is not worth the effort.
>> >> >
>> >> > As about DataSet.Dispose - it doesn't touch internal references,
> which
>> > are
>> >> > unknown to base class. That's why you must dispose tables yourself -
>> >> > use
>> >> > Clear method always and don't rely on simple Dispose call, because 
>> >> > it
>> >> > is
>> >> > same base one as for DataSet. This might apply even to rows - you 
>> >> > can
>> >> > check
>> >> > this yourself if you want to.
>> >> >
>> >> > I wonder if this "unclean" cleaning behavior will be corrected in
> next
>> >> > version of FW
>> >> >
>> >> > HTH
>> >> > Alex
>> >> >
>> >> > "Mendonca" <mendon@cso.com> wrote in message
>> >> > news:uY4t3H2SEHA.1768@TK2MSFTNGP10.phx.gbl...
>> >> >> Interesting you saying that my dispose doesnt touch tables and 
>> >> >> rows,
>> >> > because
>> >> >> i see much data from datarows surving the GC (realocated).
>> >> >> Doesnt the Dispose() method of a DataSet dispose of all underlying
>> > tables
>> >> >> and datarows?
>> >> >>
>> >> >> Cheers,
>> >> >> Mendonca
>> >> >>
>> >> >>
>> >> >> "AlexS" <salexru2000NO@SPAMsympaticoPLEASE.ca> wrote in message
>> >> >> news:uCXNZ0wSEHA.3660@tk2msftngp13.phx.gbl...
>> >> >> > Hi
>> >> >> >
>> >> >> > Quick look told me that you are creating around 300K new objects
> for
>> >> > every
>> >> >> > while loop execution. Most of the time it is tight loop - and 
>> >> >> > most
>> >> >> important
>> >> >> > it seems, that your dispose doesn't touch tables, rows and dr[0]
>> >> >> reference.
>> >> >> >
>> >> >> > GC just can't keep pace with your production speed. Also because
> it
>> > has
>> >> > no
>> >> >> > time to run really. Did you try this on multiprocessor machine? I
>> >> >> > wonder
>> >> >> how
>> >> >> > it will behave there.
>> >> >> >
>> >> >> > HTH
>> >> >> > Alex
>> >> >> >
>> >> >> > "microsoft.news" <msuot@dmod.com> wrote in message
>> >> >> > news:uGXXuBtSEHA.3768@TK2MSFTNGP11.phx.gbl...
>> >> >> > > Anynone?
>> >> >> > >
>> >> >> > > "microsoft.news" <msuot@dmod.com> wrote in message news:...
>> >> >> > > > Hi all,
>> >> >> > > >
>> >> >> > > > Here is the example i would like everyone to concentrate on:
>> >> >> > > >
>> >> >> > > > static void Main(string[] args)
>> >> >> > > > {
>> >> >> > > >  System.Collections.ArrayList list = new
>> >> >> System.Collections.ArrayList();
>> >> >> > > >  System.TimeSpan tenMins =
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >> >
>> >
> System.TimeSpan.FromTicks(DateTime.Now.Ticks).Add(System.TimeSpan.FromMinute
>> >> >> > > > s(1.5));
>> >> >> > > >
>> >> >> > > >  while(System.TimeSpan.FromTicks(DateTime.Now.Ticks) <
> tenMins)
>> >> >> > > >  {
>> >> >> > > >   System.Data.DataSet ds;
>> >> >> > > >   for(int lIndex = 0; lIndex < 100; lIndex++)
>> >> >> > > >   {
>> >> >> > > >    ds = new System.Data.DataSet();
>> >> >> > > >    DataIt(ref ds);
>> >> >> > > >    list.Add(ds);
>> >> >> > > >   }
>> >> >> > > >
>> >> >> > > >   for(int lIndex = 0; lIndex < 100; lIndex++)
>> >> >> > > >   {
>> >> >> > > >    ((System.Data.DataSet) list[lIndex]).Dispose();
>> >> >> > > >   }
>> >> >> > > >   list.Clear();
>> >> >> > > >  }
>> >> >> > > > }
>> >> >> > > >
>> >> >> > > > static void DataIt(ref System.Data.DataSet ds)
>> >> >> > > > {
>> >> >> > > >  System.Data.DataTable tb = new
> System.Data.DataTable("Table");
>> >> >> > > >  tb.Columns.Add("data",typeof(Object));
>> >> >> > > >
>> >> >> > > >  System.Data.DataRow dr = null;
>> >> >> > > >  for(int lIndex = 0; lIndex < 1500; lIndex++)
>> >> >> > > >  {
>> >> >> > > >   dr = tb.NewRow();
>> >> >> > > >   dr[0] = new object();
>> >> >> > > >   tb.Rows.Add(dr);
>> >> >> > > >  }
>> >> >> > > >
>> >> >> > > >  ds.Tables.Add(tb);
>> >> >> > > > }
>> >> >> > > >
>> >> >> > > > So this app ran for approx 1.5 mins and I ran the CLRProfiler
> on
>> >> > this
>> >> >> > and
>> >> >> > > i
>> >> >> > > > got the following:
>> >> >> > > >
>> >> >> > > > Total allocation size: 90MB
>> >> >> > > > Total re-allocated: 41MB
>> >> >> > > >
>> >> >> > > > I can see that in this case if i ran the example for 10 mins 
>> >> >> > > > i
>> > get
>> >> >> more
>> >> >> > > and
>> >> >> > > > more memory allocations. My question is simple, Shouldn't the
> GC
>> >> > clean
>> >> >> > up
>> >> >> > > > after the Dispose() method at all times? I can see a lot of
>> > garbage
>> >> >> from
>> >> >> > > the
>> >> >> > > > DataSet being promoted to gen1 and 2 and not being clean up 
>> >> >> > > > at
>> > all.
>> >> >> > > > It looks like my datarows are never being cleaned up.
>> >> >> > > >
>> >> >> > > > Thanks in advance.
>> >> >> > > >
>> >> >> > > > Mendonca.
>> >> >> > > >
>> >> >> > > >
>> >> >> > > >
>> >> >> > > >
>> >> >> > > >
>> >> >> > > >
>> >> >> > > >
>> >> >> > > >
>> >> >> > > >
>> >> >> > > >
>> >> >> > > >
>> >> >> > > >
>> >> >> > >
>> >> >> > >
>> >> >> >
>> >> >> >
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >>
>> >>
>> >
>> >
>>
>>
>
>
Loading