Re: Strategy for caches & GC

Barry, thanks for the response, see inline.

On Apr 21, 6:35 am, Barry Kelly <barry.j.ke...@xxxxxxxxx> wrote:
It sounds like you need a more aggressive cache expiration policy. As
Rico Mariani [Google] likes to say, a poorly implemented expiration
policy is just a memory leak by another name. What you say makes me
think that the cache is getting a too-low hit rate, so it needs throw
away most of it too often. Perhaps it shouldn't be caching much of this
stuff at all in the first place?

As mentioned, this was stress testing so I wanted to push the cache to
the size limit and see how it would respond while scavenging.


When dealing with large amounts of data, I find GC is often not the best
way to go. I end up writing manual allocators for byte arrays. This
makes most sense when the data you're caching is in fact a byte array,
or other value-typed array, of course. However, I still allocate those
byte arrays from the .NET GC. It's just that they're all over the large
block limit (80K or so), and I carefully recycle them through a pool.

Can you explain this further? Do you mean you alloc a huge block of
memory to insure it sits in the LOH and suballocate from there (ala
your own memory management on the managed heap?) If so, how does the
suballocations look? Does C# have a placement new like C++?



BTW, you can still do manual memory allocation in C#, using the Marshal
class or by writing the PInvoke signatures yourself. However, I'd take a
closer look at your expiration policy before I'd go off and try and
reimplement a cache.


How does one serialize/deserialize managed objects onto the native
heap if these managed objects can be derived from a common base (e.g.
CacheItem<-MyCacheItem, CacheItem<-MyOtherCacheItem)? I'm (also)
wondering if memory fragmentation will start occurring in the native
heap?

Thanks!
.

Relevant Pages

  • memory leak in (?)... (redux)
    ... the mem leaks in our long-running tcl daemons. ... when actively exercised the app grows continuously. ... the mystery is that while our audit code does show us a couple of arrays ... and memory are checked and meanwhile externally, ...
    (comp.lang.tcl)
  • Re: StackOverflowException.
    ... The Status object monitors the ... so I'm really at a total loss as to where I'm leaking stack memory. ... private static extern bool EscapeCommFunction(IntPtr hFile, ... Good to know about arrays. ...
    (microsoft.public.dotnet.framework)
  • Re: Strategy for caches & GC
    ... I end up writing manual allocators for byte arrays. ... memory to insure it sits in the LOH and suballocate from there (ala ... ArraySlicehandle to work with buffers. ... and manual management for memory and GC generation 0 for efficient ...
    (microsoft.public.dotnet.framework.clr)
  • Re: perl eating up memory slowly until program stops - any ideas ?
    ... Arrays are created, variables are set, ... For lexical scoping, a single chunk of memory, reserved by ... Private blocks, you have discussed this a number of times ...
    (comp.lang.perl.misc)
  • Re: question about negative indices in fortran 77
    ... > Intel compilers for linux), but now I'm doing some modifications and I ... > subroutine where it isn't touched. ... out memory in the order arrays are declared. ...
    (comp.lang.fortran)