Re: Strategy for caches & GC

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

devdude wrote:

I am currently using the Enterprise Library Caching Block on a server
that receives incoming calls and stores/retrieves random information.
I noticed while load testing, the system is slow to respond (a call
can take several seconds) during scavenging of old items when the
threshold is met. It is reclaiming approx 4gb of data during each
scavenge on approximately 14gb of cache. I realize this is a lot of
information but am looking for strategies to implement a performant
cache in .Net. I'm not particularly married to entlib so anything is
an option.

It sounds like you need a more aggressive cache expiration policy. As
Rico Mariani [Google] likes to say, a poorly implemented expiration
policy is just a memory leak by another name. What you say makes me
think that the cache is getting a too-low hit rate, so it needs throw
away most of it too often. Perhaps it shouldn't be caching much of this
stuff at all in the first place?

When dealing with large amounts of data, I find GC is often not the best
way to go. I end up writing manual allocators for byte arrays. This
makes most sense when the data you're caching is in fact a byte array,
or other value-typed array, of course. However, I still allocate those
byte arrays from the .NET GC. It's just that they're all over the large
block limit (80K or so), and I carefully recycle them through a pool.

One thought I had was to perhaps rewrite the cache (or a new one from
scratch) in C++/CLI to handle all the memory storage/reclamation in
native code but am concerned with marshalling data back and forth
between native/managed code. That said, the GC multisecond swings are
worse then knowing that a call to the cache will take, for example,
5ms, instead of 1ms.

BTW, you can still do manual memory allocation in C#, using the Marshal
class or by writing the PInvoke signatures yourself. However, I'd take a
closer look at your expiration policy before I'd go off and try and
reimplement a cache.

-- Barry

--
http://barrkel.blogspot.com/
.

Relevant Pages

  • Re: [patch] SLQB slab allocator
    ... Newly freed objects tend to be cache hot, ... +#ifdef CONFIG_NUMA ... we reduce the number of allocators in the tree... ...
    (Linux-Kernel)
  • Re: Strange behavior while rendering display lists or vertex arrays
    ... GPU's local memory using same order as the cache entries in ... Don't bother with indexed triangle strips, they are waste of time, ... As long as i can i will use them, because they also reduce memory usage (well for Vertex Arrays only the index arrays gets smaller and so bus traffic is reduced, but that also reduces memory needed for Display lists in VRAM). ...
    (comp.graphics.api.opengl)
  • Re: CPU starvation, L2 missed cache
    ... It is clear that all the tables won't fit in the cache. ... Have you got some advices when working with arrays? ... If the arrays aren't OpenMP private, and your OpenMP do loop is outside what you have shown, you make it appear that you have serious race conditions, so cannot expect correct results, let alone satisfactory performance. ...
    (comp.lang.fortran)
  • Re: should every thing be zero indexed?
    ... >> My guess is that in the casual heap, ... > I now believe these results are due solely to cache behavior when run ... > on large arrays. ... N hsortx1 hsortm husort husort2 ...
    (comp.programming)
  • Re: Question on Lynnfield chip layout
    ... An example is CPU based character skinning, huge arrays of tens of thousands of verts, and texture coordinates, and bone weights on those verts. ... Otherwise the cache will fetch this data from RAM to merge in sub cache line writes, not realizing this is BAD for performance. ... Then when it is purged from L2 the cache should decide to commit to memory, and NOT send to the L3, due to the zero L2 hits. ...
    (comp.arch)