heapcreate/heapfree and reserved segments
- From: "John Campbell" <jc>
- Date: Mon, 30 Jan 2006 14:02:27 +1300
Symptoms:
Memory allocations using HeapAlloc fail even when there are large amounts of
uncommitted memory available within the address space of the process.
Our Win32 applications encounter unexpected memory allocation failures at
certain sites when process reserved virtual address space approaches the 2GB
limit. When we analyze heap utilisation in process dumps from the site we
find significant amounts of unused free space in the form of free blocks
and large regions of contiguous uncommitted pages.
Background:
This application employs aggressive memory caching strategies for different
purposes and primarily uses the HeapCreate (and associated allocation
functions) to create and manage multiple heaps. Multiple heaps are created
for various reasons, primarily to achieve isolation at a module level. Some
heaps are created on demand only when a particular module is invoked.
VirtualAlloc and VirtualFree are also used within the product.
Description:
1) Observed Heap growth behaviour
When a Heap is created using HeapCreate (and zero initial and maximum size),
an initial 256KB segment is reserved. When the heap is first extended, the
next segment is 1MB and successive extensions double the size of the
previous segment until the segment size reaches a 256MB limit.
2) Observed segment retention behaviour
When the application frees blocks using HeapFree and there are segments in
the heap containing no blocks allocated by the application, reserved
segments are not freed. If you allocate blocks in a heap and grow it to 1GB
say, then free all allocated blocks the reserved virtual address space
remains at 1GB.
Each of these behavioural characteristics gives rise to excessive address
space fragmentation with a low effective usage of process virtual memory.
For example approximately 256MB of total bytes allocated in a heap can
result in 512MB virtual address space utilisation with a sparsely filled
final segment.
This poor Virtual Memory usage efficiency phenomenon makes it very difficult
for our users to configure various cache sizes and per user memory
allocations since the amount of VM address space overhead that will be
incurred in complex and varying usage contexts is too difficult to predict
with any degree of accuracy.
Questions:
1.. Is there any way for an application to influence or override the
described behaviour?
In particular, either: Control the size of segments (heap extent size) or
request that unused segments (address space) to be returned. An improved
HeapCompact perhaps??
2.. Is there any other way to mitigate this issue apart from not using
multiple heaps?
3.. Initial investigation using the HeapCompact API has shown this has no
discernable effect on heap structure? Under what conditions is this
effective ?
.
- Follow-Ups:
- Re: heapcreate/heapfree and reserved segments
- From: Pavel Lebedinsky [MSFT]
- Re: heapcreate/heapfree and reserved segments
- Prev by Date: Re: How to get DLL address with the name
- Next by Date: Re: How to get DLL address with the name
- Previous by thread: IOCTL_STORAGE_QUERY_PROPERTY and getting Inquiry info.
- Next by thread: Re: heapcreate/heapfree and reserved segments
- Index(es):
Relevant Pages
|
