Re: HeapAlloc heap fragmentation.

Tech Tip: Click here to run a free scan for Windows Errors and optimize PC performance

This is contrary to evidence and implementation (see code and debugger
evidence)
What are you using to conclude that blocks are never released ?

Given the following fragment:

#define MAX_ALLOC 256
#define HEAP_GRANULARITY (2*sizeof(ULONG_PTR))
#define HEAP_ENTRY_MAX 0xFFFE

int __cdecl
wmain(int argc, WCHAR * argv[])
/*++

--*/
{

PVOID Array[MAX_ALLOC];

for (ULONG Iter = 0; Iter < MAX_ALLOC; Iter++){
Array[Iter] = HeapAlloc(GetProcessHeap(),0,HEAP_GRANULARITY *
HEAP_ENTRY_MAX) ;
}

DbgBreakPoint();

for (ULONG Iter = 0; Iter < MAX_ALLOC; Iter++){
HeapFree(GetProcessHeap(),0,Array[Iter]) ;
}

DbgBreakPoint();

return 0;
}


At the first breakpoint:

0:000> !heap -p -all
_HEAP @ c0000
_HEAP_LOOKASIDE @ c0cd0
_HEAP_SEGMENT @ c0c50
CommittedRange @ c0cc0
HEAP_ENTRY: Size : Prev Flags - UserPtr UserSize - state
// snip
VirtualAllocdBlocks @ c0090
2b0030: fffe : N/A [N/A] - 2b0040 (fffe0) - (busy VirtualAlloc)
// other 255 blocks similar to the previous one

At the second breakpoint

0:000> !heap -p -all
_HEAP @ c0000
_HEAP_LOOKASIDE @ c0cd0
_HEAP_SEGMENT @ c0c50
CommittedRange @ c0cc0
HEAP_ENTRY: Size : Prev Flags - UserPtr UserSize - state
// snip
VirtualAllocdBlocks @ c0090
// no blocks, meaning they have been free-ed

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm


"Philip Borghesani" <PhilBorg@xxxxxxxxxxxxxxxx> wrote in message
news:eoMCG9%234FHA.1184@xxxxxxxxxxxxxxxxxxxxxxx
> Our application has bumped into what seems to be a limitation of the
> HeapAlloc system on Win32.
>
>
>
> It appears that if an application allocate a large portion of the
> available address space in object of size less then 0x7FFF8 bytes and then
> releases this memory there is no way to reclaim the address space for
> allocations of size greater then 7FFF8 bytes.
>
>
>
> Using the low fragmentation heap does not seem to help.
>
>
>
> Are there any known work a-rounds for this issue other then replacing
> HeapAlloc (actually standard c++ allocators in Visual C 7)?
>
>
>
> Example code:
>
>
>
> // TestHeapAlloc.cpp This program will cause a machine with less then
> 2gb ram to thrash badly
>
> // do no run with less then 1gb ram.
>
> // if this program is run on a machine with the -3gb boot switch
> (or win64) then the sizes
>
> // may need to be modified to produce a failure
>
>
>
> #include "stdafx.h"
>
> #include "windows.h"
>
>
>
> HANDLE heap=NULL;
>
>
>
> void TestLargeBlock(size_t size)
>
> {
>
> void *ptr=HeapAlloc(heap,0,size); // 500 mb
>
> if (ptr!=NULL) {
>
> printf("Allocated %Id bytes\n",HeapSize(heap,0,ptr));
>
> } else {
>
> printf("Failed to allocate %Id bytes\n",size);
>
> }
>
> HeapFree(heap,0,ptr);
>
> }
>
>
>
> size_t const MaxPointers=10000;
>
>
>
> void TestSmallBlocks(int NumPointers,size_t size)
>
> {
>
> void *pointers[MaxPointers];
>
> int count=0;
>
> memset(pointers,0,sizeof(pointers));
>
> for (int i=0; i< NumPointers; i++)
>
> {
>
> pointers[i]=HeapAlloc(heap,0,size);
>
> if (pointers[i]==NULL) {
>
> break;
>
> } else {
>
> count++;
>
> }
>
> }
>
> printf("Allocated %d items of size %Id for a total of %Iu
> bytes\n",count,size,count*size);
>
> // now free the memory
>
> for (int i=0; i< count; i++)
>
> {
>
> HeapFree(heap,0,pointers[i]);
>
> }
>
> }
>
>
>
> int _tmain(int argc, _TCHAR* argv[])
>
> {
>
> heap=GetProcessHeap(); // initialize program global heap
>
> size_t const AllocCutoff=0x7FFF8;
>
> size_t const SmallAlloc=AllocCutoff-8000;
>
> size_t const BiggerAlloc=AllocCutoff;
>
>
>
> // prove can access and free large memory blocks
>
> TestSmallBlocks(MaxPointers,BiggerAlloc);
>
> TestLargeBlock(0x20000000); // 500 mb
>
>
>
> /// Grow the heap
>
> TestSmallBlocks(MaxPointers,SmallAlloc);
>
>
>
> // check heap info
>
>
>
> size_t maxblock=HeapCompact(heap,0);
>
> printf("HeapCompact erroneously reports a large max block size of
> %Id\n",maxblock);
>
>
>
> // try to allocate a large block
>
>
>
> TestLargeBlock(maxblock/2); // can't even allocate ½ reported size
>
>
>
> // try to allocate slightly bigger small blocks this is the most
> painful example
>
> // possibly not even one allocation will happen
>
> TestSmallBlocks(MaxPointers,BiggerAlloc);
>
> }
>
>
>
>
>
>


.

Relevant Pages

  • Re: printf
    ... > int mainas sombody said *very* correctly. ... What's wrong with int main (void)? ... > portion of memory. ... > You still have to allocate p. ...
    (comp.lang.c)
  • [PATCH 2/2] sparsemem extreme: hotplug preparation
    ... because we'll allocate the memory for the second level in a different ... -static void sparse_index_init(unsigned long section, int nid) ... /* Record a memory area against a node. ...
    (Linux-Kernel)
  • Re: redefining operator new and STL classes
    ... >> Containers don't use new to allocate objects, ... The standard allocator gets memory and then ... which is to say, first memory is allocated using an operator new, ... form with a void* argument does), and finally the pObj is created ...
    (alt.comp.lang.learn.c-cpp)
  • Re: Filling data in function...
    ... What is the best way to do pointer arithmetic, ... a chunk of memory for an array of data. ... > But how did you allocate the space for the character strings? ...
    (alt.comp.lang.learn.c-cpp)
  • Re: Is there a maximum contiguous memory allocation?
    ... but could ALLOCATE it! ... allocate it if I had 2GB of physical memory! ... the amount of physical memory I have installed. ... Note that you can use raw VirtualAlloc to improve your ...
    (microsoft.public.vc.mfc)