Memory allocation on Windows XP

Tech Tip: Click here to run a free scan for Windows Errors and optimize PC performance

I am working on an app that need a lot of memory. On Win XP 64,
everything is fine but on the 32 bit version I run in allocation
failures even though I have plenty of virtual address space free (VM
Size < 1Gb on a machine with 2 Gb of Ram + 4 Gb of swap)..

This is probably due to the pattern of memory allocation I use.
Basically the app allocates small chunks and large chunks of memory, ~
3.5 Gb each, using new [], until new fails (I have installed a new
handler via _set_new_handler() so I get notified when new fails). At
that point the VM size is usually aroung 1.7 Gb, and I free some of
these big memory chunks to give the app some breathing room. VM Size
goes down. But as the application runs, not allocating large buffers
anymore, new will fail on a small allocation (sometimes 100 bytes)
even though I just released over 30Mb of my large chunks. At the time
new fails VM Size is less than 0.8Gb and almost all the large chunks
have been released.

I suspect from browsing the newsgroups that this is because large
allocations are routed to a special heap or list using VirtualAlloc,
and basically all the address space has been reserved with those large
allocations. Is that correct?

Is there a way I can make this address space available for the small
object heap again?

Is there a way I can make the small object heap larger when my program
starts, to ensure it won't run out of space? or maybe creating a second
heap and make it available to the new operator in C++?

Any hints or suggestions would be greatly appreciated.

Best regards,

greg

.

Relevant Pages

  • Re: When to check the return value of malloc
    ... code to handle the failure of every single allocation. ... most of the time the program will fail to get memory again ... meanwhile the app may be able to do other useful things. ... In other cases - such as the one I mentioned - an allocation failure is ...
    (comp.lang.c)
  • Re: xmalloc string functions
    ... recover from any allocation failure. ... You get the memory, or you exit. ... A complex app where there may ... Or database apps where there isn't a robust memory recovery strategy in place - e.g. anything which uses an X server. ...
    (comp.lang.c)
  • Re: Program "close to the machine"
    ... in an HRT system must be tightly bounded, ... minutes warning before we ran out of memory. ... You can test the conformance by plotting the latency jitter of the app ... allocation rate has a big spike at the worst possible moment due to ...
    (comp.lang.lisp)
  • Re: xmalloc string functions
    ... What does xmallocdo when unable to allocate memory? ... from any allocation failure. ... No, no, it makes error handling *easier*, because you don't ever need to ... you get the memory, or the app dies. ...
    (comp.lang.c)
  • Re: buddy system
    ... A good resource would be nice as well describing the buddy system. ... Memory allocation is done in ... In case there is no free chunks of a specific size, ... a larger allocation can be split in two. ...
    (comp.os.linux.development.system)