Re: Memory problems (possibly very easy question)

Allocate 100,000 bytes. Now allocate 25,000 objects. Free the 25,000 objects. Free the
100,000 bytes. Now go to allocate another 100,000 bytes. Can you get back the original
allocation? Maybe, maybe not. There could be other threads running, for example. There
could be user interactions in between (e.g., the message pump). That huge 100K block gets
fragmented. So you have to allocate a new 100K block. Repeat as necessary.

Superficial examination of allocation patterns is not sufficient to tell if you have
fragmentation. You have to actually examine the heap in detail to tell. Anything you
attempt to infer about heap behavior looking at a small number of lines of code is
suspect. You have to look at the heap as a whole.

Once you get the leaks fixed, fragmentation becomes your worst enemy.
joe
On 30 Nov 2006 05:49:06 -0800, "peter koch" <peter.koch.larsen@xxxxxxxxx> wrote:


Joseph M. Newcomer skrev:
See below...
On 29 Nov 2006 08:03:58 -0800, "peter koch" <peter.koch.larsen@xxxxxxxxx> wrote:


Joseph M. Newcomer skrev:
The question sounds more like the "I freed the memory, why didn't my program get smaller?"
(Usually based on some value like theprogram size in Task Manager). The program doesn't
get smaller because it isn't supposed to get smaller; that's not how storage allocators
work. It is a common delusion that if you free storage your program should get smaller;
it can't, and it won't. The subtleties of this are discussed in my essay on storage
mangement on my MVP Tips site.

I disagree. Read this excerpt from the OP:
But my mem usage does not reduce... so when i run the process a few
times the usage gets really really large, and windows gives me virtual
memory usage warnings (or similar).
****
This indicates the storage is not being released, since the footprint continues to grow.
But releasing it does not reduce the footprint, it merely keeps it from growing as the
released storage is reused. In the case of massively large vectors, however, the
fragmentation may mean that the footprint will continue to grow, for the reasons described
in my essay. That's yet a different problem, but that requires yet a different solution
than those posed thus far. Memory fragmentation is one of the major problems one faces in
dealing with structures of this nature. But that isn't what the question was...the first
answer, that each of the individual elements needs to be deleted, has to be addressed
first.
****

Essentially: every time the process is run memory increases. This is a
very strong indication that memory is not reused.
****
There are two concepts of "reuse" here, and even if the objects are all freed, the space
for the array itself may be freed, fragmented, and then reallocated in a new place, thus
causing the footprint to extend indefinitely. But the memory would be freed; it just
couldn't be reused. So the question remains open. Is it failure to release, or is it
fragmentation? Until the issue about releasing the objects in the array is answered, it
isn't possible to determine if the fragmentation problem exists. If the fragmentation
problem exists, there are two or three alternative architectures that can be applied, but
the question is still open.

I see no possibility for fragmentation. Theres a loop: (1) allocate
memory (b) free memory
This will not cause any fragmentation. I agree that theres a small
amount of freeing memory for the vector (as reallocations take place),
but this will be valid for the first run of the loop.

The only logical answer is that memory does not get released. I look
forward to hear if the OP solves his problem, but I'd bet its related
to a real memory leak somewhere - not fragmentation.

/Peter
Joseph M. Newcomer [MVP]
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
.

Relevant Pages

  • Re: Memory fragmentation issue in kernel mode
    ... Also, on the Performance tab, watch the Kernel Memory section. ... fragmentation, which is actually fragmentation of the system virtual ... A user level application doesn't allocate and even access the ... I have my driver that uses direct IO to perform data transfer ...
    (microsoft.public.development.device.drivers)
  • Re: calloc ( ) failed ... but enough RAM ...
    ... I know how many RAM available has my system by simply looking into the ... > You are ignoring issues like address space fragmentation. ... > safely assume that allocating a contiguous chunk of memory this size is unlikely. ... > There is no way to answer your question about how much space you can allocate; ...
    (microsoft.public.vc.mfc)
  • Re: Memory problems (possibly very easy question)
    ... It is a common delusion that if you free storage your program should get smaller; ... memory usage warnings. ... But releasing it does not reduce the footprint, it merely keeps it from growing as the ... fragmentation may mean that the footprint will continue to grow, ...
    (microsoft.public.vc.mfc)
  • Re: No memory although more than 1GB free
    ... each process is able to allocate 2GB at maximum. ... > However, when my crucial application runs out of memory, Task Manager ... That's why I think there's fragmentation at work. ... Remember - you are not running out of physical RAM, ...
    (microsoft.public.vc.language)
  • Re: No memory although more than 1GB free
    ... each process is able to allocate 2GB at maximum. ... > However, when my crucial application runs out of memory, Task Manager ... That's why I think there's fragmentation at work. ... Remember - you are not running out of physical RAM, ...
    (microsoft.public.windowsxp.perform_maintain)