Re: Calculating Virtual Memory Fragmentation

1. The entire region of pages returned has the same protection and state attributes, so you can skip ahead the size of the region for the next query.
2. The heap takes up 4k (or larger) regions and subdivides them into smaller, more managable blocks. So even thought an entire heap segment might be `committed` at the OS/VM-level, only the parts that the heap manager has handed out to applications are truly `in-use.

BTW, check out the `!address -v' command in WinDbg; it has useful facilities for diagnosing address space consumption (including reservations vs commits).

--
Ken Johnson (Skywing)
Windows SDK MVP
http://www.nynaeve.net
"ravenous_wolves" <ravenous.wolves@xxxxxxxxx> wrote in message news:1169505296.539000.205210@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
I'm seeing a problem in an long runtime application that I'm working
which which I believe to be virtual memory fragmentation. A change was
made which greatly increased the # of fast alloc/release cycles and the
system MTBF dropped significantly and the failure profile was unable to
allocate memory despite physical memory still being available.

SO, I'm trying to quantify this effect and give some type of # which
can report how fragmented the virtual address space of our process is.
I've used a MS tool (DebugDiag) which actually reports this, but MS is
unable to get this tool to work on our application (generating a dump
locks the OS).

I've got two ideas, but I'm stuck in terms of mapping my
programmer-level concept of virtual memory to the real low-level
implementation details of Windows.
1. Use VirtualQueryEx, and start at address 0x0 and work my way up to
0x80000000, and keep notes on the sizes of free blocks and the sizes of
committed blocks. One question is, if VirtualQueryEx returns and says a
"range of pages" from N to N + 2MB is committed, does that mean that a
2MB region is completely committed and 100% unfragmented, or does that
mean that the set of pages is all committed, and each page has it's own
fragmentation characteristics?

2. DebugDiag reports virtual memory fragmentation on a per-heap basis.
I understand how to enumerate all the heaps in my process with
GetProcessHeaps and HeapWalk. This seems right in terms of per-heap
fragmentation, but I don't understand virtual memory fragmentation to
be a per-heap concept, but rather a per-process concept. Furthermore,
the Region field of PROCESS_HEAP_ENTRY indicates that part of the block
is committed and part is uncommitted, which makes me confused as to
what is being enumerated.

Sorry if that doesn't make much sense, if anyone has some brief
guidance on what kind of API calls and calculation would need to be
done to quantify virtual memory fragmentation that would be great....

Thanks,
-ken


.

Relevant Pages

  • Calculating Virtual Memory Fragmentation
    ... DebugDiag reports virtual memory fragmentation on a per-heap basis. ... I understand how to enumerate all the heaps in my process with ...
    (microsoft.public.win32.programmer.kernel)
  • Re: Calculating Virtual Memory Fragmentation
    ... DebugDiag reports virtual memory fragmentation on a per-heap basis. ... GetProcessHeaps and HeapWalk. ...
    (microsoft.public.win32.programmer.kernel)
Loading