Re: CStrings and memory

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

From: Joseph M. Newcomer (newcomer_at_flounder.com)
Date: 03/09/04 

Date: Mon, 08 Mar 2004 21:07:40 -0500

The behavior you see is the expected behavior, and perfectly normal. Suppose memory is
free:

FFFFF

Now you allocate a string:

SFFFF

Then you allocate a few more strings

SSSSS

Now you need to allocate another string, but there is no more memory. So the program calls
the kernel to get more memory:

SSSSSFFFFF

and allocates the new string from that

SSSSSSFFFF

In addition to strings, there are other things that get allocated:

SSSSSSXYZW

now you free up all the strings

FFFFFFXYZW

Do you see why memory usage doesn't go down?

perfmon and similar tools simply measure address space footprint. They don't tell you (nor
could they ever find out) how much of that space is available for reallocation relative to
how your program behaves.

If you need to allocate strings in the fashion you describe, one way to do it is to create
your own heap and manage that. When you are all done, you can free the heap.
Unfortunately, MFC does not make it possible to do this and still use CString or any other
MFC component. However, STL DOES allow you to write an allocator (someone pointed this out
a few weeks ago in response to some other question) of your own, which would mean that you
could use std::string and std::vector with your own custom allocator.

However, this raises another question: why do you think this matters in the slightest? You
have perhaps confused "address space" with "working set". If the strings form a massive
allocation (let's say each of those letters I used above represents one page of memory),
then how many pages are used after all the strings are freed? 10? 4? The answer would be 4
in the example I show; although the address space is still 10 pages, the actual pages
required in memory are 4 pages, and the rest just go out to the disk and stay there. If,
however, you have allocations interspersed with the strings so that other things are also
allocated, then indeed you have a more serious problem, address space fragmentation, where
all the pages have to be there because each page contains at least one useful thing. But
that is not what you have measured, so until you know the actual working set size, it is
meaningless to talk about "size" as if there is some absolute criteria for deciding a
program is "good" or "bad". If you have the situation where all the pages that had strings
are simply no longer in use, it probably doesn't matter much.

Also, you have not given numbers to describe these. Unless you have absolute numbers to
express the issues, talk about "optimization" is completely meaningless.

What you need to measure are the working set size before the strings are allocated (which
has nothing to do with the amount of memory that perfmon may report; that's just memory
footprint and largely irrelevant), the working set size after all the strings are freed,
the percentage of the expected CPU this represents (if you are writing for a 64MB machine,
100MB of working set is a Bad Idea. If you're writing for a 256MB machine, 10MB of working
set are not even worthy of discussion). But without knowing all these numbers, it is
impossible to tell you the correct approach, which could be anything from the one extreme
of learning how to write STL allocators to the other extreme of "you are worrying about an
issue that is completely and utterly irrelevant". The truth may be somewhere in between,
but it i impossible to tell from the vague handwave here.

I've found people describing "huge" meaning "1MB". Words and phrases like "huge", "a lot",
"massive amounts" and so on have different calibrations. I don't think of heap usage as
being an issue until it becomes at least 20% of the machine, which I currently calibrate
at a 256MB machine. So pretty much anything under 50MB isn't worth my time to deal with.
I'll map tens of megabytes of data into my address space without giving it a thought.
Others think 10MB to be outrageous (they're wrong; it isn't even on the radar). So
quantify. And only working set matters, not address space. Working set is dynamic and
takes a while to decrease after the strings have been freed.

You may still have a problem, but without numbers I can't tell.
                                joe

On Mon, 8 Mar 2004 08:16:08 -0800, Vicent Soler <anonymous@discussions.microsoft.com>
wrote:

>Hi all,
>
>I'm developing a tool which uses CString and CStringList a lot. When adding data to both, CString and CStringList I can see that memory usage of my application increases. After deleting all data I can see using Perfmon.exe that memory does not decrease but is constant. If afterwards, I try to use again the same method than before I can see that memory does not increase this time as before. It seems that CString and CStringList reserves that memory after being deleted, is it possible?? I mean, after using CStrings and CStringList, memory in use does not decrease in the Perfmon.exe graph but it is available for being use again.
>
>The problem for me is that I've a memory control procedure for my application because it uses a huge amount of memory and I need to get the memory decreased after deleting the strings.
>
>NOTE: I have check my application using every available Memory Leak detector and there is not a failure. I've also checked whether CStrings are correctly deleted and they are. So it is not a memory leak failure.
>
>Thanks in advance,
>
>Vicent

Joseph M. Newcomer [MVP]
email: newcomer@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm

Relevant Pages

  • Re: CStrings and memory
    ... Then you allocate a few more strings ... Now you need to allocate another string, but there is no more memory. ... What you need to measure are the working set size before the strings are allocated (which ...
    (microsoft.public.vc.mfc)
  • Re: Basic question : Replacing CString when moving from MFC to Win32
    ... you'll have to allocate the memory because LPTSTR is a pointer. ... >> Regarding the strings, will they be defined as LPCTSTR or LPTSTR? ...
    (microsoft.public.vc.mfc)
  • Re: Basic question : Replacing CString when moving from MFC to Win32
    ... you'll have to allocate the memory because LPTSTR is a pointer. ... >> Regarding the strings, will they be defined as LPCTSTR or LPTSTR? ...
    (microsoft.public.pocketpc.developer)
  • Re: memory allocation questions (newbie)
    ... I might want to allocate an array of strings. ... [releasing memory once it is no longer required]. ... Reading email is like searching for food in the garbage, ...
    (comp.lang.c)
  • Re: mem-f-nomem error
    ... the program won't fail, but performance will be less than ideal. ... memory block to be in memory. ... Consider a case where your working set is 10 megabytes. ... allocate a 15megabyte chunk of contiguous memory and have it all zeroed. ...
    (comp.os.vms)