Re: Bug in MFC CList?

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

What can I say, I'm enlightened... And I'm not kidding.
I can surely 'hear' (or read) your experience between the lines,
And it is nice to get some things clear from someone with more experience...


I guess really can't even guess where are the performance issues in my code,
until I'll make an alpha version of the code, that I can make tests on.
And BTW, if you have a recommendation for any tool that can assist in
tracking performance problems (including memory
allocation/deallocation/leaks),
It will be great...

About the irrelevance of the amount of code to the performance, I truly
understand what you are saying...
I encountered this thinking in a more riducilous way:
A friend of mine who was programming only for 6 years, had a habit of making
his comments the shortest possible, and write as much as he can in one line,
using operators to make it look shorter.
So when he gave me a piece of code to debug for him, I told him that he is
just making his life harder, and in many cases the shortest code is not
optimum,
because he didn't think of what he is making the compiler do behind the
scenes...
And that /*comments*/ do not take any space in the final compiled code... :-D

Thanks for your time and knowledge, you have put quite an effort there for
me...
I very appreciate it!

Daniel C. Gindi

"Joseph M. Newcomer" wrote:

See below...
On Mon, 23 Jul 2007 14:12:02 -0700, Dani <Dani@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote:

First about the last thing you wrote, 'char' or 'unsigned char' can also
represent a single byte, which you might have to make calculations on, or
read from some stream, or whatever. 'char' is not dead yet. In all of my
applications I make full Unicode support, but sometimes you really need
'char's, when it is not referring to strings...
*****
Typically I find 'BYTE' more useful than 'char'. It also makes it clear that I'm working
with byte data. A problem with using 'char' is so many people don't know about Unicode
and keep using 'char' as a string representation, so it helps to disambiguate if you are
using it to hold signed 8-bit data. But for stream data, mostly it is unsigned data
anyway.
****

About the managed code, I really cannot assert anything about it,
because i'm not using managed code, and i'm not really sure I need it...
Any way I mixed up 'managed code' with the 'vector'... I was just referring
to the fact the the compiler manages all the memory allocation.
*****
Managed code and the std:: classes are not related. So there's little reason to not use
std:: constructs unless you have truly measured their impact.

I spent 15 years doing performance measurement (I had produced the deifnitive performance
measurement tool for our environment, and spent the rest of my career on that platform
helping people optimize their code when they used it to get measurements). I learned
several things
- if you ask a programmer where the hotspot is in their code, you will get the
wrong answer
- architecture accounts for orders-of-magnitude change, lines of code accounts
for single-digit percentages
- without performance numbers, any opinion of where performance is going
is meaningless
- Improvement in code performance is unrelated to effort involved

I learned that even with my own code, I was almost always wrong about where performance
was going. So I've learned that until you know where the performance problems are, it is
far more important to write clear, understandable code using standard libraries than try
to make it "small and fast" (this usually means that instead of your cleverly-optimized
code taking 0.05% of the total execution time, it now takes 0.025% of the execution time,
and the performance hog that is consuming 30% of the time is missed entirely).

Compilers do not manage memory allocation; run-time libraries manage memory allocation.

For example, one poster complained that he could not create more than about 5,000 elements
per second in a CArray. It turns out that he was the victim of two issues. One was that
in the debug version, allocation of array objects that have complex structure really IS a
performance bottleneck, and the other was that he was doing an Add() for each of about a
million objects, getting him into what is known as the "exponential growth" problem. In a
release version where a massive nGrowBy parameter was given to the SetSize call, I was
able to create about 12,500,000 elements per second. So never, ever base performance
numbers on debug versions, and observe that small algorithmic changes can have massive
impact on performance. std::vector also has excellent growth performance, using an
exponential allocation algorithm.

So in the absence of actual performance numbers, opinions rarely matter. You have fallen
into one of the common fallacies of programmers: that the cost of using libraries is too
high. It is only too high when you have numbers to prove it. And you can only apply
these numbers in release mode versions.
*****

BTW, To all of those who said that there's no need to use pointers, they are
really very very wrong.
There are many cases when pointers would spare a lot of unnecessary work,
including memory allocations
For example, returning a class type from a function: You spare calling copy
constructors if you do not need a copy.
*****
Or you could use & parameters in which case you pass back a reference, in which case a
pointer is not required. Don't confuse a bad design that requires copy constructors with
a need to use pointers.
*****

Finally, about that 'infinitesimal amount of code':
I AM writing complex algorithms, and many times, a single extra memory
allocation, or assignment, in a loop, would make a great difference.
*****
But as soon as you call malloc, you are imposing a truly large amount of
overhead...perhaps an order of magnitude greater than the loop cost. For example, some
years ago, my highly-optimized allocator was said to be faulty because 30% of the time
spent in the program was in my allocator. So I tweaked a couple performance counters, and
found that my allocator was being called 4,000,000 times! It turns out there was an
unnecessary allocate/free being done in an inner loop. Once you move the allocation out
of the loop, the incremental cost of using a std:: class or CArray, for example,
disappears and is not measurable. There are lots of useful tricks for doing
optimizations, but worrying about the difference between 'new' and 'malloc' is not a
useful one. A more important decision is the decision between doing dynamic allocation
and not doing dynamic allocation. That can have an order of magnitude effect on your
performance. Compared to this, the tiny difference bewteen 'new' and 'malloc' no longer
matters.

This is what I mean by "architecture matters more than lines of code". Don't worry about
the difference between 'new' and 'malloc'; worry about why you need to do dynamic
allocation in the middle of an intense calculation! Why not pre-allocate outside the
loop?

I've written an awful lot of performance-sensitive code in my career, and the way I get it
to run fast is to look at the big picture, not at lines of code. (For example, did you
know that the difference between an L2 cache miss and a page fault is six orders of
magnitude performance degradation? Or that for array-based computations, designing an
algorithm that takes advantage of cache line hits can give you a performance improvement
of 20X the unoptimized code? Look at the algorithms, not at the lines of code. For
example, to optimize a page-fault issue some years ago, I had to write about five times
the code I previously had. The code got larger, and it got more complex, and had more
control constructs, and ran in seconds instead of hours!) Only when you have convinced
yourself that you have made all the possible architectural changes is it worth worrying
about lines of code. I would treat any algorithm that required enough allocations to make
the malloc/new difference noticeable could benefit far more by avoiding the frequency of
allocation entirely!
joe
*****

Thanks for your comments!
Daniel

"Joseph M. Newcomer" wrote:

See below...
On Mon, 23 Jul 2007 08:18:08 -0700, Dani <Dani@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote:

But can you tell me, if I'm allocating the way you are suggesting,
how much does it cost me "behind the scenes"?
I mean, in assembly code.
****
An infinitesimal amount of code not worth worrying about

Seriously, unless you are doing this in the inner loop of some complicated algorithm,
worrying about the generated or executed code is not particularly useful. It just doesn't
matter.
*****
I normally use malloc()/free() or new/delete, knowing that it is the less
expensive way.
*****
This assumes that you have a clue as to what either of these cost, and have the slightest
clue of what extra expense is incurred to do the job right instead of just doing raw
allocation.
I do not like the managed code very much, because usually my
applications are of the type which have to do a lot of work in the shortest
time....
*****
You are probably concerned about the wrong problem. Managed code solves a particular set
of problems that usually result in otherwise complex code. When you need performance, you
need performance, but then you have to question why you would be doing allocation at all.
The real question is the incremental cost of doing the job right; if you do a malloc(),
you still have to initialize the fields, so the cost of doing this in a constructor is
about the same as writing the code to do it after a malloc, so it doesn't make any
measurable difference.

Managed code tends to incur performance penalties at different places than non-managed
code, but if your algorithm does not hit these situations, it can be as efficient as
native code. In the absence of actual performance numbers, you have no basis to assert
than any one method is faster or slower than any other method.
joe
*****

Thanks for the new[]/delete[] tip.

But it seems that if i'm doing something like this: new char[20], then
simply delete will free the memory completly, or am I wrong?
*****
If you do the unlikely thing of writing
char * p = new char[20];
then indeed
delete [] p;
deletes the entire block of storage pointed to by p.

But why would you be allocating char in a modern program? TCHAR, yes, but char?
joe


"Doug Harrison [MVP]" wrote:

On Sun, 22 Jul 2007 23:12:02 -0700, Dani <Dani@xxxxxxxxxxxxxxxxxxxxxxxxx>
wrote:

Hello there!

I have just noticed a bug in MFC, in the CList class.
To make sure there are no unexpected memory leaks,
Just create a new clean MFC solution, and write the following, somewhere...:

CList<int> * lstBuggy = new CList<int>[5];
delete lstBuggy; // Exception thrown - Corrupted Heap!!!

Something is going wrong when dynamically allocating an array of CList...

I'm working with VS2005 SP1, if somebody out there can test this code with
other version of VS, it'll be great...

No need. You are using scalar-delete with array-new, which is undefined.
You must always match allocation and deallocation methods, which means
matching up new/delete, new[]/delete[], malloc/calloc/realloc/free, etc.

Of course, you should simply be using:

CList<int> lstBuggy[5];

If you truly need to dynamically allocate it, you should use a container
class such as std::vector, and then you don't have to worry about deleting
the CList array. In general, if you're using delete on a regular basis, you
aren't using C++ effectively. You should be using smart pointer classes
such as std::auto_ptr and boost::shared_ptr for scalars and container
classes such as std::vector for arrays.

--
Doug Harrison
Visual C++ MVP

Joseph M. Newcomer [MVP]
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm

Joseph M. Newcomer [MVP]
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm

.

Relevant Pages

  • Re: !! Help !! Finding an open range within a range of numbers
    ... > This is similar to disk storage where one wants not only a contigious ... > set of sectors for _this_ storage, but an algorithm which reduces the ... There are three basic allocation schemes: ... covering both memory and file systems. ...
    (comp.lang.pascal.delphi.misc)
  • Re: Bug in MFC CList?
    ... Typically I find 'BYTE' more useful than 'char'. ... Compilers do not manage memory allocation; ... exponential allocation algorithm. ... overhead...perhaps an order of magnitude greater than the loop cost. ...
    (microsoft.public.vc.mfc)
  • Re: Bug in MFC CList?
    ... memory to allocate, and I can't allocate a fixed amount". ... run with a dynamic allocation. ... exponential allocation algorithm. ... overhead...perhaps an order of magnitude greater than the loop cost. ...
    (microsoft.public.vc.mfc)
  • Re: contributing to fbsd
    ... In linux it is addressed ... > in which allocation in mmap was discussed. ... > As it was appeared that fbsd use Oalgorithm here compared ... tjr has some patches to do Opid allocation, ...
    (freebsd-hackers)
  • Re: xmalloc string functions
    ... applications doing complex tasks for which memory allocation failure is only ... applications which do try to handle malloc failure of course). ...
    (comp.lang.c)