Re: CObArray with huge number of elements

Tech-Archive recommends: Fix windows errors by optimizing your registry

Actually, not true. You can read the code, or, as I did, single-step through the code to
see that it does the obvious thing. Add calls SetAtGrow. SetAtGrow examines the size that
has been allocated, and if it is within the current bounds, simply initializes the element
to 0 and returns, allowing the assignment to take place without any additional allocation.
I found that the first time I did an Add() (having done SetSize(0, 100000), it allocated
one item (having taken the path that the array was empty and its data pointer was NULL).
The next time, it added 100000 elements, and the third time it just set the element.

However, he has not stated if he is doing this in the debug version, and it is worth
examining the output of this little console app which was created as "Uses MFC", because I
was using CDWordArray. The argv[1] is the nGrowBy value for SetSize.

int _tmain(int argc, TCHAR* argv[], TCHAR* envp[])
{
int nRetCode = 0;

// initialize MFC and print and error on failure
if (!AfxWinInit(::GetModuleHandle(NULL), NULL, ::GetCommandLine(), 0))
{
// TODO: change error code to suit your needs
cerr << _T("Fatal Error: MFC initialization failed") << endl;
return 1;
}

CDWordArray dw;
int GrowBy = 100000;
if(argc > 1)
GrowBy = atoi(argv[1]);
dw.SetSize(0, GrowBy);
printf("Start: size=%d, GrowBy=%d\n", dw.GetSize(), GrowBy);
LARGE_INTEGER start;
QueryPerformanceCounter(&start);
int i;
for(i = 0; i < 250000; i++)
dw.Add(i);
LARGE_INTEGER stop;
QueryPerformanceCounter(&stop);

LARGE_INTEGER deltaT;
deltaT.QuadPart = stop.QuadPart - start.QuadPart;

LARGE_INTEGER frequency;
QueryPerformanceFrequency(&frequency);

LARGE_INTEGER ms;
ms.QuadPart = (deltaT.QuadPart * 1000) / frequency.QuadPart;
double seconds = (double)ms.QuadPart / 1000.0;
printf("%d elements, deltaT = %8.3f sec, rate = %1.0f/sec\n", i, seconds, (double)(i)
/ seconds);

printf("done adding items: size=%d\n", dw.GetSize());

return nRetCode;
}

Note the following:
In the debug mode, the larger GrowBy is (the input parameter), the slower it runs. For a
simple loop, I get 26K/sec for 10K, 17K/sec for 100K, and 2693/sec for 1M. But now look
at the data in the release version: for 10K, I get nearl 6M/sec, for 100K I get 12.5M/sec,
and for 1M I get 15.5M/sec.

I hand-edited the commas into the numbers so they were easier to read.

Engineering means getting data to validate your decisions. Collecting performance data on
a debug version clearly has nothing to do with reality, and therefore can be dismissed are
irrelevant to solving the problem.

Now, just to show the overheads are in the debug library support, and not in the code
itself, I created a new build configuration, "Unoptimized", in which I built a release
version wtih all optimizations disabled. Check out the numbers at the end.

It is important when solving problems to solve tie RIGHT problem, and performance is not
even something to consider in the debug version. Sure, it slows down the development, but
it cannot in any way be considered relevant to what release performance is going to be. To
determine release performance, the only valid way is to measure the release version.

By the way, it makes a cool Excel spread***, particularly if the y-axis is plotted
logarithmically.

c:\>c:\tests\add\debug\add 10000
Start: size=0, GrowBy=10000
250000 elements, deltaT = 9.385 sec, rate = 26,638/sec
done adding items: size=250000

c:\>c:\tests\add\debug\add 100000
Start: size=0, GrowBy=100000
250000 elements, deltaT = 14.090 sec, rate = 17,743/sec
done adding items: size=250000

c:\>c:\tests\add\debug\add 1000000
Start: size=0, GrowBy=1000000
250000 elements, deltaT = 92.834 sec, rate = 2,693/sec
done adding items: size=250000

c:\>c:\tests\add\release\add 10000
Start: size=0, GrowBy=10000
250000 elements, deltaT = 0.043 sec, rate = 5,813,953/sec
done adding items: size=250000

c:\>c:\tests\add\release\add 100000
Start: size=0, GrowBy=100000
250000 elements, deltaT = 0.020 sec, rate = 12,500,000/sec
done adding items: size=250000

c:\>c:\tests\add\release\add 1000000
Start: size=0, GrowBy=1000000
250000 elements, deltaT = 0.016 sec, rate = 15,625,000/sec
done adding items: size=250000

c:>c:\tests\add\unoptimized\add 10000
Start: size=0, GrowBy=10000
250000 elements, deltaT = 0.045 sec, rate = 5,555,556/sec
done adding items: size=250000
96% of the rate of the optimized version

c:>c:\tests\add\unoptimized\add 100000
Start: size=0, GrowBy=100000
250000 elements, deltaT = 0.021 sec, rate = 11,904,762/sec
done adding items: size=250000
95% of the rate of the optimized version

c:>c:\tests\add\unoptimized\add 1000000
Start: size=0, GrowBy=1000000
250000 elements, deltaT = 0.019 sec, rate = 13,157,895/sec
done adding items: size=250000
84% of the rate of the optimized version
joe

On Fri, 01 Jul 2005 18:28:22 -0500, "Scott McPhillips [MVP]" <org-dot-mvps-at-scottmcp>
wrote:

>phil wrote:
>
>> thanks for the quick reply
>>
>> I've tried the following
>>
>> As a simple test I tried this...
>> CObject *this = new MyObject();
>> CObArray things;
>> things.SetSize(0,1000000);
>> for (long y=0;y<1000000;y++)
>> things.Add(object); // just add the same one repeatedly
>>
>>
>> while this does not grind to a halt it is VERY slow - about 1000 element per
>> second
>>
>>
>> this
>> CObject *this = new MyObject();
>> CObArray things;
>> things.SetSize(1000000); // set the size before hand
>> for (long y=0;y<1000000;y++)
>> things[y]=object;
>>
>> is too fast to notice anything has happened -
>>
>> Is there some bad overhead with CObArray::Add ?
>>
>> thanks for the reply and any other ideas will be much appreciated
>
>That "too fast" version is the right answer. After you set the size you
>don't want to use 'Add' any more. 'Add' makes the array size larger.
>This seems to be stated pretty clearly in the docs for CObArray::Add
>that I am looking at. If you haven't found the docs yet try clicking on
>CobArray in your code, then hitting the F1 key.

Joseph M. Newcomer [MVP]
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
.

Quantcast