Perfomance problem while passing large data volumes via WMI

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

From: Stanislav Efremov (hell_at_sw.ru)
Date: 08/18/04 

Date: 18 Aug 2004 08:34:24 -0700

Greetings,
I have noticed a perfomance problem while trying to pass large data
through WMI. Suppose we have large file at server side (the size of
the file is about 300 Mb) and we want to pass it to the client side.

Here's my code
unsigned nCount; //size of data
CComVariant varBuf; //CComVariant where we store the data

//fill the variant, nothing of interest here
HRESULT hr;
SAFEARRAY * psaData;
SAFEARRAYBOUND saBound[1];
saBound[0].lLbound = 0;
saBound[0].cElements = nCount;

psaData = SafeArrayCreate(VT_UI1, 1, saBound);
if (!psaData)
{
        return E_OUTOFMEMORY;
}

byte *pVarBuf;
hr = SafeArrayAccessData(psaData, (void HUGEP**)&pVarBuf);
if (FAILED(hr))
{
        SafeArrayDestroy(psaData);
        return hr;
}

//copy stored data to safearray
memcpy(pVarBuf, pBuf, nCount);
SafeArrayUnaccessData(psaData);

varBuf.vt = vt | VT_ARRAY;
varBuf.parray = psaData;

//objects to fill
CComPtr<IWbemClassObject> spOutParams;
CComPtr<IWbemClassObject> spObject;
hr = GetOutParams(&spOutParams, &spObject, pCtx); //fill objects

//put variant to the WMI out-object
//these methods are memory and time consuming.
hr = pObj->Put(L"arrBuf", 0, const_cast<CComVariant*>(&var), 0);
hr = pResponseHandler->Indicate(1, &spOutParams.p);

Well, I do understand that filling the variant this way is not
optimal, but nontheless if we look at my logs:
1) we read and fill CComVariant with data from file rather fast, for
example we read 300Mb file in memory and fill the variant whithin 30
seconds
2) two methods are the most time-consuming, they are
pObj->Put(L"arrBuf", 0, const_cast<CComVariant*>(&var), 0);
and
pResponseHandler->Indicate(1, &spOutParams.p);

These two methods lasts 30 minutes, so the perfomance problem really
exists. Actually there are two processes that are most CPU consuming -
my provider process and svchost.exe which is Windows Management
Instrumentation service. My provider consumes about 1,5 Gb of memory
and about 50% CPU time while passing this data to client. And svchost
also consumes large amount of memory and about 50% of CPU time while
passing data to client. The client-side is OK, there was not any
network drawback while this experiment.
Also I should mention that I can pass rather small files very fast, so
I pass 13Mb file in 1 second and I pass 25Mb file in 3 seconds. So I
think the main drawback is in memory consuming.

My questions are:
1) Why did the WMI process consumes CPU while passing data to client?
2) Why did my provider consumes about 1.5 Gb of memory while passing
data to client? It doesn't use Virtual memory, it tryes to allocate
all available memory instead.
3) What is the best way to pass large data to client.

Thank you in advance,
   Stanislav Efremov.

Relevant Pages

  • Re: Perfomance problem while passing large data volumes via WMI
    ... There are inherent problems passing too much data that way. ... > the file is about 300 Mb) and we want to pass it to the client side. ... > //these methods are memory and time consuming. ... My provider consumes about 1,5 Gb of memory ...
    (microsoft.public.win32.programmer.wmi)
  • CSRSS.exe
    ... I have a client and csrss.exe consumes more and more ... memory until XP says he's low on VM. ... He had netsky on his ...
    (microsoft.public.windowsxp.perform_maintain)
  • Re: Error Using Remoting, System.OutOfMemoryException anyone ever seen this?
    ... it to transfer the DataSet to the client like I wanted. ... >> As Integer, ByVal pnUserID As Integer, ByVal piView As Short) As DataSet ... >> As for memory use, the client which is hung is not peaked out of memory, ... >> it is running 1% cpu usage and barely any memory used. ...
    (microsoft.public.dotnet.framework.remoting)
  • Re: Memory usage and large number of pages
    ... The problem is the overhead of compiling these pages is huge - and because ... have also found that for each file that is compiled the memory increases. ... >> directory on the web server. ... >> returns the result of this page back to the client. ...
    (microsoft.public.dotnet.framework.aspnet)
  • Re: x86 instructions guide
    ... one thing that is interesting is how the client ... of the data even if it resides in client side memory as array, ... --> The driver cannot return from the draw primitive command until it ... efficient but it is phased out; OpenGL ES for example has abandoned DL ...
    (alt.lang.asm)