RE: Excessive Frame Drops in pimg application during video capture

It does not appear that there is a registry key to change pimg to use
something other than the DirectShow video renderer. I wasn't able to find one
looking through the application code, but I haven't heard back from the
application owners.

The error you're seeing has an ASCII/unicode bug, "OK" and "wrong" are ASCII
strings but it's a unicode debug output. It should say (input OK output
wrong). I suspect this error is because the filtergraph mapper is trying to
insert your filter in the middle between the vcap and the renderer, but your
filter doesn't have an output since it is a renderer. That's just a guess
based on the debug output and the fact that the video renderer is explicitly
put into the graph.

If you support RGB overlays in DDraw, the video renderer should dynamically
switch over at run time. RGB overlays are ideal because the video renderer
can switch between GDI and DDraw seamlessly without renegotiating the format.
DDraw requires top down bitmaps, so make sure your biHeight is negative. If
the height is positive then it'll never switch over to DDraw. YV12 is
supported in the color converter, so i don't know why the RenderStream is
failing. There must be something incompatible in the media type.

The exception in pimg is probably the watson dump that the application team
set up to occur any time there's a problem building the graph. The
application throws an unhandled exception to trigger watson. Once you get the
RenderStream to work the exception will disappear.

Gary Daniels
Windows Mobile Multimedia Test


"ABatWork" wrote:

Hi Gary,

We already have our video renderer, so writing a coustomized intermediate
filter won't be a good idea at this point. Also I am able to get more than
double performance with our renderer via our Dshow application.

Also Can you please provide me any update regarding whether pimg
application have any registry key to change its default MS renderer to any
coutomized video renderer.
Infact We tried even connecting renderer, with our major GUID in camera
driver for its preview output format instead of MediaType_Video, and we got
following debug information from quartz prints:

Types match: Req (2cfc3df4 e436eb7b) Found(2cfc3df4 e436eb7b
RegEnumFilterInfo: AB Video Renderer not wanted (input O output wog)


Where "AB Video Renderer" is our video renderer name. Can you please comment
on what is the meaning of ""(input O output wog)"".


Apart from above question, I am passing RGB data to preview pin from my
camera driver. To switch MS video renderer to Ddraw for using hw overlay,
should i pass YV12 data from our camera driver's preview pin.
Infact I tried even passing YV12 data from camera driver to its preview
pin. But got a exception in pimg application. While my Dshow based
application failed in RenderStream function. What can be possible cause for
this issue.

Thanks in advance.
Ab

"fdbck" wrote:

I'm not sure whether or not the pimg application has a registry key to
control what video renderer it uses. I'll have to follow up on that.

Ideally you would use the DirectShow provided video renderer rather than
making a new one. If your overlay is available through DirectDraw then the
video renderer will try to use it. There is one caveat with that, the video
renderer requires an RGB fallback in the event the overlay in use cannot do
color keying to deal with complex clipping surfaces... To do this the VR
always negotiates an RGB format at graph build time, but then dynamically
switches over to DDraw at run time. To get your custom format to negotiate to
the VR you'll need to write a custom filter that can take in your custom
format and convert it to RGB. When the dynamic format changes at run time the
custom filter can change into a pass through filter with no impact on
performance. It might sound like a lot of overhead, but it's probably easier
than writing a video renderer. For the standard YUV formats the color
converter can already handle them and do the pass through switch to have the
best perf automatically. If your custom format could be adjusted to be YV12,
YV16, UYVY, YVYU, YUY2, YVU9, Y41P, or CLJR then you could use the color
convert to do this. We're working on fixing this requirement in the future.

For the ptype of RenderStream, that's for the capture graph builder to
identify the stream and not to specify the color format to render with.
Assuming your video capture filter has only 1 preview pin, then
RenderStream(&PIN_CATEGORY_PREVIEW, NULL, pVideoCaptureFilter, NULL,
pVidRenderer) is functionally identical to what you're doing. I say this just
in case you believed setting the pType could change the color format. To set
the color format of the preview output pin use IAMStreamConfig.

It sounds as though pVidRenderer points to your custom video enderer filter?
It sounds like your filter is rejecting something with the video format
coming from the video capture filter.

If you're interested, there's some other spots you might be able to optimize
perf. If you can have your camera driver use client unlimited buffering
rather than driver limited buffering, then the DirectShow buffering filter
will buffer the images from the driver without doing a memcopy and new
buffers will be allocated and given to the driver whenever needed. If your
hardware can do this it's a great thing to do, if your hardware cannot then
the buffering filter will do the memcopy instead of your driver. If you have
a high resolution still output and hardware jpeg encoding there's two
options, you can encode to jpeg inside of your driver or you can expose the
hardware through the GDI+ imaging library. If you can do both JPEG encode and
decode in hardware through the GDI+ imaging library then there's some great
potential for still image perf improvements.

I'll get back to you on whether or not pimg can use your renderer.

Gary Daniels
Windows Mobile Multimedia Test

"ABatWork" wrote:

Hi Gary,

Thanks for awesome information. It proven to be very useful. And yes It was
preview only which was causing problem.
I tried video capture using Dshow based application which render only
capture stream and not preview, and 6 FPS performance for high resolution
changed to 30 FPS.

Now to achieve performance, we left with only one option to change pimg
video renderer to our hw overlay renderer. First I tried first to implement
this our Dhsow based application which uses MS Video Capture filter.

My camera driver supports Customized Major GUID(Lets name it as
MediaType_ABVideo). Same is supported inside our renderer. Here is the
function where I am getting failure:

RenderStream( &PIN_CATEGORY_PREVIEW,
&MEDIATYPE_AbVideo,
pVideoCaptureFilter,
NULL, pVidRenderer);

Where pVidRender is our Video Renderer.

I turned on messages of quartz.dll and found following just before render
stream:

EnumRegFilters returning 0 filters
ConnectVia: failed level 1
Connect Ended hr=80040217

Which means VFW_E_CANNOT_CONNECT "No combination of intermediate filters
could be found to make the connection"

Can you please comment your views on this.

Also Later on If I try to use pimg application with our renderer, Will it be
possible?

Thanks
Amit



We have coustomized major type
"fdbck" wrote:

I suspect it's the buffering filter that's dropping rather than the vcap
filter. The vcap filter doesn't drop frames, that's handled by the camera
driver and the buffering filter.

What has happened in the past is the CPU is unable to encode video fast
enough to keep up with the frame rate that it's recieving the video, so the
buffering filter buffers up the unencoded video until it runs out of ram.
Once RAM is full the filter dropps the oldest frame in it's queue that hasn't
been encoded and replaces it with the new sample it received. There's no
logic to evenly distribute the dropped frames throughout the video file. Once
the video capture is stopped the buffered video is encoded, which results in
a delay from the time when you stop the capture to when the preview resumes
and you're able to start the next capture. This algorithm works well for
transient conditions, but not sustained problems with encoding. In extreme
cases the resulting video file will look like a slide show with video frames
only changing every few seconds until just before the end of the video, at
which point the video will run smoothly for however much video was buffered
up at the time the capture was stopped.

If this is the behavior you're seeing, there's a few things that can be done
to improve the quality of the video. Usually the problem is because the
preview is using 100% of the CPU and leaving nothing for encoding video. The
preview is using CPU because there may not be overlay hardware in the
graphics chip, so it's rendered in software. The biggest thing you can do is
if you have display hardware figure out what you need to do with the preview
formats exposed by the camera driver to use hardware. Reduce the resolution
of the preview, fewer pixels requires less CPU to move. Along with reducing
the resolution of the preview, try to set a preview format that doesn't
require any stretching or shrinking to be rendered. Unless the stretching or
shrinking is done is hardware, it's extrememly CPU intensive. Reduce the
frame rate of the preview, slower preview reduces CPU utilization. Once you
look at how the preview window is rendererd and how much CPU it's using, try
using the monte carlo profiling tools to see where the CPU is going.

Gary Daniels
Windows Mobile Multimedia Test

"ABatWork" wrote:

I am working on ARMV4I device with WM6 OS using pimg application. My stream
interface camera driver followes 3 pin architecture and use "client limited"
model drived from MSFT provided NULLCAM driver.
For around 4 mins of wmv video capture i am getting around 30KB video
seems to have only 9-10 frames. I am passing RGB data on both Preview and
capture pin. Capture resolution is QCIF(176x144).

Is there any way, by which i can tell Video Capture filter to not drop even
a single frames anyhow ?
Because of frames drops of video capture filter, we are getting very bad
quality and performance. Is there any way we can overcome this problem?

Thanks
ab
.

Relevant Pages
Loading