Click here to Skip to main content
Click here to Skip to main content

Capture Sample with DirectX and .NET

, 10 Aug 2003
Rate this:
Please Sign up or sign in to vote.
A solution to capture video and frames simultaneously

Introduction

This article tries to explain how to capture video and frames simultaneously. It is based on SampleGrabber program done by NETMaster and DirectX.Capture by Brian Low. The root of all of this was the amazing work of NETMaster, DirectShow.NET, SampleGrabber is just a sample application. I said amazing because I'm close to understand completely the other projects or classes, but DirectShow.NET is far of my knowledge.

Before begin with the explanation, I have to say that I am not an expert programmer, this work is one of the first serious programs that I have ever typed, so probably someone could find a best solution to do the same.

At this point you could probably guess that my mother tongue is not English, it is Spanish, sorry for my grammatical mistakes. I have write the code in Visual Studio .NET Spanish version, therefore the automatically generated comments are in Spanish, but my comments are in English.

The VB code

First of all, to run the demo program you will need to have installed NET Framework 1.1. check windows update if you don't have it.

Here I want to explain how the project works in VB, I will explain the sampleGrabber filter in the next section. The main project is CapSample, it has three forms; MW, AddCam and CW, and a module ModCap. Let's begin with the description.

MW stands for "Main Window", it's the initial form. When it is created, it create a new form, AddCam, that is responsible to the selection of an available camera. If OK button is clicked, AddCam will connect to the camera. The connection is done by using the capture class of DirectX.Capture, as I said before in the next section I will discuss my modifications to this class.

Once you have the camera connected (AddCam finished, if you click Cancel an error will happen), MW continues its initialization and set up the preview board (I am not sure about this word, in Spanish this control it's call "Panel"), then it add a handler to the FrameCaptureComplete event of the capture class and initialize the counters. Finishing with the creation of CW (It stands for "Configuration Window"), which is responsible of the configuration of the camera. By clicking OK ConfParamCam() will set the selected parameters, then it will set the capture directory (If it doesn't exist an error will happen), and capture a frame and render the preview stream by CaptureInformation.CaptureInfo.CaptureFrame().

Once you have configured the camera, you are able to capture frames by clicking the Frame button and start the video capture by clicking the Start button. When you click Stop, it stops the capture but it calls ConfParamCam() to set the previous selected parameters and PrepareCam() to increment the name of the captured file. And that is all I have to say about the VB code. Let's begin with the capture class.

Modifications in capture class of DirectX.Capture

My capture class it is a mixed class between capture class of Brian Low and SampleGrabberNET program of NETMaster.

In createGraph(), I have added the next code

AMMediaType media = new AMMediaType();
media.majorType= MediaType.Video;
media.subType = MediaSubType.RGB24;
media.formatType = FormatType.VideoInfo;
hr = sampGrabber.SetMediaType( media );
if( hr<0 ) Marshal.ThrowExceptionForHR( hr );

and

mediaEvt    = (IMediaEventEx)    graphBuilder;
baseGrabFlt = (IBaseFilter) sampGrabber;
hr = graphBuilder.AddFilter( baseGrabFlt, "DS.NET Grabber" );
if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

The first block is to can use the filter and the second bock is the filter itself. The important things come with renderGraph(), before show you the code I have to say that all WDM devices have two PINs, capture and preview, sometimes only one (capture) but the smart tee filter convert the capture PIN into capture and preview, if it is needed Intelligent Connect will add it for us. So, I will use the capture PIN to capture video to a file and the preview PIN to capture frames. RenderGraph() is divided in two ifs, one to prepare all the filters to capture video to a file, it is controlled by wantCaptureRendered, and the other it is to render the preview stream, I have added the variable renderStream to avoid the normal functioning of the original class of Brian Low, because when baseGrabFlt is set-up I don´t know how to configure the cameras and when a capture was stopped, camera lost its configuration parameters, and I stop render until this parameters are updated with renderStream.

The first if is equal to the first if in Brian Low class, and the second looks like this.

// Render preview stream and launch the baseGrabFlt to capture frames
// ==================================================================
if ( wantPreviewRendered && renderStream && !isPreviewRendered )
{
    /// Render preview (video.PinPreview -> baseGrabFlt -> renderer)
    /// At this point intelligent connect is used, because my 
    /// webcams don't have a preview pin and
    /// a capture pin, so Smart Tee filter will be used. 
    /// I have tested it using GraphEdit.
    /// I can type hr = captureGraphBuilder.RenderStream( ref cat, 
    /// ref med, videoDeviceFilter, null, baseGrabFlt); 
    /// because baseGrabFlt is a transform filter, 
    /// like videoCompressorFilter.
    
    cat = PinCategory.Preview;
    med = MediaType.Video;
    hr = captureGraphBuilder.RenderStream( ref cat, ref med, 
        videoDeviceFilter, baseGrabFlt, null ); 
    if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

    // Get the IVideoWindow interface
    videoWindow = (IVideoWindow) graphBuilder;

    // Set the video window to be a child of the main window
    hr = videoWindow.put_Owner( previewWindow.Handle );
    if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

    // Set video window style
    hr = videoWindow.put_WindowStyle( WS_CHILD | WS_CLIPCHILDREN 
       | WS_CLIPSIBLINGS);
    if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

    // Position video window in client rect of owner window
    previewWindow.Resize += new EventHandler( onPreviewWindowResize );
    onPreviewWindowResize( this, null );

    // Make the video window visible, now that it is properly positioned
    hr = videoWindow.put_Visible( DsHlp.OATRUE );
    if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

    hr = mediaEvt.SetNotifyWindow( this.Handle, 
         WM_GRAPHNOTIFY, IntPtr.Zero );
    if( hr < 0 )
        Marshal.ThrowExceptionForHR( hr );

    isPreviewRendered = true;
    didSomething = true;

    // Begin Configuration of SampGrabber    <<<<<<-----------------
                
    AMMediaType media = new AMMediaType();
    hr = sampGrabber.GetConnectedMediaType( media );
    if( hr < 0 )
        Marshal.ThrowExceptionForHR( hr );
    if( (media.formatType != FormatType.VideoInfo) || 
              (media.formatPtr == IntPtr.Zero) )
        throw new NotSupportedException( "Unknown Grabber Media Format" ); 

    videoInfoHeader = (VideoInfoHeader) Marshal.PtrToStructure( 
               media.formatPtr, typeof(VideoInfoHeader) );
    Marshal.FreeCoTaskMem( media.formatPtr ); 
    media.formatPtr = IntPtr.Zero;

    hr = sampGrabber.SetBufferSamples( false );
    if( hr == 0 )
        hr = sampGrabber.SetOneShot( false );
    if( hr == 0 )
        hr = sampGrabber.SetCallback( null, 0 );
    if( hr < 0 )
        Marshal.ThrowExceptionForHR( hr );    
                
    // Finish Configuration of SampGrabber    <<<<<<----------------
}
            
if ( didSomething )
    graphState = GraphState.Rendered;

In this four lines is when the baseGrabFlt is set-up, note that I use baseGrabFlt as a transform filter.

cat = PinCategory.Preview;
med = MediaType.Video;
hr = captureGraphBuilder.RenderStream( ref cat, ref med, 
    videoDeviceFilter, baseGrabFlt, null ); 
if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

About the configuration of sampleGrabber, I only comment next lines.

hr = sampGrabber.SetBufferSamples( false );
if( hr == 0 )
    hr = sampGrabber.SetOneShot( false );
if( hr == 0 )
    hr = sampGrabber.SetCallback( null, 0 );
if( hr < 0 )
    Marshal.ThrowExceptionForHR( hr );

SampleGrabber has three different methods to capture a frame, I choose SetCallBack because I found it more useful, sampleGrabber will capture a frame when a call to a function was done.

We need more things to complete the capture of a frame, and it is done by the next block of code.

void OnCaptureDone()
{
    int hr;
    if( sampGrabber == null )
        return;
    hr = sampGrabber.SetCallback( null, 0 );

    int w = videoInfoHeader.BmiHeader.Width;
    int h = videoInfoHeader.BmiHeader.Height;
    if( ((w & 0x03) != 0) || (w < 32) || (w > 4096) 
            || (h < 32) || (h > 4096) )
        return;
    int stride = w * 3;

    GCHandle handle = GCHandle.Alloc( savedArray, GCHandleType.Pinned );
    int scan0 = (int) handle.AddrOfPinnedObject();
    scan0 += (h - 1) * stride;
    Bitmap b = new Bitmap( w, h, -stride, 
        PixelFormat.Format24bppRgb, (IntPtr) scan0 );
    handle.Free();
    savedArray = null;
    ImageCaptured.Image = b;

    //Launch the event

    FrameCaptureComplete(ImageCaptured);

    return;
        
}

protected override void WndProc( ref Message m )
{
    if( m.Msg == WM_GRAPHNOTIFY )
    {
        if( mediaEvt != null )
            OnGraphNotify();
                return;
    }
    base.WndProc( ref m );
}

// graph event (WM_GRAPHNOTIFY) handler
void OnGraphNotify()
{
    DsEvCode    code;
    int p1, p2, hr = 0;
    do
    {
        hr = mediaEvt.GetEvent( out code, out p1, out p2, 0 );
        if( hr < 0 )
            break;
        hr = mediaEvt.FreeEventParams( code, p1, p2 );
    }
    while( hr == 0 );
}

int ISampleGrabberCB.SampleCB( double SampleTime, 
    IMediaSample pSample )
{
    return 0;
}
        
int ISampleGrabberCB.BufferCB(double SampleTime, 
    IntPtr pBuffer, int BufferLen )
{
    if( captured || (savedArray == null) )
    {
        return 0;
    }
    captured = true;
    bufferedSize = BufferLen;
    if( (pBuffer != IntPtr.Zero) && (BufferLen > 1000)
            && (BufferLen <= savedArray.Length) )
        Marshal.Copy( pBuffer, savedArray, 0, BufferLen );
    try
    {
        this.BeginInvoke( new CaptureDone( this.OnCaptureDone ) );
    }
    catch (ThreadInterruptedException e)
    {
        MessageBox.Show(e.Message);
    }
    catch (Exception we)
    {
        MessageBox.Show(we.Message);
    }
    return 0;
}

Beginning at the end, ISampleGraberCB.BufferCB(...) is a function that is filling a buffer, and when the buffer is full it have the frame, and call OnCaptureDone() that is the function that create the image. When the image was built it issues an event to send it to the receiver of that event.

ISampleGrabberCB.SampleCB(...) have to be defined instead it was not be used. For me the other two functions are black magic, I can guess what they are doing but I don't understand it.

And finally the method invoked to capture a frame.

public void CaptureFrame()
{
    int hr;

    if(firstFrame)
    {
        assertStopped();

        // Re-render the graph (if necessary)
        renderStream = true;
        renderGraph();

        // Start the filter graph: begin capturing
        hr = mediaControl.Run();
        if ( hr != 0 ) Marshal.ThrowExceptionForHR( hr ); 

        firstFrame = false;
    }

    captured = false;

    if(savedArray == null )
    {
        int size = videoInfoHeader.BmiHeader.ImageSize;
        if( (size<1000) || (size > 16000000) )
            return;
        savedArray = new byte[ size + 64000];
    }
    hr = sampGrabber.SetCallback( this, 1 );
}

firstFrame is to avoid execute more than once next line.

hr = mediaControl.Run()

I think the description is completed. I hope it will be useful to you Smile | :)

System Requirements

I have been working with an AMDXP@1500, 512Mb RAM and WIN2k. I have installed DirectX 9.0 but, I think, it could work with DirectX 8.1.

As cameras I have been using a Philips ToUcam Pro and a Creative Video Blaster Web Cam 5 without problems. I have also installed a Pinacle PCTV Pro and it doesn't work capture a video to a file. I don't know why, but at the moment I don't have enough time to search the source of the problem.

Feedback and Improvements

I will try to keep track of the forum, if you need help I will be glad to give you a hand. I think we have to share code to improve the performance of our programs, and to help people do new things, for example, if I have not read the code of Brian Low I would never have written this code. Thank you!

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

About the Author

Blas5
Web Developer
Spain Spain
No Biography provided

Comments and Discussions

 
QuestionHow do I rotate a camera view got with DirectX in Vb.Net ? PinmemberOlayèmi Ouabi12-May-14 23:10 
Questionsir why is this code PinmemberMember 1021801526-Feb-14 16:26 
QuestionException Pinmembertweber201225-Feb-14 10:08 
QuestionIs it possible to Rotate live view 90 degree to use webcam in portrait view? Pinmembersakda12313-Feb-13 22:00 
QuestionException has been thrown by the target of an invocation PinmemberSuman Lata Gupta22-Jan-13 19:54 
GeneralMy vote of 5 PinmembermanojIITKGP21-Jan-13 8:38 
QuestionDraw rectangle Pinmembermanganat17-Jun-12 23:03 
GeneralMy vote of 5 Pinmembermorteza fakoor9-Dec-11 20:16 
Generalthe video displayed mirrored Pinmemberli5715-May-11 19:56 
GeneralMy vote of 5 Pinmemberli5715-May-11 19:51 
GeneralMy vote of 1 Pinmemberjohnson37511-Feb-11 13:46 
GeneralRe: My vote of 1 PinmemberNasenbaaer14-Mar-11 6:06 
Generalpause video Pinmemberniks231418-Nov-10 0:29 
Generalmouse events PinmemberDafnwa31-Oct-10 2:09 
GeneralRe: mouse events Pinmember959AVA13-May-12 21:36 
Questionwhy check for (w & 0x03) != 0) in OnCaptureDone() ? PinmemberRiversideWalruses13-Oct-10 12:50 
Generalsetup project for DirectShow applications PinmemberJoJo-jOjO19-Jun-10 2:17 
GeneralMultiple Cams causing Error Pinmemberrobin2421-Jan-10 4:46 
GeneralMy vote of 2 PinmemberHyperByte1111-Dec-09 0:05 
GeneralRe: My vote of 2 PinmembermanojIITKGP21-Jan-13 8:45 
Questionmultiple webcam - one avi Pinmembersolyom6-Aug-09 4:23 
Generalcrashes when choose comressor Pinmemberkatran.ek10-Mar-09 23:39 
QuestionCapture inherit's Form? Pinmemberkatran.ek10-Mar-09 23:29 
GeneralManage colors during Capturing Pinmemberanki12310-Mar-09 1:35 
GeneralRequire help urgently Pinmemberairness13200228-May-08 16:58 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140709.1 | Last Updated 11 Aug 2003
Article Copyright 2003 by Blas5
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid