Click here to Skip to main content
Click here to Skip to main content

Video Preview and Frames Capture to Memory with SampleGrabber in Buffered Mode.

, 26 Oct 2007
Rate this:
Please Sign up or sign in to vote.
This article demonstrates video preview and frames capture to memory from external video devices using ISampleGrabber interface in buffered mode.
Screenshot - vidcap.jpg

Introduction

I looked around for a project capturing incoming video from external video devices to memory buffer for image processing applications and could not find the one I needed. The existing articles on The Code Project, An Easy Video Processing Framework by Grabbing Frames as Bitmaps using DirectShow, CAviCap and CFrameGrabber - Wrappers for AVICap Window, Real-time Video Image Processing / Frame Grabber using a Minimalistic Approach, Simultaneous Previewing & Video Capture using DirectShow do not provide the preferred buffering mode for capturing raw data from the stream. Some of them were tested only on WinXP and Windows 2000, pretty outdated, the latest one goes back to 1999 or captures samples from video files and so on.

I have Vista on my computer and installed Windows SDK for Vista and DirectX SDK 2007. Now Direct Show is a part of Windows SDK and if you want to use ISampleGrabber for raw data capture, you need to install DirectX SDK too. I developed a simple MFC application supporting enumeration of existing video devices, preview of the video stream, capture raw image data in buffered mode with the desired frame rate, display it to the window with GDI+, taking snap shots to JPEG files. With that application as a skeleton, you can easily start with image/video processing tasks (video codecs development, motion estimation, edge detection). I use that one in my face detection program I'm going to post later.

Background

You need to have an understanding in Direct Show programming. Have a look at the above mentioned articles or Windows Vista SDK help. Understanding of COM technology is desired - have a look at the article, Introduction to COM - What It Is and How to Use It.

Using the Code

Click the enum button that will enumerate available video devices and select one. Specify the desired raw image data capture interval in milliseconds (default one is 1000, capture data every second), click the Run button. The left top small static will preview the video output and the center large one will show captured raw images. To take a snapshot, left mouse double click the center capture static window and the captured image will be saved on disk to the same directory with Snapshot X.jpg name, where X is the number of the image.

For code reuse, I provided Sample Grabber and Video Capture implementation in separate files. I used the Video Capture routines from the SDK example (SDK\Samples\Multimedia\DirectShow\Capture\PlayCap). I wrapped them in functions easy to use with an external application, and also added video devices enumeration:

  • void vcGetCaptureDevices(CComboBox& adaptersBox);
  • HRESULT vcCaptureVideo(HWND msgWindow, HWND prvWindow, unsigned int devIndex = 1);
  • void vcStopCaptureVideo();

vcGetCaptureDevices() enumerate video devices and add them to ComboBox. vcCaptureVideo() starts video preview to prvWindow with msgWindow application that will handle Filter Graph notification messages. devIndex is the index of the video device to capture data from. vcStopCaptureVideo stops capturing video data.
The Sample Grabber added to Filter Graph in vcCaptureVideo functions this way:

//...

hr = sgAddSampleGrabber(g_pGraph);
if (FAILED(hr)) {
        Msg(TEXT("Couldn't add the SampleGrabber filter to the graph!  hr=0x%x"), hr);
        return hr;
}
hr = sgSetSampleGrabberMediaType();
if (FAILED(hr)) {
        Msg(TEXT("Couldn't set the SampleGrabber media type!  hr=0x%x"), hr);
        return hr;
}
IBaseFilter* pGrabber = sgGetSampleGrabber();

// Render the preview pin on the video capture filter
// Use this instead of g_pGraph->RenderFile
hr = g_pCapture->RenderStream(&PIN_CATEGORY_PREVIEW, &MEDIATYPE_Video,
                              pSrcFilter, pGrabber/*NULL*/, NULL);
if (FAILED(hr)) {
        Msg(TEXT("Couldn't render the video capture stream.  hr=0x%x\r\n")
            TEXT("The capture device may already be in use by another application.
                  \r\n\r\n")
            TEXT("The sample will now close."), hr);
        pSrcFilter->Release();
        return hr;
}

hr = sgGetSampleGrabberMediaType();

//...

The corresponding sg* functions are located in samplegrab.cpp:

HRESULT sgAddSampleGrabber(IGraphBuilder *pGraph)
{
        // Create the Sample Grabber.
        HRESULT hr = CoCreateInstance(CLSID_SampleGrabber, NULL, CLSCTX_INPROC_SERVER,
                                      IID_IBaseFilter, (void**) & pGrabberFilter);
        if (FAILED(hr)) {
                return hr;
        }
        hr = pGraph->AddFilter(pGrabberFilter, L"Sample Grabber");
        if (FAILED(hr)) {
                return hr;
        }

        pGrabberFilter->QueryInterface(IID_ISampleGrabber, (void**)&pGrabber);
        return hr;
}

HRESULT sgSetSampleGrabberMediaType()
{
        AM_MEDIA_TYPE mt;
        ZeroMemory(&mt, sizeof(AM_MEDIA_TYPE));
        mt.majortype = MEDIATYPE_Video;
        mt.subtype = MEDIASUBTYPE_RGB24;
        HRESULT hr = pGrabber->SetMediaType(&mt);
        if (FAILED(hr)) {
                return hr;
        }
        //Do not stop the graph after one shot
        hr = pGrabber->SetOneShot(FALSE);
        //Use buffered mode
        hr = pGrabber->SetBufferSamples(TRUE);
        return hr;
}

IBaseFilter* sgGetSampleGrabber()
{
        return pGrabberFilter;
}

HRESULT sgGetSampleGrabberMediaType()
{
        AM_MEDIA_TYPE mt;
        HRESULT hr = pGrabber->GetConnectedMediaType(&mt);
        if (FAILED(hr)) {
                return hr;
        }

        VIDEOINFOHEADER *pVih = (VIDEOINFOHEADER *)mt.pbFormat;
        gChannels = pVih->bmiHeader.biBitCount / 8;
        gWidth = pVih->bmiHeader.biWidth;
        gHeight = pVih->bmiHeader.biHeight;

        sgFreeMediaType(mt);
        return hr;
}

The initialization of video preview and capture is implemented in the Run button click routine CVidCapDlg::OnBnClickedRunButton(), where the timer is created to query captured raw image data by Sample Grabber filter which is the preferred way of using it as described in SDK help under the topic "Using the Sample Grabber".

To grab the raw image data, use the following functions from samplegrab.h:

  • unsigned char* sgGrabData();
  • Gdiplus::Bitmap* sgGetBitmap();
  • long sgGetBufferSize();

The first one returns a pointer to the buffer with raw image data, or NULL if the Sample Grabber is not ready yet. Note that Sample Grabber filter fills that buffer with the image turned upside down, as a bottom up bitmap. sgGetBitmap() returns GDI+ bitmap filled with raw image data after sgGrabData() call or NULL if the data was not captured. I use sgFlipUpDown() function to flip the raw image before copying it to the Bitmap. You should use those functions in that order, first call sgGrabData() to get the raw image and if you want Bitmap object to draw it into the window, call sgGetBitmap() later. To get the length of the buffer with raw image, call sgGetBufferSize().

The code for the grabbing is shown below:

unsigned char* sgGrabData()
{
        HRESULT hr;

        if (pGrabber == 0)
                return 0;

        long Size = 0;
        hr = pGrabber->GetCurrentBuffer(&Size, NULL);
        if (FAILED(hr))
                return 0;
        else if (Size != pBufferSize) {
                pBufferSize = Size;
                if (pBuffer != 0)
                        delete[] pBuffer;
                pBuffer = new unsigned char[pBufferSize];
        }

        hr = pGrabber->GetCurrentBuffer(&pBufferSize, (long*)pBuffer);
        if (FAILED(hr))
                return 0;
        else {
                sgFlipUpDown(pBuffer);
                return pBuffer;
        }
}

To determine the image size, call these functions:

  • unsigned int sgGetDataWidth();
  • unsigned int sgGetDataHeight();
  • unsigned int sgGetDataChannels();

License

This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)

Share

About the Author

Chesnokov Yuriy
Engineer
Russian Federation Russian Federation
No Biography provided

Comments and Discussions

 
QuestionTv Tuner RAW Data Capture [modified] PinmemberMember 76604078-Jun-14 9:38 
QuestionError 0x8004005 "The capture device may already be in use by another application" PinmemberAdrià Gil26-Feb-14 0:34 
GeneralThanks a loooooooooooooot Pinmembershaima'26-Jan-14 2:11 
QuestionHow to get multiple live video feed for multiple web cam using DirectShow PinmemberMayur S778920-Oct-13 22:47 
QuestionGives error for 1>d:\camera capture\vidcap_src\src\stdafx.h(64): fatal error C1083: Cannot open include file: 'qedit.h': No such file or directory PinmemberMayur S778920-Oct-13 22:42 
GeneralMy vote of 5 PinmemberMember 964686714-Dec-12 19:09 
QuestionGetting error!!!!!!!!!!!! How to fix that ????????????????????????? Pinmemberios1981-Oct-12 21:45 
AnswerRe: Getting error!!!!!!!!!!!! How to fix that ????????????????????????? PinmemberChesnokov Yuriy3-Oct-12 21:37 
GeneralThanks a lot for this great posting.... Pinmemberstarmoon77112-Feb-12 1:15 
QuestionPlay with delay PinmemberMember 828422620-Jan-12 21:19 
QuestionExposure setting Pinmemberche ho kim5-Dec-11 0:38 
QuestionSample Grabber Callback Pinmembervanselm21-Nov-11 0:33 
GeneralA bug in the Application Pinmemberuvik14-Oct-10 1:16 
GeneralDear Yuriy, Pinmemberuvik1-Oct-10 0:17 
GeneralRe: Dear Yuriy, Pinmemberuvik11-Oct-10 23:51 
QuestionSetting Frame size?? PinmemberHaniSalehi15-Sep-10 18:30 
GeneralRecord to file PinmemberLiz Brogan25-Aug-10 5:33 
GeneralRe: Record to file PinmemberChesnokov Yuriy26-Aug-10 9:39 
GeneralRe: Record to file PinmemberLiz Brogan30-Aug-10 21:33 
GeneralDevice Selection Problem PinmemberBinni shah3-Jul-10 1:41 
Generalopencv Pinmemberbumbacul9-Jun-10 10:56 
GeneralDecent article... Code won't compile. PinmemberDestiny77722-Jan-10 15:27 
AnswerRe: Decent article... Code won't compile. PinmemberChesnokov Yuriy23-Jan-10 20:45 
GeneralRe: Decent article... Code won't compile. Pinmembermerano6-Jun-10 1:56 
GeneralRe: Decent article... compile fixed... PinmemberDestiny7777-Nov-11 15:08 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140827.1 | Last Updated 26 Oct 2007
Article Copyright 2007 by Chesnokov Yuriy
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid