Click here to Skip to main content
11,415,711 members (88,665 online)
Click here to Skip to main content

Webcamera, Multithreading and VFW

, 15 Feb 2008 CPOL
Rate this:
Please Sign up or sign in to vote.
An article on webcamera frame-grabbing in a multi-thread environment
Screenshot - vfwwebcam1.jpg

Introduction

There are several ways to grab and process webcamera images: WIA, DirectShow, VFW... There are lots of C# VFW examples on the Internet and most of them use .NET clipboard to transfer each frame's data from buffer to Bitmap-recognizable object. Unfortunately, this makes multithreading unavailable and reduces FPS (frames per second). The native Win32 clipboard and multithreading solve the speed problem, but I thought that it wasn't the most elegant solution and there should be another way to get frames from Avicap. I have referred to MSDN (see VFW link above) and discovered that function callback was available. This article explains, step-by-step, how to capture frames using avicap32.dll (VFW) in a multi-thread environment.

The Idea

This is an approach that you can find in lots of examples from the Web:
  1. Create a capture window.
  2. Connect the capture window to the device.
  3. Set the video format (height and width in pixels).
  4. Capture the frame to temporary unreachable buffer.
  5. Copy the contents of the video frame buffer and associated palette to the clipboard.
  6. Get the frame from the clipboard, converting data to RGB Bitmap.
  7. Process Bitmap. Go to Step 4.
This is an approach I worked with:
  1. Create a capture window.
  2. Connect the capture window to the device.
  3. Set the video format (height and width in pixels, bits per frame).
  4. Capture the frame to temporary unreachable buffer.
  5. Make a callback (make buffer data available in VIDEOHDR structure).
  6. Get the frame data, converting data to RGB Bitmap.
  7. Process Bitmap. Go to Step 4.

API

[DllImport("avicap32.dll", EntryPoint = "capCreateCaptureWindow")]
static extern int capCreateCaptureWindow(string lpszWindowName,
    int dwStyle, int X, int Y,
    int nWidth, int nHeight, int hwndParent, int nID);

The capCreateCaptureWindow function creates a new window for video stream capturing and returns its handle. This function is called in step 1.

[DllImport("user32", EntryPoint = "SendMessage")]
static extern bool SendMessage(int hWnd, uint wMsg, int wParam, int lParam);

The SendMessage function sends the specified message to a window or windows. It calls the window procedure for the specified window and does not return until the window procedure has processed the message. Note that both wParam and lParam specify additional message-specific information, i.e. numbers, pointers to structures, buffers. SendMessage is overloaded in this project with different lParam types. It is called in steps 2, 3, 4 and 5.

[DllImport("avicap32.dll")]
static extern bool capGetDriverDescription(intwDriverIndex,
    [MarshalAs(UnmanagedType.VBByRefStr)] ref String lpszName, int cbName,
    [MarshalAs(UnmanagedType.VBByRefStr)] ref String lpszVer, int cbVer);

The capGetDriverDescription function retrieves the version description of the capture driver. The wDriverIndex parameter specifies the index of the capture driver. The index can range from 0 through 9.

The structures are: BITMAPINFO, BITMAPINFOHEADER and VIDEOHDR. The VIDEOHDR structure is used by the callback function. It contains the buffered frame data. The BITMAPINFO structure defines the dimensions and color information of a Windows-based, device-independent Bitmap (DIB). The BITMAPINFOHEADER structure contains information about the dimensions and color format of a DIB. In our case, it defines the video format (frame size and bits per frame).

Inside the Code

The main part of the solution is the WebCamera class library, which consists of three classes:

  • WebCameraDevice
  • WebCameraEventArgs
  • WebCameraDeviceManager

WebCameraDevice

public WebCameraDevice(int frameWidth, int frameHeight, int preferredFPS,
    int camID, int parentHwnd)
{
    /*...*/
}

This initializes a new instance of the WebCameraDevice object. Focus of preferredFPS parameter: generally, Web cameras support a maximum of 30 FPS. The maximum FPS I could get on my A4Tech webcam was 20. Also, FPS depends on driver details. For example, enabling flicker slightly reduces FPS. Use the WebCamDeviceManager class to get all available devices and their indices. The camID parameter represents the selected device's index.

public void Start()
{
    /*...*/
    camHwnd = capCreateCaptureWindow("WebCam", 0, 0, 0, frameWidth,
        frameHeight, parentHwnd, camID); // Step 2

    //Try to connect a capture window to a capture driver

    if (SendMessage(camHwnd, WM_CAP_DRIVER_CONNECT, 0, 0))
    {
        //Step 3, fill bitmap structure (see source)
        /*...*/
        // Enables preview mode
        SendMessage(camHwnd, WM_CAP_SET_PREVIEW, 1, 0);
        // Sets the frame display rate in preview mode. 34 ms ~ 29FPS
        SendMessage(camHwnd, WM_CAP_SET_PREVIEWRATE, 34, 0);
        // Sets the format of captured video data.
        SendBitmapMessage(camHwnd, WM_CAP_SET_VIDEOFORMAT,
            Marshal.SizeOf(bInfo), ref bInfo);

        // Multithreading begins here
        frameThread = new Thread(new ThreadStart(this.FrameGrabber));
        bStart = true;       // Flag variable

        frameThread.Priority = ThreadPriority.Lowest;
        frameThread.Start();
    }
        /*...*/
}

The multithreading mechanism in WebCameraDevice consists of the AutoResetEvent object and frame-grabbing worker thread. Setting preferredFPS to 0 allows the user to control the frame capturing process manually. The worker thread waits (WaitOne() is called) until the user calls the AutoResetEvent object's Set() method (WaitHandle receives a signal). Otherwise, WaitOne(..) with the preferredFPSms (1000 / preferredFPS) parameter is called to wait for a defined amount of milliseconds.

After calling the Start() method, the worker thread starts capturing frames to the buffer. The WebCameraDevice object raises the OnCameraFrame event that contains frame data in Bitmap form.

private void FrameGrabber()
{
    while (bStart) // if worker active thread is still required
    {
        /*...*/
        // get the next frame. This is the SLOWEST part of the program
        SendMessage(camHwnd, WM_CAP_GRAB_FRAME_NOSTOP, 0, 0);
        //Make a function callback
        SendHeaderMessage(camHwnd, WM_CAP_SET_CALLBACK_FRAME, 0,
            delegateFrameCallBack);
        /*...*/
    }
}

What happens in this block of code? The bStart variable is a flag that turns to false when the Stop() method is called. While bStart remains true, the WM_CAP_GRAB_FRAME_NOSTOP message fills the frame buffer with a single uncompressed frame from the capture device. Then a callback is made. We use the delegateFrameCallBack variable instead of a direct callback function's name to avoid GC errors. Try replacing delegateFrameCallBack with FrameCallBack (callback function's name) and see what happens. The callback function looks like this:

private void <a class="code-string" name="<span">"FrameCallBack">FrameCallBack</a>(IntPtr hwnd, ref VIDEOHEADER hdr)
{
    if (OnCameraFrame != null)
    {
        Bitmap bmp = new Bitmap(frameWidth, frameHeight, 3 *
            frameWidth, System.Drawing.Imaging.PixelFormat.Format24bppRgb,
            hdr.lpData);
        OnCameraFrame(this, new WebCameraEventArgs(bmp));
    }

    if (preferredFPSms == 0)
    {
        // blocks thread until WaitHandle receives a signal
        autoEvent.WaitOne();
    }
    else
    {
        // blocks thread for preferred milliseconds
        autoEvent.WaitOne(preferredFPSms, false);
    }
}

As you can see, the function contains all Bitmap converting, event raising and WaitHandler operating stuff. That's it! The remaining methods are:

public void Set()
{
    //Send a signal to the current WainHandle and allow blocked worker
    //(FrameGrabber) thread to proceed
    autoEvent.Set();
}

public void Stop()
{
    try
    {
        bStart = false;
        Set();
        SendMessage(camHwnd, WM_CAP_DRIVER_DISCONNECT, 0, 0);
    }
    catch { }
}

public void ShowVideoDialog()
{
    SendMessage(camHwnd, WM_CAP_DLG_VIDEODISPLAY, 0, 0);
}

How Does It Work?

We have a class library with all the necessary Web camera image capturing classes. First of all, we have to get the available VFW devices and display them to the user:

public FormMain()
{
    InitializeComponent();
    WebCameraDeviceManager camManager = new WebCameraDeviceManager();
    // fill combo box with available devices' names
    cmbDevices.Items.AddRange(camManager.Devices);
    // First available video device.
    // I always receive "Microsoft WDM Image Capture (Win32)"
    cmbDevices.SelectedIndex = 0;
}

The start button and the OnCameraFrame event handler's code:

private void btnStart_Click(object sender, EventArgs e)
{
    /*...*/
    camDevice = new WebCameraDevice
        (320, 200, 0, cmbDevices.SelectedIndex, this.Handle.ToInt32());
    // Register for event notification
    camDevice.OnCameraFrame +=
        new WebCameraFrameDelegate(camDevice_OnCameraFrame);
    camDevice.Start();
    /*...*/
}

void camDevice_OnCameraFrame(object sender, WebCameraEventArgs e)
{
    /*...*/
    ImageProcessing.Filters.Flip(e.Frame, false, true); // Explained below
    pictureBox.Image = e.Frame;
    camDevice.Set(); // comment this if prefferedFPS != 0
    /*...*/
}

Have you noticed the prefferedFPS parameter's 0 value in WebCameraDevice's constructor? That's why the Set() method is called in the camDevice_OnCameraFrame event handler. Do you remember what happens inside the camDevice object? If not, check FrameCallBack above.

Unexpected Image Flip

There was an unexpected vertical image flip. I haven't discovered why this happens yet. It happens only in the case of BITMAPINFOHEADER's buffer conversion. Maybe there is a bug in the Bitmap class. To avoid flipping, I've referred to a great Image Processing for Dummies with C# and GDI+ article by Christian Graus. A fast grayscale filter was found on Bob Powell's site.

History

  • Release - 12 September, 2007

P.S.

I'd like to ask you to be lenient with the article because it's my first article on The Code Project. Please let me know if you have liked/disliked it or have any questions about it.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Zaur Nasibov
Software Developer
Finland Finland
I'm a Master degree student, studying at the University of Joensuu, Finland.

Comments and Discussions

 
QuestionImage Flip Pin
Michael Taub1-Jan-12 2:45
memberMichael Taub1-Jan-12 2:45 
Generalimage from cam Pin
blgonc19-Nov-10 9:02
memberblgonc19-Nov-10 9:02 
GeneralI got just many lines in the image, not the picture. Pin
reborn_zhang6-Dec-09 1:40
memberreborn_zhang6-Dec-09 1:40 
GeneralRe: I got just many lines in the image, not the picture. Pin
reborn_zhang6-Dec-09 2:29
memberreborn_zhang6-Dec-09 2:29 
Generaltwo webcams Pin
mehrdad33322-Nov-09 0:08
membermehrdad33322-Nov-09 0:08 
Generalhi It can't work with uvc camarea Pin
lw840120-Apr-09 22:57
memberlw840120-Apr-09 22:57 
Can vfw used for uvc camarea?
QuestionYour Help is highly needed Pin
Tekle4-Apr-09 8:33
memberTekle4-Apr-09 8:33 
AnswerRe: Your Help is highly needed Pin
Zaur Nasibov4-Apr-09 9:12
memberZaur Nasibov4-Apr-09 9:12 
GeneralThread wait in wrong place Pin
jerome_punzalan26-Sep-08 1:20
memberjerome_punzalan26-Sep-08 1:20 
QuestionAwesome, but 1/3 of a second to grab 2 frames? Pin
thewondersnarf18-Jul-08 4:37
memberthewondersnarf18-Jul-08 4:37 
AnswerRe: Awesome, but 1/3 of a second to grab 2 frames? Pin
thewondersnarf19-Jul-08 2:38
memberthewondersnarf19-Jul-08 2:38 
GeneralChange resolution? [modified] Pin
Demaker27-Jun-08 9:55
memberDemaker27-Jun-08 9:55 
GeneralRe: Change resolution? Pin
Zaur Nasibov27-Jun-08 10:13
memberZaur Nasibov27-Jun-08 10:13 
GeneralRe: Change resolution? Pin
Demaker27-Jun-08 11:34
memberDemaker27-Jun-08 11:34 
GeneralAV error on XP. Pin
KSFJ9-Apr-08 8:59
memberKSFJ9-Apr-08 8:59 
GeneralRe: AV error on XP. Pin
Zaur Nasibov9-Apr-08 9:07
memberZaur Nasibov9-Apr-08 9:07 
GeneralRe: AV error on XP. Pin
KSFJ9-Apr-08 12:02
memberKSFJ9-Apr-08 12:02 
GeneralRe: AV error on XP. Pin
KSFJ13-Apr-08 5:22
memberKSFJ13-Apr-08 5:22 
GeneralRe: AV error on XP. Pin
Zaur Nasibov13-Apr-08 6:42
memberZaur Nasibov13-Apr-08 6:42 
GeneralRe: AV error on XP. Pin
KSFJ17-Apr-08 12:36
memberKSFJ17-Apr-08 12:36 
GeneralRe: AV error on XP. Pin
alhambra-eidos8-May-08 9:01
memberalhambra-eidos8-May-08 9:01 
Questionvfw.h and capDlgVideoDisplay Pin
Tomice19-Feb-08 0:38
memberTomice19-Feb-08 0:38 
GeneralRe: vfw.h and capDlgVideoDisplay Pin
Zaur Nasibov19-Feb-08 1:12
memberZaur Nasibov19-Feb-08 1:12 
GeneralRe: vfw.h and capDlgVideoDisplay Pin
Tomice19-Feb-08 7:35
memberTomice19-Feb-08 7:35 
GeneralRe: vfw.h and capDlgVideoDisplay Pin
Zaur Nasibov19-Feb-08 9:55
memberZaur Nasibov19-Feb-08 9:55 
GeneralRe: vfw.h and capDlgVideoDisplay Pin
jk2l27-Mar-08 1:27
memberjk2l27-Mar-08 1:27 
GeneralRe: vfw.h and capDlgVideoDisplay Pin
dalve6-Dec-09 14:47
memberdalve6-Dec-09 14:47 
GeneralCreative WebCam Live! Pin
Member 370802018-Feb-08 22:25
memberMember 370802018-Feb-08 22:25 
GeneralRe: Creative WebCam Live! Pin
Zaur Nasibov19-Feb-08 1:09
memberZaur Nasibov19-Feb-08 1:09 
GeneralRe: Creative WebCam Live! Pin
Member 370802019-Feb-08 7:27
memberMember 370802019-Feb-08 7:27 
GeneralRe: Creative WebCam Live! Pin
dchen11110-Sep-09 16:43
memberdchen11110-Sep-09 16:43 
GeneralSpasibo zemleak Pin
Nariman15-Feb-08 6:01
memberNariman15-Feb-08 6:01 
QuestionDevice Pin
watta48313-Feb-08 5:51
memberwatta48313-Feb-08 5:51 
AnswerRe: Device [modified] Pin
watta48313-Feb-08 13:40
memberwatta48313-Feb-08 13:40 
GeneralAbsolutely terrific Pin
Member 413875016-Jan-08 21:16
memberMember 413875016-Jan-08 21:16 
QuestionDivide By Zero Exception Pin
Tron825-Oct-07 23:51
memberTron825-Oct-07 23:51 
AnswerRe: Divide By Zero Exception Pin
Zaur Nasibov6-Oct-07 6:19
memberZaur Nasibov6-Oct-07 6:19 
GeneralRe: Divide By Zero Exception Pin
Tron826-Oct-07 21:27
memberTron826-Oct-07 21:27 
GeneralRe: Divide By Zero Exception Pin
Zaur Nasibov6-Oct-07 22:59
memberZaur Nasibov6-Oct-07 22:59 
GeneralRe: Divide By Zero Exception Pin
Tron827-Oct-07 7:03
memberTron827-Oct-07 7:03 
GeneralRe: Divide By Zero Exception Pin
Zaur Nasibov7-Oct-07 9:24
memberZaur Nasibov7-Oct-07 9:24 
GeneralRe: Divide By Zero Exception Pin
Tron828-Oct-07 4:35
memberTron828-Oct-07 4:35 
GeneralRe: Divide By Zero Exception Pin
Zaur Nasibov9-Oct-07 3:48
memberZaur Nasibov9-Oct-07 3:48 
GeneralRe: Divide By Zero Exception Pin
Tron829-Oct-07 4:14
memberTron829-Oct-07 4:14 
JokeUnexpected Image Flip Pin
KarstenK24-Sep-07 23:41
memberKarstenK24-Sep-07 23:41 
GeneralRe: Unexpected Image Flip Pin
Zaur Nasibov25-Sep-07 2:05
memberZaur Nasibov25-Sep-07 2:05 
GeneralFantastic Pin
monkeyliar19-Sep-07 7:25
membermonkeyliar19-Sep-07 7:25 
GeneralRe: Fantastic Pin
Zaur Nasibov19-Sep-07 8:48
memberZaur Nasibov19-Sep-07 8:48 
GeneralRe: Fantastic Pin
monkeyliar21-Sep-07 6:39
membermonkeyliar21-Sep-07 6:39 
GeneralRe: Fantastic Pin
Zaur Nasibov21-Sep-07 9:04
memberZaur Nasibov21-Sep-07 9:04 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web04 | 2.8.150427.4 | Last Updated 16 Feb 2008
Article Copyright 2007 by Zaur Nasibov
Everything else Copyright © CodeProject, 1999-2015
Layout: fixed | fluid