Click here to Skip to main content
Click here to Skip to main content

FrameGrabber

, 16 Apr 2008 CPOL
Rate this:
Please Sign up or sign in to vote.
A class for extracting individual frames from a video as bitmaps
FrameGrabber_src

Introduction

FrameGrabber allows you to access individual frames in a video file as standard Bitmap objects. It can iterate through based on timestamp or frame number.

Background

This DLL is based on the standard MediaDet object from DirectShow (see MSDN). FrameGrabber alleviates two problems with IMediaDet:

  • It takes a lot of scaffolding code just to use the IMediaDet interface (at least if you're used to the managed world it seems like a lot)
  • IMediaDet gives you access to a raw byte buffer for the image data, not a Bitmap object

FrameGrabber simplifies the process by letting you just specify a source video file, then retrieve bitmaps by frame numbers or timestamps.

Using the Code

Create a FrameGrabber object, specifying the video file path in the constructor or the FileName property prior to calling methods.

Call one of the accessor methods to get a bitmap (GetImage, GetFrame, this[], GetEnumerator(), etc). Under the covers, every accessor ends up calling the GetImageAtTime() method, but they all return the image data in different ways that are convenient depending on the context in which you are calling them.

Every accessor has two versions — one to retrieve a Frame, and another to retrieve a Bitmap. A Frame is a simple class that FrameGrabber exposes that aggregates together the image associated with a specific frame and its indexing information (frame number, timestamp, etc). This is handy because FrameGrabber implements IEnumerable<Frame>, meaning that even when iterating via a foreach statement, you will be able to access indexing information.

Included in the zip is complete MSDN-style documentation of all the methods (docs/index.html).

Points of Interest

FrameGrabber uses the DirectshowNet library, which is an open-source project that provides a managed wrapper around the standard DirectShow libraries. I prefer this approach to using Microsoft's own Managed DirectX since a) Managed DirectX hasn't been updated in a long time and b) there is little to no documentation for Managed DirectX

Even though the DirectShow architecture does not fit very well with the managed view of the world, at least it is relatively well-documented and there are plenty of real-world examples which you can inspect.

Here is an example showing how to initialize the MediaDet object with a video file. This is nearly identical to how you would do it in raw C++, minus some of the memory allocation worries. Notice, however, that AMMediaType objects still require manual destruction:

mediaDet = (IMediaDet)new MediaDet();
DsError.ThrowExceptionForHR(mediaDet.put_Filename(fileName));

// find the video stream in the file
int index = 0;
Guid type = Guid.Empty;
while(type != MediaType.Video)
{
    mediaDet.put_CurrentStream(index++);
    mediaDet.get_StreamType(out type);
}

// retrieve some measurements from the video
mediaDet.get_FrameRate(out frameRate);

mediaType = new AMMediaType();
mediaDet.get_StreamMediaType(mediaType);
videoInfo = (VideoInfoHeader)Marshal.PtrToStructure(mediaType.formatPtr,
    typeof(VideoInfoHeader));
DsUtils.FreeAMMediaType(mediaType);
mediaType = null;
width = videoInfo.BmiHeader.Width;
height = videoInfo.BmiHeader.Height;

mediaDet.get_StreamLength(out mediaLength);
frameCount = (int)(frameRate * mediaLength);

The following code shows how to retrieve an image from the MediaDet once it has been initialized. The key call is to MediaDet.GetBitmapBits() — notice though that the code calls it twice, the first time with a null pointer (ie. IntPtr.Zero), and the second with a pointer to the actual destination buffer. Why is this? When you call GetBitmapBits with a null pointer, the MediaDet object returns the number of bytes it needs the buffer to hold via the out bufferSize parameter. Then the code allocates a buffer with the expected size and makes the call again with the correct pointer to the buffer:

// create a buffer to hold the image data from the MediaDet
int bufferSize;
mediaDet.GetBitmapBits(seconds, out bufferSize, IntPtr.Zero, width, height);
bufferPtr = Marshal.AllocHGlobal(bufferSize);
mediaDet.GetBitmapBits(seconds, out bufferSize, bufferPtr, width, height);

This last chunk of code copies pixel data from the buffer supplied by the MediaDet object straight into a managed Bitmap object. I chose to copy the buffer using int* rather than byte* since that cuts the number of copy operations by a factor of four. Also, since most machines have a 32-bit word size, a single int copy operation is (theoretically) faster than a single byte copy operation because no work needs to be done to extract sub-bytes from the word. Finally, it is necessary to flip the pixels since DirectShow returns them in a different order than the one the Bitmap class expects. You could achieve this with some clever pointer manipulation when copying the image buffer, but it's cleaner, safer, and clearer to do a straightforward copy, then let the Bitmap class handle the flip:

// compose a bitmap from the data in the managed buffer    
unsafe
{
    returnValue = new Bitmap(width, height, PixelFormat.Format24bppRgb);
    BitmapData imageData = returnValue.LockBits(new Rectangle(0, 0, width, height),
        ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
    int* imagePtr = (int*)imageData.Scan0;

    int bitmapHeaderSize = Marshal.SizeOf(videoInfo.BmiHeader);
    int* sourcePtr = (int*)((byte*)bufferPtr.ToPointer() + bitmapHeaderSize);

    for(int i = 0; i < (bufferSize - bitmapHeaderSize) / 4; i++)
    {
        *imagePtr = *sourcePtr;
        imagePtr++;
        sourcePtr++;
    }

    returnValue.UnlockBits(imageData);
    // DirectShow stores pixels in a different order than Bitmaps do
    returnValue.RotateFlip(RotateFlipType.Rotate180FlipX); 
}

Marshal.FreeHGlobal(bufferPtr);

return returnValue;

History

4/9/2008 - initial release

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Jacob Klint
Software Developer
United States United States
No Biography provided

Comments and Discussions

 
QuestionSupport mp4 video format PinmemberAdar1015-Jun-14 19:14 
Questionhow to dispose the object of the framegrabber class PinmemberAdi555523-Apr-14 4:53 
QuestionMemory Leakage? PinmemberTarPista22-May-13 0:25 
QuestionMedia Support in frame grabber Pinmember123sandipkumar@gmail.com18-Mar-13 2:58 
QuestionDivX videos PinmemberMember 430850911-Feb-12 5:44 
GeneralSolution for 64bit compatibility Pinmemberliran_cohen4-May-11 5:49 
GeneralFrameGrabber Compatibility PinmemberChina201017-Mar-11 9:06 
GeneralGrab frame by another thread PinmemberTiramisung27-Apr-10 23:19 
QuestionPixel format Pinmemberapol524 p16-May-09 19:09 
GeneralDoesn't seems to work with mpeg2 PinmemberLogRaam23-Apr-09 11:26 
QuestionHow can I modify the code to work with Windows-CE? Pinmemberafikiminfo14-Jan-09 19:29 
General64bit related issue in webapplication implemented DirectShowLib-2005.dll(v2.0) Pinmemberajayakumarjena19-Nov-08 19:00 
QuestionOnly grabbing the first frame PinmemberBR_Richo19-Sep-08 15:41 
QuestionRe: Only grabbing the first frame [modified] PinmemberBR_Richo19-Sep-08 16:06 
AnswerRe: Only grabbing the first frame PinmemberBR_Richo22-Sep-08 0:52 
GeneralWorks on Vista 64bit? + performance improvement PinmemberJockerSoft24-May-08 10:56 
GeneralOutput generates "black" bmp... PinmemberMember 279939229-Apr-08 1:24 
GeneralRe: Output generates "black" bmp... PinmemberJacob Klint30-Apr-08 9:29 
QuestionSome videos can't be used with your class [modified] PinmemberMystery12322-Apr-08 6:11 
AnswerRe: Some videos can't be used with your class PinmemberJacob Klint30-Apr-08 9:28 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web03 | 2.8.141030.1 | Last Updated 16 Apr 2008
Article Copyright 2008 by Jacob Klint
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid