Click here to Skip to main content
12,443,996 members (44,081 online)
Click here to Skip to main content
Add your own
alternative version

Stats

67.8K views
4.3K downloads
46 bookmarked
Posted

FrameGrabber

, 16 Apr 2008 CPOL
Rate this:
Please Sign up or sign in to vote.
A class for extracting individual frames from a video as bitmaps
FrameGrabber_src

Introduction

FrameGrabber allows you to access individual frames in a video file as standard Bitmap objects. It can iterate through based on timestamp or frame number.

Background

This DLL is based on the standard MediaDet object from DirectShow (see MSDN). FrameGrabber alleviates two problems with IMediaDet:

  • It takes a lot of scaffolding code just to use the IMediaDet interface (at least if you're used to the managed world it seems like a lot)
  • IMediaDet gives you access to a raw byte buffer for the image data, not a Bitmap object

FrameGrabber simplifies the process by letting you just specify a source video file, then retrieve bitmaps by frame numbers or timestamps.

Using the Code

Create a FrameGrabber object, specifying the video file path in the constructor or the FileName property prior to calling methods.

Call one of the accessor methods to get a bitmap (GetImage, GetFrame, this[], GetEnumerator(), etc). Under the covers, every accessor ends up calling the GetImageAtTime() method, but they all return the image data in different ways that are convenient depending on the context in which you are calling them.

Every accessor has two versions — one to retrieve a Frame, and another to retrieve a Bitmap. A Frame is a simple class that FrameGrabber exposes that aggregates together the image associated with a specific frame and its indexing information (frame number, timestamp, etc). This is handy because FrameGrabber implements IEnumerable<Frame>, meaning that even when iterating via a foreach statement, you will be able to access indexing information.

Included in the zip is complete MSDN-style documentation of all the methods (docs/index.html).

Points of Interest

FrameGrabber uses the DirectshowNet library, which is an open-source project that provides a managed wrapper around the standard DirectShow libraries. I prefer this approach to using Microsoft's own Managed DirectX since a) Managed DirectX hasn't been updated in a long time and b) there is little to no documentation for Managed DirectX

Even though the DirectShow architecture does not fit very well with the managed view of the world, at least it is relatively well-documented and there are plenty of real-world examples which you can inspect.

Here is an example showing how to initialize the MediaDet object with a video file. This is nearly identical to how you would do it in raw C++, minus some of the memory allocation worries. Notice, however, that AMMediaType objects still require manual destruction:

mediaDet = (IMediaDet)new MediaDet();
DsError.ThrowExceptionForHR(mediaDet.put_Filename(fileName));

// find the video stream in the file
int index = 0;
Guid type = Guid.Empty;
while(type != MediaType.Video)
{
    mediaDet.put_CurrentStream(index++);
    mediaDet.get_StreamType(out type);
}

// retrieve some measurements from the video
mediaDet.get_FrameRate(out frameRate);

mediaType = new AMMediaType();
mediaDet.get_StreamMediaType(mediaType);
videoInfo = (VideoInfoHeader)Marshal.PtrToStructure(mediaType.formatPtr,
    typeof(VideoInfoHeader));
DsUtils.FreeAMMediaType(mediaType);
mediaType = null;
width = videoInfo.BmiHeader.Width;
height = videoInfo.BmiHeader.Height;

mediaDet.get_StreamLength(out mediaLength);
frameCount = (int)(frameRate * mediaLength);

The following code shows how to retrieve an image from the MediaDet once it has been initialized. The key call is to MediaDet.GetBitmapBits() — notice though that the code calls it twice, the first time with a null pointer (ie. IntPtr.Zero), and the second with a pointer to the actual destination buffer. Why is this? When you call GetBitmapBits with a null pointer, the MediaDet object returns the number of bytes it needs the buffer to hold via the out bufferSize parameter. Then the code allocates a buffer with the expected size and makes the call again with the correct pointer to the buffer:

// create a buffer to hold the image data from the MediaDet
int bufferSize;
mediaDet.GetBitmapBits(seconds, out bufferSize, IntPtr.Zero, width, height);
bufferPtr = Marshal.AllocHGlobal(bufferSize);
mediaDet.GetBitmapBits(seconds, out bufferSize, bufferPtr, width, height);

This last chunk of code copies pixel data from the buffer supplied by the MediaDet object straight into a managed Bitmap object. I chose to copy the buffer using int* rather than byte* since that cuts the number of copy operations by a factor of four. Also, since most machines have a 32-bit word size, a single int copy operation is (theoretically) faster than a single byte copy operation because no work needs to be done to extract sub-bytes from the word. Finally, it is necessary to flip the pixels since DirectShow returns them in a different order than the one the Bitmap class expects. You could achieve this with some clever pointer manipulation when copying the image buffer, but it's cleaner, safer, and clearer to do a straightforward copy, then let the Bitmap class handle the flip:

// compose a bitmap from the data in the managed buffer    
unsafe
{
    returnValue = new Bitmap(width, height, PixelFormat.Format24bppRgb);
    BitmapData imageData = returnValue.LockBits(new Rectangle(0, 0, width, height),
        ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
    int* imagePtr = (int*)imageData.Scan0;

    int bitmapHeaderSize = Marshal.SizeOf(videoInfo.BmiHeader);
    int* sourcePtr = (int*)((byte*)bufferPtr.ToPointer() + bitmapHeaderSize);

    for(int i = 0; i < (bufferSize - bitmapHeaderSize) / 4; i++)
    {
        *imagePtr = *sourcePtr;
        imagePtr++;
        sourcePtr++;
    }

    returnValue.UnlockBits(imageData);
    // DirectShow stores pixels in a different order than Bitmaps do
    returnValue.RotateFlip(RotateFlipType.Rotate180FlipX); 
}

Marshal.FreeHGlobal(bufferPtr);

return returnValue;

History

4/9/2008 - initial release

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Jacob Klint
Software Developer
United States United States
No Biography provided

You may also be interested in...

Pro
Pro

Comments and Discussions

 
QuestionMyFrameGrabber.GetImageAtTime doesnt works at any time Pin
Member 1157253418-Jul-15 6:08
memberMember 1157253418-Jul-15 6:08 
QuestionMessage Removed Pin
Member 109556235-Jun-15 22:25
memberMember 109556235-Jun-15 22:25 
QuestionSupport mp4 video format Pin
Adar1015-Jun-14 19:14
memberAdar1015-Jun-14 19:14 
Questionhow to dispose the object of the framegrabber class Pin
Adi555523-Apr-14 4:53
memberAdi555523-Apr-14 4:53 
AnswerRe: how to dispose the object of the framegrabber class Pin
Member 1095562330-Sep-15 19:48
memberMember 1095562330-Sep-15 19:48 
QuestionMemory Leakage? Pin
TarPista22-May-13 0:25
memberTarPista22-May-13 0:25 
QuestionMedia Support in frame grabber Pin
123sandipkumar@gmail.com18-Mar-13 2:58
member123sandipkumar@gmail.com18-Mar-13 2:58 
QuestionDivX videos Pin
Member 430850911-Feb-12 5:44
memberMember 430850911-Feb-12 5:44 
GeneralSolution for 64bit compatibility Pin
liran_cohen4-May-11 5:49
memberliran_cohen4-May-11 5:49 
GeneralFrameGrabber Compatibility Pin
China201017-Mar-11 9:06
memberChina201017-Mar-11 9:06 
GeneralGrab frame by another thread Pin
Tiramisung27-Apr-10 23:19
memberTiramisung27-Apr-10 23:19 
QuestionPixel format Pin
apol524 p16-May-09 19:09
memberapol524 p16-May-09 19:09 
GeneralDoesn't seems to work with mpeg2 Pin
LogRaam23-Apr-09 11:26
memberLogRaam23-Apr-09 11:26 
QuestionHow can I modify the code to work with Windows-CE? Pin
afikiminfo14-Jan-09 19:29
memberafikiminfo14-Jan-09 19:29 
General64bit related issue in webapplication implemented DirectShowLib-2005.dll(v2.0) Pin
ajayakumarjena19-Nov-08 19:00
memberajayakumarjena19-Nov-08 19:00 
QuestionOnly grabbing the first frame Pin
BR_Richo19-Sep-08 15:41
memberBR_Richo19-Sep-08 15:41 
QuestionRe: Only grabbing the first frame [modified] Pin
BR_Richo19-Sep-08 16:06
memberBR_Richo19-Sep-08 16:06 
AnswerRe: Only grabbing the first frame Pin
BR_Richo22-Sep-08 0:52
memberBR_Richo22-Sep-08 0:52 
GeneralWorks on Vista 64bit? + performance improvement Pin
JockerSoft24-May-08 10:56
memberJockerSoft24-May-08 10:56 
GeneralOutput generates "black" bmp... Pin
Member 279939229-Apr-08 1:24
memberMember 279939229-Apr-08 1:24 
GeneralRe: Output generates "black" bmp... Pin
Jacob Klint30-Apr-08 9:29
memberJacob Klint30-Apr-08 9:29 
QuestionSome videos can't be used with your class [modified] Pin
Mystery12322-Apr-08 6:11
memberMystery12322-Apr-08 6:11 
AnswerRe: Some videos can't be used with your class Pin
Jacob Klint30-Apr-08 9:28
memberJacob Klint30-Apr-08 9:28 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web02 | 2.8.160811.3 | Last Updated 16 Apr 2008
Article Copyright 2008 by Jacob Klint
Everything else Copyright © CodeProject, 1999-2016
Layout: fixed | fluid