Click here to Skip to main content
Click here to Skip to main content

Extract Frames from Video Files

, 27 Sep 2007 CPOL
Rate this:
Please Sign up or sign in to vote.
Class to extract frames from most video file formats using IMediaDet
Sample Image - extractvideoframes.jpg

Introduction

This class contains methods to use the IMediaDet interface that can be found in Microsoft DirectShow. The Media Detector object, among other things, can be used to extract still pictures from several file formats including *.avi, *.wmv and some *.mpeg files.

This class exposes the GetFrameFromVideo, GetVideoSize and SaveFrameFromVideo methods that can be used from any .NET application. The class also takes care of translating HRESULTs returned from the functions to meaningful .NET exceptions.

Using the Code

Just add a reference to JockerSoft.Media.dll in your project (or include the source code). Remember also to distribute Interop.DexterLib.dll.
All the methods are static, so to use them just do something like this:

try
{
    this.pictureBox1.Image = FrameGrabber.GetFrameFromVideo(strVideoFile, 0.2d);
}
catch (InvalidVideoFileException ex)
{
    MessageBox.Show(ex.Message, "Extraction failed");
}
catch (StackOverflowException)
{
    MessageBox.Show("The target image size is too big", "Extraction failed");
}

or

try
{
    FrameGrabber.SaveFrameFromVideo(strVideoFile, 0.2d, strBitmapFile);
}
catch (InvalidVideoFileException ex)
{
    MessageBox.Show(ex.Message, "Extraction failed");
}

Here, we used the simplest of the three overloads of GetFrameFromVideo and SaveFrameFromVideo methods, presented in this article.

Points of Interest

The IMediaDet and linked interfaces/classes are exposed in qedit.dll, that can be found in System32 directory. Fortunately this DLL can be imported automatically using tlbimp, so no code is needed to wrap it.

To extract images, there are two methods: extract them in memory (using GetBitmapBits - here GetFrameFromVideo) or extract them and save to a bitmap file (using WriteBitmapBits - here SaveFrameFromVideo).

WriteBitmapBits is really simple to be used: we just need to find the video stream on the file, open it and specify an output file name for the bitmap image.

public static void SaveFrameFromVideo(string videoFile,
         double percentagePosition, string outputBitmapFile,
         out double streamLength, Size target)
{
    if (percentagePosition > 1 || percentagePosition < 0)
        throw new ArgumentOutOfRangeException("percentagePosition", 
                percentagePosition, "Valid range is 0.0 .. 1.0");

    try
    {
        MediaDetClass mediaDet;
        _AMMediaType mediaType;
        if (openVideoStream(videoFile, out mediaDet, out mediaType))
        {
            streamLength = mediaDet.StreamLength;
            
            //calculates the REAL target size of our frame
            if (target == Size.Empty)
                target = getVideoSize(mediaType);
            else
                target = scaleToFit(target, getVideoSize(mediaType));

            mediaDet.WriteBitmapBits(streamLength * percentagePosition, 
                        target.Width, target.Height, outputBitmapFile);

            return;
        }
    }
    catch (COMException ex)
    {
        throw new InvalidVideoFileException(getErrorMsg((uint)ex.ErrorCode), ex);
    }

    throw new InvalidVideoFileException("No video stream was found");
}

You'll notice that two private methods are used here. They are openVideoStream and getVideoSize. Their implementation is straight forward:

private static bool openVideoStream(string videoFile, 
            out MediaDetClass mediaDetClass, out _AMMediaType aMMediaType)
{
    MediaDetClass mediaDet = new MediaDetClass();
    
    //loads file
    mediaDet.Filename = videoFile;

    //gets # of streams
    int streamsNumber = mediaDet.OutputStreams;

    //finds a video stream and grabs a frame
    for (int i = 0; i < streamsNumber; i++)
    {
        mediaDet.CurrentStream = i;
        _AMMediaType mediaType = mediaDet.StreamMediaType;

        if (mediaType.majortype == JockerSoft.Media.MayorTypes.MEDIATYPE_Video)
        {
            mediaDetClass = mediaDet;
            aMMediaType = mediaType;
            return true;
        }
    }

    mediaDetClass = null;
    aMMediaType = new _AMMediaType();
    return false;
}

(where MEDIATYPE_Video is the GUID used for video files).

private static Size getVideoSize(_AMMediaType mediaType)
{
    WinStructs.VIDEOINFOHEADER videoInfo = 
        (WinStructs.VIDEOINFOHEADER)Marshal.PtrToStructure(mediaType.pbFormat, 
            typeof(WinStructs.VIDEOINFOHEADER));
    
    return new Size(videoInfo.bmiHeader.biWidth, videoInfo.bmiHeader.biHeight);
}

Using GetBitmapBits to avoid saving the image on disk is a bit trickier, since we need to deal with direct access to memory.

The first part is identical to SaveFrameFromVideo, then we have to call GetBitmapBits with the pBuffer parameter set to null to get the size for the buffer of bytes that will contain the 24bpp image (GetBitmapBits always returns 24bpp images).

Once we have the size of the buffer, we allocate memory on the heap to receive the image (in the first version of this code, memory was allocated on the stack which is fine if the target image is small sized, but if it is big we may get a nice StackOverflowException because stack memory is rather limited).

After this, we call GetBitmapBits again, but this time the buffer will be filled with image bytes. Now we create a bitmap from these bytes (remembering that they start with BITMAPINFOHEADER structure, the size of which is 40 bytes).

public static Bitmap GetFrameFromVideo(string videoFile, 
            double percentagePosition, out double streamLength, Size target)
{
    if (percentagePosition > 1 || percentagePosition < 0)
        throw new ArgumentOutOfRangeException("percentagePosition", 
                percentagePosition, "Valid range is 0.0 .. 1.0");

    try 
    {
        MediaDetClass mediaDet;
        _AMMediaType mediaType;
        if (openVideoStream(videoFile, out mediaDet, out mediaType))
        {
            streamLength = mediaDet.StreamLength;

            //calculates the REAL target size of our frame
            if (target == Size.Empty)
                target = getVideoSize(mediaType);
            else
                target = scaleToFit(target, getVideoSize(mediaType));

            unsafe 
            {
                Size s= GetVideoSize(videoFile);
                //equal to sizeof(CommonClasses.BITMAPINFOHEADER);
                int bmpinfoheaderSize = 40;                 

                //get size for buffer
                int bufferSize = (((s.Width * s.Height) * 24) / 8 ) + bmpinfoheaderSize;
                //equal to mediaDet.GetBitmapBits
                //    (0d, ref bufferSize, ref *buffer, target.Width, target.Height);    

                //allocates enough memory to store the frame
                IntPtr frameBuffer = 
                    System.Runtime.InteropServices.Marshal.AllocHGlobal(bufferSize);
                byte* frameBuffer2 = (byte*)frameBuffer.ToPointer();

                //gets bitmap, save in frameBuffer2
                mediaDet.GetBitmapBits(streamLength * percentagePosition, 
                    ref bufferSize, ref *frameBuffer2, target.Width, target.Height);

                //now in buffer2 we have a BITMAPINFOHEADER structure 
                //followed by the DIB bits
                Bitmap bmp = new Bitmap(target.Width, target.Height, target.Width * 3, 
                    System.Drawing.Imaging.PixelFormat.Format24bppRgb, 
                    new IntPtr(frameBuffer2 + bmpinfoheaderSize));

                bmp.RotateFlip(RotateFlipType.Rotate180FlipX);
                System.Runtime.InteropServices.Marshal.FreeHGlobal(frameBuffer);
                return bmp;
            }
        }
    }
    catch (COMException ex)
    {
        throw new InvalidVideoFileException(getErrorMsg((uint)ex.ErrorCode), ex);
    }

    throw new InvalidVideoFileException("No video stream was found");
}

Known Limitations

  • The biggest one is the StackOverflowException using stackalloc. There must be a way to pass to GetBitmapBits a buffer created on the heap, but I'm not yet very good with this unmanaged stuff.
    • This problem has been resolved. The solution is provided by _coder_ in the comments below.
  • On my machine, I got random errors when passing certain target sizes to GetBitmapBits: when the target size is 125x125 for example, the Bitmap constructor fails.
    • Only sizes multiples of 4 are allowed. Thanks to ujr (see comments).
  • "The IMediaDet interface does not support VIDEOINFOHEADER2 formats": This means it cannot open some *.mpeg video files.

History

  • 27th February, 2006: Initial release
  • 17th March, 2006: Replaced memory allocation on the stack
  • 27th September, 2007: Some fixes

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author


Comments and Discussions

 
Questioncan we extract key frame using this? PinmemberMember 110823658-Nov-14 22:34 
Questiondoes not works if PinmemberAbdEllah Gogop21-Oct-14 0:30 
QuestionMP4 Pinmembervincent6420-Jun-14 0:05 
GeneralMy vote of 1 PinmemberGMBarak1-Apr-14 12:04 
QuestionCrash Pinmemberjeff parnau12-Nov-13 2:59 
QuestionMP4 format file not working PinmemberSabyasachi Maity15-May-13 22:57 
QuestionMaximum number of accesses before it returns error Pinmemberhariprasad1020-Nov-12 17:08 
GeneralMy vote of 1 PinmemberMember 94896058-Oct-12 8:37 
Questionasking for flv video image extract in c# PinmemberMember 927580727-Sep-12 19:24 
Questionmp4 files Pinmemberigorchirok9-Jul-12 9:50 
QuestionGetting an error when using example PinmemberMunken_0622-Jun-12 0:07 
GeneralMy vote of 5 PinmemberJoe Sonderegger15-May-12 8:49 
QuestionPossibe to extract from MKV file? Pinmemberbabzog25-Apr-12 7:39 
QuestionExtracting all frames but only first frame is extracted from .avi file Pinmemberfarid vanzara121-Apr-12 2:32 
QuestionWhy does Target size must be a multiple of 4 ? Pinmemberrolbackse22-Jan-12 18:39 
GeneralDo not use this PinmemberCuchuk Sergey21-Dec-10 9:21 
GeneralRe: Do not use this PinmemberCuchuk Sergey21-Dec-10 10:11 
GeneralDexterlib (distribution or source location) PinmemberMember 39968537-Dec-10 15:05 
GeneralSoftware is not supporting to mpg Pinmembervijayr18-Nov-10 20:17 
GeneralCapture frame at given time within video Pinmemberakula7702-Jun-10 6:39 
GeneralJockerSoft.Media.dll PinmemberSaira138-Apr-10 6:18 
QuestionHow to get it working on 64-bit CPU PinmemberDmitri Nesteruk25-Aug-09 4:45 
Questionusage of DLL in commercial product -- any patent issues? Pinmemberrtswguru4-May-09 8:35 
AnswerRe: usage of DLL in commercial product -- any patent issues? Pinmemberakula7703-Jun-10 9:01 
GeneralShow only the first frame Pinmemberparadoxshark3-Apr-09 12:23 
GeneralRe: Show only the first frame PinmemberJockerSoft4-Apr-09 3:27 
GeneralRe: Show only the first frame Pinmemberjkkghjhjgh4-Nov-09 18:05 
GeneralRe: Show only the first frame Pinmemberjkkghjhjgh25-Nov-09 3:43 
QuestionRe: Show only the first frame Pinmemberxmu_research9-Dec-09 17:05 
NewsOpen Video with browse........... Pinmembersushilabhanvar1-Apr-09 1:44 
GeneralIssue with track bar in vedio cutter................... Pinmembersushilabhanvar1-Apr-09 1:38 
GeneralInsert and edit frame. Pinmemberneglewis11-Mar-09 4:46 
GeneralRe: Insert and edit frame. PinmemberJockerSoft4-Apr-09 3:25 
GeneralRe: Insert and edit frame. Pinmemberhelakhedira4-Jun-11 11:46 
GeneralThank you Pinmemberxeinet14-Jan-09 23:46 
GeneralOperations on yuv files using C++ Pinmembersaeed.ullah14-Dec-08 23:36 
GeneralThe system cannot find the path specified. (Exception from HRESULT: 0x80070003) PinmemberMember 304769129-Oct-08 22:02 
Questionwhat is name of that movie? PinmemberUnruled Boy25-Sep-08 6:37 
AnswerRe: what is name of that movie? PinmemberJockerSoft25-Sep-08 9:15 
GeneralRe: what is name of that movie? PinmemberUnruled Boy25-Sep-08 16:57 
GeneralMultimedia Pinmembersanaritasages30-Aug-08 3:19 
QuestionProblem while running FrameGrabber.SaveFrameFromVideo Pinmemberminamohebn10-Jul-08 12:51 
GeneralErroneous image grabber PinmemberRadu Simionescu6-Jul-08 13:53 
GeneralRe: Erroneous image grabber PinmemberJockerSoft25-Sep-08 9:12 
GeneralException from HRESULT: 0x80040200 Pinmembernithydurai27-Jun-08 1:38 
GeneralRe: Exception from HRESULT: 0x80040200 PinmemberJockerSoft25-Sep-08 9:10 
GeneralProblem with Vista Pinmembermaaldo6-May-08 14:41 
GeneralRe: Problem with Vista Pinmemberkasadi6-May-08 19:04 
GeneralRe: Problem with Vista Pinmember_runeb9-Sep-08 10:09 
QuestionInstalling Requirement Pinmemberiyasin20-Feb-08 21:36 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web04 | 2.8.141216.1 | Last Updated 27 Sep 2007
Article Copyright 2006 by JockerSoft
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid