Click here to Skip to main content
12,065,756 members (27,732 online)
Click here to Skip to main content
Add your own
alternative version

Stats

337.5K views
28.4K downloads
124 bookmarked
Posted

Extract Frames from Video Files

, 27 Sep 2007 CPOL
Rate this:
Please Sign up or sign in to vote.
Class to extract frames from most video file formats using IMediaDet
Sample Image - extractvideoframes.jpg

Introduction

This class contains methods to use the IMediaDet interface that can be found in Microsoft DirectShow. The Media Detector object, among other things, can be used to extract still pictures from several file formats including *.avi, *.wmv and some *.mpeg files.

This class exposes the GetFrameFromVideo, GetVideoSize and SaveFrameFromVideo methods that can be used from any .NET application. The class also takes care of translating HRESULTs returned from the functions to meaningful .NET exceptions.

Using the Code

Just add a reference to JockerSoft.Media.dll in your project (or include the source code). Remember also to distribute Interop.DexterLib.dll.
All the methods are static, so to use them just do something like this:

try
{
    this.pictureBox1.Image = FrameGrabber.GetFrameFromVideo(strVideoFile, 0.2d);
}
catch (InvalidVideoFileException ex)
{
    MessageBox.Show(ex.Message, "Extraction failed");
}
catch (StackOverflowException)
{
    MessageBox.Show("The target image size is too big", "Extraction failed");
}

or

try
{
    FrameGrabber.SaveFrameFromVideo(strVideoFile, 0.2d, strBitmapFile);
}
catch (InvalidVideoFileException ex)
{
    MessageBox.Show(ex.Message, "Extraction failed");
}

Here, we used the simplest of the three overloads of GetFrameFromVideo and SaveFrameFromVideo methods, presented in this article.

Points of Interest

The IMediaDet and linked interfaces/classes are exposed in qedit.dll, that can be found in System32 directory. Fortunately this DLL can be imported automatically using tlbimp, so no code is needed to wrap it.

To extract images, there are two methods: extract them in memory (using GetBitmapBits - here GetFrameFromVideo) or extract them and save to a bitmap file (using WriteBitmapBits - here SaveFrameFromVideo).

WriteBitmapBits is really simple to be used: we just need to find the video stream on the file, open it and specify an output file name for the bitmap image.

public static void SaveFrameFromVideo(string videoFile,
         double percentagePosition, string outputBitmapFile,
         out double streamLength, Size target)
{
    if (percentagePosition > 1 || percentagePosition < 0)
        throw new ArgumentOutOfRangeException("percentagePosition", 
                percentagePosition, "Valid range is 0.0 .. 1.0");

    try
    {
        MediaDetClass mediaDet;
        _AMMediaType mediaType;
        if (openVideoStream(videoFile, out mediaDet, out mediaType))
        {
            streamLength = mediaDet.StreamLength;
            
            //calculates the REAL target size of our frame
            if (target == Size.Empty)
                target = getVideoSize(mediaType);
            else
                target = scaleToFit(target, getVideoSize(mediaType));

            mediaDet.WriteBitmapBits(streamLength * percentagePosition, 
                        target.Width, target.Height, outputBitmapFile);

            return;
        }
    }
    catch (COMException ex)
    {
        throw new InvalidVideoFileException(getErrorMsg((uint)ex.ErrorCode), ex);
    }

    throw new InvalidVideoFileException("No video stream was found");
}

You'll notice that two private methods are used here. They are openVideoStream and getVideoSize. Their implementation is straight forward:

private static bool openVideoStream(string videoFile, 
            out MediaDetClass mediaDetClass, out _AMMediaType aMMediaType)
{
    MediaDetClass mediaDet = new MediaDetClass();
    
    //loads file
    mediaDet.Filename = videoFile;

    //gets # of streams
    int streamsNumber = mediaDet.OutputStreams;

    //finds a video stream and grabs a frame
    for (int i = 0; i < streamsNumber; i++)
    {
        mediaDet.CurrentStream = i;
        _AMMediaType mediaType = mediaDet.StreamMediaType;

        if (mediaType.majortype == JockerSoft.Media.MayorTypes.MEDIATYPE_Video)
        {
            mediaDetClass = mediaDet;
            aMMediaType = mediaType;
            return true;
        }
    }

    mediaDetClass = null;
    aMMediaType = new _AMMediaType();
    return false;
}

(where MEDIATYPE_Video is the GUID used for video files).

private static Size getVideoSize(_AMMediaType mediaType)
{
    WinStructs.VIDEOINFOHEADER videoInfo = 
        (WinStructs.VIDEOINFOHEADER)Marshal.PtrToStructure(mediaType.pbFormat, 
            typeof(WinStructs.VIDEOINFOHEADER));
    
    return new Size(videoInfo.bmiHeader.biWidth, videoInfo.bmiHeader.biHeight);
}

Using GetBitmapBits to avoid saving the image on disk is a bit trickier, since we need to deal with direct access to memory.

The first part is identical to SaveFrameFromVideo, then we have to call GetBitmapBits with the pBuffer parameter set to null to get the size for the buffer of bytes that will contain the 24bpp image (GetBitmapBits always returns 24bpp images).

Once we have the size of the buffer, we allocate memory on the heap to receive the image (in the first version of this code, memory was allocated on the stack which is fine if the target image is small sized, but if it is big we may get a nice StackOverflowException because stack memory is rather limited).

After this, we call GetBitmapBits again, but this time the buffer will be filled with image bytes. Now we create a bitmap from these bytes (remembering that they start with BITMAPINFOHEADER structure, the size of which is 40 bytes).

public static Bitmap GetFrameFromVideo(string videoFile, 
            double percentagePosition, out double streamLength, Size target)
{
    if (percentagePosition > 1 || percentagePosition < 0)
        throw new ArgumentOutOfRangeException("percentagePosition", 
                percentagePosition, "Valid range is 0.0 .. 1.0");

    try 
    {
        MediaDetClass mediaDet;
        _AMMediaType mediaType;
        if (openVideoStream(videoFile, out mediaDet, out mediaType))
        {
            streamLength = mediaDet.StreamLength;

            //calculates the REAL target size of our frame
            if (target == Size.Empty)
                target = getVideoSize(mediaType);
            else
                target = scaleToFit(target, getVideoSize(mediaType));

            unsafe 
            {
                Size s= GetVideoSize(videoFile);
                //equal to sizeof(CommonClasses.BITMAPINFOHEADER);
                int bmpinfoheaderSize = 40;                 

                //get size for buffer
                int bufferSize = (((s.Width * s.Height) * 24) / 8 ) + bmpinfoheaderSize;
                //equal to mediaDet.GetBitmapBits
                //    (0d, ref bufferSize, ref *buffer, target.Width, target.Height);    

                //allocates enough memory to store the frame
                IntPtr frameBuffer = 
                    System.Runtime.InteropServices.Marshal.AllocHGlobal(bufferSize);
                byte* frameBuffer2 = (byte*)frameBuffer.ToPointer();

                //gets bitmap, save in frameBuffer2
                mediaDet.GetBitmapBits(streamLength * percentagePosition, 
                    ref bufferSize, ref *frameBuffer2, target.Width, target.Height);

                //now in buffer2 we have a BITMAPINFOHEADER structure 
                //followed by the DIB bits
                Bitmap bmp = new Bitmap(target.Width, target.Height, target.Width * 3, 
                    System.Drawing.Imaging.PixelFormat.Format24bppRgb, 
                    new IntPtr(frameBuffer2 + bmpinfoheaderSize));

                bmp.RotateFlip(RotateFlipType.Rotate180FlipX);
                System.Runtime.InteropServices.Marshal.FreeHGlobal(frameBuffer);
                return bmp;
            }
        }
    }
    catch (COMException ex)
    {
        throw new InvalidVideoFileException(getErrorMsg((uint)ex.ErrorCode), ex);
    }

    throw new InvalidVideoFileException("No video stream was found");
}

Known Limitations

  • The biggest one is the StackOverflowException using stackalloc. There must be a way to pass to GetBitmapBits a buffer created on the heap, but I'm not yet very good with this unmanaged stuff.
    • This problem has been resolved. The solution is provided by _coder_ in the comments below.
  • On my machine, I got random errors when passing certain target sizes to GetBitmapBits: when the target size is 125x125 for example, the Bitmap constructor fails.
    • Only sizes multiples of 4 are allowed. Thanks to ujr (see comments).
  • "The IMediaDet interface does not support VIDEOINFOHEADER2 formats": This means it cannot open some *.mpeg video files.

History

  • 27th February, 2006: Initial release
  • 17th March, 2006: Replaced memory allocation on the stack
  • 27th September, 2007: Some fixes

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author


You may also be interested in...

Comments and Discussions

 
BugVIDEOINFOHEADER2 won't initialize correctly in x64 Pin
Member 997965110-Aug-15 13:11
memberMember 997965110-Aug-15 13:11 
Questioncan we extract key frame using this? Pin
Member 110823658-Nov-14 22:34
memberMember 110823658-Nov-14 22:34 
Questiondoes not works if Pin
AbdEllah Gogop21-Oct-14 0:30
memberAbdEllah Gogop21-Oct-14 0:30 
QuestionMP4 Pin
vincent6420-Jun-14 0:05
membervincent6420-Jun-14 0:05 
GeneralMy vote of 1 Pin
GMBarak1-Apr-14 12:04
memberGMBarak1-Apr-14 12:04 
QuestionCrash Pin
jeff parnau12-Nov-13 2:59
memberjeff parnau12-Nov-13 2:59 
QuestionMP4 format file not working Pin
Sabyasachi Maity15-May-13 22:57
memberSabyasachi Maity15-May-13 22:57 
QuestionMaximum number of accesses before it returns error Pin
hariprasad1020-Nov-12 17:08
memberhariprasad1020-Nov-12 17:08 
GeneralMy vote of 1 Pin
Member 94896058-Oct-12 8:37
memberMember 94896058-Oct-12 8:37 
Questionasking for flv video image extract in c# Pin
Member 927580727-Sep-12 19:24
memberMember 927580727-Sep-12 19:24 
Questionmp4 files Pin
igorchirok9-Jul-12 9:50
memberigorchirok9-Jul-12 9:50 
QuestionGetting an error when using example Pin
Munken_0622-Jun-12 0:07
memberMunken_0622-Jun-12 0:07 
GeneralMy vote of 5 Pin
Joe Sonderegger15-May-12 8:49
memberJoe Sonderegger15-May-12 8:49 
QuestionPossibe to extract from MKV file? Pin
babzog25-Apr-12 7:39
memberbabzog25-Apr-12 7:39 
QuestionExtracting all frames but only first frame is extracted from .avi file Pin
farid vanzara121-Apr-12 2:32
memberfarid vanzara121-Apr-12 2:32 
QuestionWhy does Target size must be a multiple of 4 ? Pin
rolbackse22-Jan-12 18:39
memberrolbackse22-Jan-12 18:39 
GeneralDo not use this Pin
Cuchuk Sergey21-Dec-10 9:21
memberCuchuk Sergey21-Dec-10 9:21 
GeneralRe: Do not use this Pin
Cuchuk Sergey21-Dec-10 10:11
memberCuchuk Sergey21-Dec-10 10:11 
GeneralDexterlib (distribution or source location) Pin
Member 39968537-Dec-10 15:05
memberMember 39968537-Dec-10 15:05 
GeneralSoftware is not supporting to mpg Pin
vijayr18-Nov-10 20:17
membervijayr18-Nov-10 20:17 
GeneralCapture frame at given time within video Pin
akula7702-Jun-10 6:39
memberakula7702-Jun-10 6:39 
GeneralJockerSoft.Media.dll Pin
Saira138-Apr-10 6:18
memberSaira138-Apr-10 6:18 
QuestionHow to get it working on 64-bit CPU Pin
Dmitri Nesteruk25-Aug-09 4:45
memberDmitri Nesteruk25-Aug-09 4:45 
Questionusage of DLL in commercial product -- any patent issues? Pin
rtswguru4-May-09 8:35
memberrtswguru4-May-09 8:35 
AnswerRe: usage of DLL in commercial product -- any patent issues? Pin
akula7703-Jun-10 9:01
memberakula7703-Jun-10 9:01 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web04 | 2.8.160207.1 | Last Updated 27 Sep 2007
Article Copyright 2006 by JockerSoft
Everything else Copyright © CodeProject, 1999-2016
Layout: fixed | fluid