Video File Saving in Windows Media Video Format for the DirectX.Capture Class Library

• Download demo project - 62.2 KB
• Download source - 129.89 KB

Introduction

This article is a follow-up of my previous article Audio File Saving for the DirectX.Capture Class. This article describes the saving of video in Windows Media file format (WMV). Saving video is more complex than saving audio. This is because saving video can mean saving video with and without audio. Because of that, there is a bigger chance that video saving fails due to a conflict with the selected save format. Furthermore, there are many more predefined video formats than audio formats, so it would be nice to have a user friendly solution of selecting a specific video format that suits your needs the best.

The DirectX.Capture class example was a big help for me in finding out how video file saving in the Windows Media format could be done. Since the DirectX.Capture class example supports AVI file saving only, providing Windows Media file saving is an interesting enhancement. This because the files may become (much) smaller. The article C# Windows Media Format SDK Translation by Idael Cardoso gave me insightful information on the Windows Media Format SDK. It took much effort to find out how the IWMProfile interface could be used for retrieving useful information. This article will provide an example of how Windows Media can be used in an application.

Using Profiles...

Why use profiles? Well, each Windows Media format is represented by a profile. There is a long list of system profiles that can be used for Windows Media. What's interesting is that it is also possible to make a profile or modify an existing profile. The system profiles can be found in the file WMSysPrx.prx in the Windows directory. A PRX file contains the description of the profile in XML format. Maybe you already knew, but Microsoft provides in the Windows Media Encoder software package with a Windows Media profile editor named WMProEdt.exe. It's worth looking into that. Here's part of a Windows Media Encoder profile:

<profile version="589824"
        storageformat="1"
        name="Higher quality video (VBR 97)"
        description="">
            <streamconfig majortype="{73646976-0000-0010-8000-00AA00389B71}"
                        streamnumber="1"
                        streamname="Video Stream"
                        inputname="Video409"
                        bitrate="100000"
                        bufferwindow="-1"
                        reliabletransport="0"
                        decodercomplexity=""
                        rfc1766langid="en-us"
                        vbrenabled="1"
                        vbrquality="97"
                        bitratemax="0"
                        bufferwindowmax="0">
            <videomediaprops maxkeyframespacing="80000000"
                        quality="0"/>
            <wmmediatype subtype="{33564D57-0000-0010-8000-00AA00389B71}"
                        bfixedsizesamples="0"
                        btemporalcompression="1"
                        lsamplesize="0">
                        <videoinfoheader dwbitrate="100000"
                                dwbiterrorrate="0"
                                avgtimeperframe="333667">
                ...

</profile> 

It is an XML file containing the name, the description, the Windows structure names, the attribute names and their values of the profile. So, a profile provides all details about the supported audio and video streams. A profile is used to configure the ASF file writer. The ASF file writer is the filter that will handle the actual multiplexing, the encoding of the video and/or audio and the actual file saving! By configuring the ASF file writer, the video (and/or audio) will be saved in the chosen format. Much more importantly for me, however, the profile can also be used as an information source for a user friendly solution of selecting a Windows Media file format.

A good start for learning more about Windows Media and profiles is to read about the Windows Media Format SDK at MSDN. Just search for this article, the link is updated quite often. This SDK contains samples, header files, programs and documentation. The good news is that the Windows Media Format SDK can also be downloaded.

Problems with Using the ASF File Writer

In my previous article about audio saving, Audio File Saving for the DirectX.Capture Class, I mentioned that the ASF file writer gives conflicts when it is added to the graph, but is not connected. Because of that, I could not use the default property window for selecting a Windows Media format for file saving. A new form is needed to provide similar functionality as the default property window. That is not too bad because now more information can be shown about the profile itself than the default property window shows. For example, information about supporting audio and video, the bit rate and the description of a profile can be shown. It is even possible to show the frame size, but that is some more work because the frame size is a little bit harder to retrieve. To have the specific profile information is handy because you will know in advance if audio and/or video can be saved. Another good reason is that the program can use this information also. If a selected format does not match the rendered audio and/or video streams to the file writer, the program can detect that and can give a warning.

IWMProfile

To get a profile, the IWMProfile interface needs to be used. However, more interfaces are needed. First, the profile manager (IWMProfileManager and IWMProfileManager2 interfaces) needs to be initialized. Then the Windows Media version needs to be set; Windows Media version 8 will be used here. The IWMProfileManager2 interface is used because via this interface the profiles can be retrieved using GetSystemProfileCount(), LoadProfileByID() and LoadSystemProfile().

If the profile is found, then the name can be retrieved via GetName() and the description can be retrieved via GetDescription(). To get the audio and video stream information, some more actions are needed. With GetStreamCount(), the number of streams can be retrieved. With GetStreamByNumber(), the stream configuration information can be retrieved (IWMStreamConfig interface). Then it is possible to check for each stream if it is an audio or an video stream. The frame size is more difficult; for this, the video info header information (VideoInfoHeader) is needed. There is no direct interface that retrieves that information, but there is a workaround: use the IWMWriter interface.

IWMWriter

The IWMWriter interface gives access to the Windows Media writer. This writer can be used for writing audio and video streams to file. While the ASF file writer is a filter, the Windows Media writer is not a filter and that is the big difference. Why use the Windows Media writer for this? Well, it can take the same profile as the ASF file writer and can save files in the same format. Furthermore, the Windows Media writer can access the video info header information indirectly via an input property. An input property can be retrieved via GetInputProps() of the Windows Media writer.

If a video input property is found, then via GetType() and GetMediaType() of the IWMInputMediaProps, the video information can be retrieved and stored in a AMMediaType structure. With the data in the AMMediaType data structure, the VideoInfoHeader can be accessed, if available. Because of its complexity and the additional research of releasing allocated COM objects, the use of IWMWriter is not supported in this example yet.

Another interesting feature might be the use of own-made profiles. Using the StreamReader, IWMProfileManager and IConfigAsfWriter2 interfaces with the functions LoadProfileByData and ConfigureFilterUsingProfile, it is possible to load your own profile. An example is described in the "Fun with DVR-MS" MSDN article written by Stephen Toub.

Using the Code

For saving video in a Windows Media format, four new classes are introduced: WMLib, WMProfileData, AsfFormat and AsfForm. The class WMProfileData is used to store profile data. The class AsfFormat is used to store the profiles, using the class WMProfileData. The class AsfFormat has a number of functions that will be used in this coding example. The class AsfForm enables the user to select a profile. The class AsfForm uses AsfFormat to store and retrieve profile information. The class WMLib provides the Windows Media Format SDK interfaces that are really needed in this coding example.

I wrote the WMLib class because I encountered problems with the Windows Media Format SDK described in the article C# Windows Media Format SDK Translation by Idael Cardoso. When I finished the example, I did not spend time on using the Yeti Windows Media SDK again. Feel free to make it work and maybe it will work right away. The main problem is that the libraries are mostly untested. I did not want to use untested interfaces that are not needed in the DirectX.Capture class example. Also, some structures and interfaces looked very similar to the structures and interfaces in DShowNet and this confuses me at least. So, I could not make a decision about which library could be used best and I made my own version. Because of that, the code in the WMLib.cs file might look quite similar to the Yeti version and the unofficial Sourceforge Windows Media library at CVS: directshownet/windowsmedialib, Sourceforge.net.

WMProfileData

The class WMProfileData declares the attributes that hold the data of a profile. The application can use this data to retrieve, for example, the name of the profile, its description and some stream information.

public class WMProfileData
{
    /// <summary> Name of the profile </summary>
    protected string name;

    /// <summary> Guid of the profile </summary>
    protected Guid guid;

    /// <summary> Description of the profile </summary>
    protected string description;

    /// <summary> Audio bit rate </summary>
    protected int audioBitrate;

    /// <summary> Video bit rate </summary>
    protected int videoBitrate;

    /// <summary> Indicates whether this profile supports an audio
    /// stream
    /// </summary>

    protected bool audio;

    /// <summary> Indicates whether this profile supports an video
    /// stream
    /// <summary>
    protected bool video;

    /// <summary> Indicates whether this profile is the one currently in
    /// use </summary>
    protected bool enabled;
}

AsfFormat

This class uses the WMProfileData class to build a list of profile information items. The next sample shows how the stream type and the bit rate are retrieved.

hr = profile.GetStreamCount(out streamCount);

if((hr >= 0)&&((streamCount > 0))
{
    IWMStreamConfig streamConfig = null;
    Guid streamGuid = Guid.Empty;
    audio = false;
    video = false;
    audioBitrate = 0;
    videoBitrate = 0;

    for(short i = 1;(i <= streamCount)&&(hr >= 0); i++)
    {
        hr = profile.GetStreamByNumber(i, out streamConfig);
        if((hr >= 0)&&(streamConfig != null))
       {
            hr = streamConfig.GetStreamType(out streamGuid);
            if(hr >= 0)
            {
                if(streamGuid == MediaType.Video)
                {
                    video = true;
                    hr = streamConfig.GetBitrate(out videoBitrate);
                    if(hr < 0)
                    {
                        videoBitrate = 0;
                    }
                }
                else
                    if(streamGuid == MediaType.Audio)
                {
                    audio = true;
                    hr = streamConfig.GetBitrate(out audioBitrate);
                    if(hr < 0)
                    {
                        audioBitrate = 0;
                    }
                }
                hr = 0; // Allow possible unreadable bitrates
            }
        }
    } // for i
}

The AsfFormat class has a number of functions that can be used by the application. With the constructor, the class can be initialized. With the function GetProfileFormatInfo(), the profile information can be updated (e.g. show all video formats or show all video formats having no audio). This functionality is very useful if the application switches from audio file saving to video file saving. For video file saving, the menu should show a list of profiles that do support at least a video stream. In contrast, for audio file saving, the menu should show a list of profiles that support an audio stream and do not support a video stream.

AsfForm

The AsfForm class provides a form that gives (useful) information about the selected profile. In addition, the user is able to select a different profile. The class AsfForm is added as file AsfForm.cs to CaptureTest.

Code Changes in Capture.cs

The following code shows possible use of the AsfFormat constructor and the GetProfileFormatInfo() function in the Capture class.

switch(recFileMode)
{
    case RecFileModeType.Wmv:
        if(asfFormat == null)
        {
          asfFormat = new AsfFormat(AsfFormat.AsfFormatSelection.Video);
        }
        else
        {
          asfFormat.GetProfileFormatInfo(AsfFormat.AsfFormatSelection.Video);
        }
        break;
    case RecFileModeType.Avi:
        break;
    default:
        // Unsupported file format
        return;
}

In addition, there is a one-liner that changes the extension of the file name.

// Change filename extension
this.filename = Path.ChangeExtension(this.filename,
                                     RecFileMode.ToString().ToLower());

In the function renderGraph(), the actual file saving is initialized. The original AVI file saving is still possible. To prevent problems with the Windows Media formats, the video compressor is used for AVI file saving only. For a Windows Media format, no video compressor will be used. For a Windows Media format, the profile information must be retrieved to configure the ASF file writer. Furthermore, the stream information must be retrieved.

// Render the file writer portion of graph (mux -> file)
// Record captured audio/video in Avi, Wmv or Wma format
Guid mediaSubType; // Media sub type

bool captureAudio = true;
bool captureVideo = true;
IBaseFilter videoCompressorfilter = null;

// Set media sub type and video compressor filter if needed
if(RecFileMode == RecFileModeType.Avi)
{
    mediaSubType = MediaSubType.Avi;
    // For Avi file saving a video compressor must be used
    // If one is selected, that one will be used.
    videoCompressorfilter = videoCompressorFilter;
}
else
{
    mediaSubType = MediaSubType.Asf;
}
// Initialize the Avi or Asf file writer
hr = captureGraphBuilder.SetOutputFileName( ref mediaSubType, Filename,
                                     out muxFilter, out fileWriterFilter );
if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );
// For Wma (and Wmv) a suitable profile must be selected. This
// can be done via a property window, however the muxFilter is
// just created. if needed, the property windows should show up
// right now!
// Another solution is to configure the Asf file writer, the
// use interface must ensure the proper format has been selected.
if((RecFileMode == RecFileModeType.Wma)||
    (RecFileMode == RecFileModeType.Wmv))
{
    if(this.AsfFormat != null)
    {
        this.AsfFormat.UpdateAsfAVFormat(this.muxFilter);
        this.AsfFormat.GetCurrentAsfAVInfo(out captureAudio,
                                            out captureVideo);
    }
}

For saving video, the video stream check is performed. This check will prevent video file saving if an audio-only file save was requested. For AVI only, the video compressor value will be used.

// Render video (video -> mux) if needed or possible
if((VideoDevice != null)&&(captureVideo))
{
    // Try interleaved first, because if the device supports it,
    // it's the only way to get audio as well as video
    cat = PinCategory.Capture;
    med = MediaType.Interleaved;
    hr = captureGraphBuilder.RenderStream( ref cat, ref med,
                       videoDeviceFilter, videoCompressorfilter, muxFilter );
    if( hr < 0 )
    {
        med = MediaType.Video;
        hr = captureGraphBuilder.RenderStream( ref cat, ref med,
                       videoDeviceFilter, videoCompressorfilter, muxFilter );
        if ( hr == -2147220969 )
            throw new DeviceInUseException( "Video device", hr );
        if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );
    }
}

For saving audio, the audio stream check is performed. This check will prevent audio file saving if a video-only file save was requested.

// Render audio (audio -> mux) if possible
if((AudioDevice != null)&&(captureAudio))
{
    // If this Asf file format than please keep in mind that
    // certain Wmv formats do not have an audio stream, so
    // when using this code, please ensure you use a format
    // which supports audio!
    cat = PinCategory.Capture;
    med = MediaType.Audio;
    hr = captureGraphBuilder.RenderStream( ref cat, ref med,
                    audioDeviceFilter, audioCompressorFilter, muxFilter );
    if( hr < 0 )
         Marshal.ThrowExceptionForHR( hr );
}
isCaptureRendered = true;
didSomething = true;

Code Changes in CaptureTest.cs

In CaptureTest.cs, the updateMenu() function needs to be modified for adding a menu with the possible audio/video recording modes. A new function menuAVRecFileModes_Click() is added; this function is called when the user selects an item in the menu.

menuAVRecFileModes.MenuItems.Clear();
// Fill in all file modes, use enumerations also as string (and file
// extension)
for(int i = 0; i < 3; i++)
{
    m = new MenuItem(((DirectX.Capture.Capture.RecFileModeType)i).ToString(),
                     new EventHandler(menuAVRecFileModes_Click));
    m.Checked = (i == (int)capture.RecFileMode);
    menuAVRecFileModes.MenuItems.Add(m);
}
menuAVRecFileModes.Enabled = true;

Points of Interest

This example has additional support to get audible sound when using a TV card. It provides audio via the PCI bus, so no wired audio connection is involved. During testing, I noticed that there were often no audio sources listed. If the sources were available, I noticed that the sources were invalid, so a source could not be selected. For the last problem, I found two solutions. The first solution was to reload AudioSources and VideoSources and then reload AudioSource and VideoSource. The second solution was to not release the crossbar object via Marshal.ReleaseComObject in CrossbarSource.cs. Both solutions look bad. In this example, the second solution was chosen because of its simplicity. To get the list of AudioSources, a few more modifications were needed. These modifications are needed only when AudioSources cannot be found via the audio device or the audio device is found via the video device.

I also did some testing with saving files in Mpeg2 format. There are already some interesting articles on this subject, Preview and Record with MPEG2 Capture Device and Working with Stream Buffer Engine - TIME SHIFT on Windows XP Service Pack 1 written by BeCapture. The second article gives a very good explanation of how the Mpeg2 demultiplexer should be configured. Still, I could not get the last example working. Looking at some of the problems I got, I am wondering if this is a working example. Nevertheless, these examples helped me to get a working C# solution. The concern I have is that the solution depends very much on the Mpeg2 encoder and decoder filters that are available. Graphedt is a big help in checking whether a combination may work, but still it is tough to get a good result. Sometimes a design seems to work, but the file stays zero bytes. Sometimes the property page information of a filter needs to be modified to get a good result. The easiest one to use for file saving that I found was a Nero audio/video encoder. Using a hardware Mpeg2 encoder is also not too difficult. More difficult is finding a proper dump filter. For the moment, I do not think an article on this subject will give new information. However, I can imagine that for pulling the pieces together, some help will be very useful. So, if you have questions on this subject, please let me know.

During testing, I did some investigation of the good old GetCurFile() function that is part of the (ASF) file writer filter. Via this function, the filename and the video information can be retrieved. The filename is returned in a string and the video information in a AMMediaType structure. More information on AMMediaType can be found at Specifying Stream Formats. When checking the formatPtr, formatSize and formatType attributes, I found out that the information seemed to be unusable. I checked this function because I had the idea that maybe I could access VideoInfoHeader via this interface. That would simplify the code a little bit and the IWMWriter interface would not be needed. Unhappily, the format data does not look usable. For changing the filename in this example, the video information is not needed. I also noticed (on MSDN) that GetCurFile() might use a null pointer. According to me, using null instead of the AMMediaType structure with the formatPtr looks much safer.

I noticed that the DirectX.Capture class example (and so also my version of this example) usually gives a good picture when the program is started and the preview is selected for the first time. I do not know what is really causing the slow-down of the video preview (usually a black picture). What I noticed is that problems occur when mediaControl.Stop is called. Actually, the preview pin seems to be handled a little bit differently than the capture pin. Using the capture pin instead of the preview pin gives the best result. Here's a sample of the original implementation for rendering video to the default video renderer.

cat = PinCategory.Preview;
med = MediaType.Video;
hr = captureGraphBuilder.RenderStream( ref cat, ref med, videoDeviceFilter,
                                       null, null );
if( hr < 0 )
     Marshal.ThrowExceptionForHR( hr );

Unfortunately, the file save functionality needs the capture pin. Using Video Mixing Renderer 9 for video rendering or using a lower video preview resolution will quite often also give a good picture. Here is an example for using Video Mixing Renderer 9 as the video renderer.

cat = PinCategory.Preview;
med = MediaType.Video;
IBaseFilter VMRfilter = (IBaseFilter) new VideoMixingRenderer9();
hr = graphBuilder.AddFilter(VMRfilter, "Video mixing renderer 9");
if( hr < 0 )
{
    Marshal.ThrowExceptionForHR( hr );
}
hr = captureGraphBuilder.RenderStream(ref cat, ref med, videoDeviceFilter,
                                      null, VMRfilter);
if( hr < 0 )
{
    Marshal.ThrowExceptionForHR( hr );
}

I did not find a solution that worked on all my capture cards, i.e. Hauppauge PVR150, Pinnacle PCTV, MX460 Vivo, Radeon 8500 Vivo. The Hauppauge cards in particular caused me problems. On one system it works great, but on another system I usually get a black picture.

The code example has been tested with Visual Studio 2003 as well as Visual Studio 2005. Conflicts between these two compiler versions might occur. I added the conditional VS2003 to show the code differences. A difference was that in Visual Studio 2003, a signal was named Closed, while in Visual Studio 2005, this signal had the name FormClosed. In this code example, I removed the line of code that specifies the function needing to be called when closing the main form (but okay, that line should be put back). But there is more, the form specific code is not the same for Visual Studio 2003 and Visual Studio 2005, the code is generated by Visual Studio. If both versions are needed, then put them in a different directory! It is not my intention to solve these types of conflicts.

Feedback and Improvements

I hope this code helps you in understanding the structure of the DirectX.Capture class. I also hope I provided you with an enhancement that might be useful to you. Feel free to post any comments and questions.

History

• April 19, 2006: First release
• February 1, 2007: Added support for capturing audio via video device filter and added TV fine-tuning. The article DirectShow - TV Finetuning using the IKsPropertySet in C# describes these modifications in more detail.
• August 1, 2007: Added support for FM Radio and video de-interlacing. The solution of capturing audio via the video device filter has been improved. Furthermore, the code example supports either DShowNET or DirectShowLib as the interface library via the conditional DSHOWNET. The DirectX.Capture class example uses DShowNET and the DirectX.Capture Class Library (Refresh) uses DirectShowLib, which is more complete than DShowNET. It is up to you what to use. A more detailed description of these modifications and the code example can be found in the article DirectShow - TV Finetuning using the IKsPropertySet in C#.
• August 10, 2007: Added SampleGrabber and Video Mixing Renderer 9 support in the extra code example. For now, this extra code example is described in the article DirectShow - TV Finetuning using the IKsPropertySet in C# as an extra code example.
• November 28, 2007: Fixed minor bug in source download
• February 17, 2008: Minor text changes and links corrected