VideoTexture Class and Webcam Application for XNA

• Download XnaVideo executables - 100.6 Kb
• Download XnaVideo source - 25.2 Kb
• Download MSDN-style XnaVideo documentation - 65.8 Kb

Figure 1. Screenshot of the xnaVideoViewer application in action

Introduction

"Video textures" are an important class of multimedia objects with interesting uses in video conferencing, visualization, gaming, robotics and image analysis. The most obvious use -- and therefore the usage that I have focused on in this project -- is to map streaming video from a webcam onto a Texture2D object. This is then applied to a sprite and presented as a live video window within an XNA application. In addition to consuming live video from any of your DirectX video devices, the VideoTexture class is also able to consume AVI, MPEG and WMV format video clips.

The VideoTexture class exposes a property called VideoTexture2D, which is intended to be usable wherever a normal Texture2D object might be used within XNA. This means that it can be used to apply a dynamic video texture onto both sprites and geometric meshes/models. It can also be manipulated by pixel, vertex and geometry shaders, just like any other texture.

Using the code

To use the VideoTexture class, you should:

1. Copy the VideoTexture.cs file into your own project.
2. Change the namespace in the VideoTexture.cs file to the name of your project namespace.
3. Add a reference to DirectShowLib-2005.dll from the DirectShow.NET library.
4. Compile it. The VideoTexture class should now be available in your project.

The XnaVideoViewer application demonstrates the usage of VideoComponent, which in turn demonstrates the usage of the VideoTexture class.

Points of interest

• The VideoTexture class depends upon the managed DirectShow.NET library.
• The VideoTexture.VideoTexture2D property can be polled by other classes to get the most current frame of video. This is what happens, for example, in the VideoComponent.Draw() method. Alternatively, classes that want to use VideoTexture can subscribe to VideoTexture.NewVideoTextureEvent and have the current frame of video pushed to them in NewVideoTextureEventArgs each time a new frame is grabbed from the video stream.
• VideoComponent automatically registers itself as a service within the game application that uses it. It can be referenced through the Services collection via any other components used by the application.
• Due to framework limitations within XNA -- specifically, the absence of the System.Windows namespace -- the desired source of video input (i.e. cameras or files) needs to be coded manually in the xnaVideoViewer application. This is done in the LoadComponents() method.
• The xnaVideoViewer application maps the following commands to the XBox 360 controller: Run = A (Green); Pause = Y (Yellow); Stop = B (Red); Exit = Back.
• The VideoTexture class has been used to provide the video stream for an experimental HUD application. It has recently been used to stream video textures through vertex shaders to correct lens distortions in real-time as a prelude to stitching the streams into a video mosaic.

History

• 30 July 2007 - Version 1.1.0.0 uploaded to CodeProject. Changes include renaming and significant refactoring of the xnaVideoViewer, VideoComponent and VideoTexture classes. Support for AVI, MPEG and WMV video files was added. Support for Run, Pause and Stop was added. Support for Looping video clips was added. Two separate methods for capturing and handling DirectShow Filter Graph events were added. Additional error handling code was added.
• 21 July 2007 - Version 1.0.0.0 uploaded to CodeProject.

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)