Figure 1. Screenshot of the xnaVideoViewer application in action
"Video textures" are an important class of multimedia objects with interesting uses in video conferencing, visualization, gaming, robotics and image analysis. The most obvious use -- and therefore the usage that I have focused on in this project -- is to map streaming video from a webcam onto a
Texture2D object. This is then applied to a sprite and presented as a live video window within an XNA application. In addition to consuming live video from any of your DirectX video devices, the
VideoTexture class is also able to consume AVI, MPEG and WMV format video clips.
VideoTexture class exposes a property called
VideoTexture2D, which is intended to be usable wherever a normal
Texture2D object might be used within XNA. This means that it can be used to apply a dynamic video texture onto both sprites and geometric meshes/models. It can also be manipulated by pixel, vertex and geometry shaders, just like any other texture.
Using the code
To use the
VideoTexture class, you should:
- Copy the VideoTexture.cs file into your own project.
- Change the namespace in the VideoTexture.cs file to the name of your project namespace.
- Add a reference to DirectShowLib-2005.dll from the DirectShow.NET library.
- Compile it. The
VideoTexture class should now be available in your project.
XnaVideoViewer application demonstrates the usage of
VideoComponent, which in turn demonstrates the usage of the
Points of interest
VideoTexture class depends upon the managed DirectShow.NET library.
VideoTexture.VideoTexture2D property can be polled by other classes to get the most current frame of video. This is what happens, for example, in the
VideoComponent.Draw() method. Alternatively, classes that want to use
VideoTexture can subscribe to
VideoTexture.NewVideoTextureEvent and have the current frame of video pushed to them in
NewVideoTextureEventArgs each time a new frame is grabbed from the video stream.
VideoComponent automatically registers itself as a service within the game application that uses it. It can be referenced through the Services collection via any other components used by the application.
- Due to framework limitations within XNA -- specifically, the absence of the
System.Windows namespace -- the desired source of video input (i.e. cameras or files) needs to be coded manually in the
xnaVideoViewer application. This is done in the
xnaVideoViewer application maps the following commands to the XBox 360 controller: Run = A (Green); Pause = Y (Yellow); Stop = B (Red); Exit = Back.
VideoTexture class has been used to provide the video stream for an experimental HUD application. It has recently been used to stream video textures through vertex shaders to correct lens distortions in real-time as a prelude to stitching the streams into a video mosaic.
- 30 July 2007 - Version 184.108.40.206 uploaded to CodeProject. Changes include renaming and significant refactoring of the
VideoTexture classes. Support for AVI, MPEG and WMV video files was added. Support for Run, Pause and Stop was added. Support for Looping video clips was added. Two separate methods for capturing and handling DirectShow Filter Graph events were added. Additional error handling code was added.
- 21 July 2007 - Version 220.127.116.11 uploaded to CodeProject.
Stephen Bogner is a Senior Research Engineer with Defence R&D Canada. As the Head Autonomous Applications Group, Autonomous Intelligent Systems Section, he only programs when it can't be avoided, and then only in C#.