Click here to Skip to main content
11,477,834 members (72,212 online)
Click here to Skip to main content

Prototyping DirectShow filters in C#

, 14 Dec 2005 204.6K 2.9K 64
Rate this:
Please Sign up or sign in to vote.
This article shows how to implement a DirectShow filter in C#.

Sample Image

Introduction

DirectShow filters are the basic building blocks of multimedia applications on the Windows platform. Normally, they are written in C++. In this article, I'll show how to implement a couple of DirectShow transformation filters in C#. Since Microsoft doesn't recommend using managed code for filters, I'll call them prototypes but they perform quite well in most circumstances. We'll develop Sobel transformation as a DirectShow filter. Sobel transformations are often used for video segmentation which is the process of identifying objects in a scene.

Background

The strategy we use to implement a DirectShow filter in C# relies on having a C++ class that contains a pointer to a managed object which implements our filter. The C++ class just forwards all the processing to the managed object. The C++ class is generated using Cutler's DirectShow Filters Wizard. We just clean up the wizard-generated code to include a pointer to our C# object and implement all the filter processing as methods of the managed object.

I've written this Sobel filter to have an idea of the kind of performance we'd get with C#. If you want to "take your CPU for a spin", this is a good test. As written (i.e., with no optimization effort), there are around 20 integer multiplications per pixel. Usually, you'd transform an image to black and white before applying Sobel transformation to the image. So I've also written a transformation filter that converts a RGB24 bitmap to black and white.

Using the code

If you just want to have a filter that implements Sobel transformations, you can download and install the managed DLL and Ax files (from the download link). Then after registering the Ax file and installing the managed DLL in the Global Assembly Cache (GAC), you can start the GraphEdit utility and add the "CsSobelV2" filter (found in the DirectShow filters category of GraphEdit). Then render a media file to see a black and white "sketch" of the frames in the video. (Note that the open-source DirectShowLib must also be in the GAC.) More information can be found in the "readme.txt" file that is included in the download.

If you want to implement your own filter using these filters as a starting point, you can modify the Transform and ProcessFrame methods of the MySobelV2 class. This is sufficient if you are satisfied with a single media type for the video frame, here Rgb24, and you can live with the default memory allocator of the C++ DirectShow base classes (this covers a lot of situations).

Sobel transformation

The Sobel transformation uses the following two matrices:

  // define the horizontal Sobel mat
  int [,] hx = new  int [,] { {-1, 0, 1}, 
                    {-2, 0, 2}, {-1, 0, 1} };

  // define the vertical Sobel mat
  int[,] hy = new int [,]  { { 1,  2,  1}, 
                  { 0,  0,  0}, {-1, -2, -1} };

And consists of the following code, found in ProcessFrame, which is repeated for every pixel in every frame of a video:

        gradX = 0;
        gradY = 0;
        for (int row = -1; row <= 1; row++)
        {
          for (int col = -1; col <= 1; col++)
          {
            position = GetPosition( x + col, y + row ); 
            intensity = pwSource[position];
            gradX += (intensity * hx[col + 1,row + 1]);
            gradY += (intensity * hy[col + 1,row + 1]); 
          }
        }
         
        // compute the square of the gradient        
        gradMag = gradX * gradX + gradY * gradY;

        // threshold default at 128
        gradMag = (gradMag < threshold*threshold) ? 255 : 0;
               
        // assign the pixel to the destination
        position = GetPosition( x, y );
        pwTarget[position] = pwTarget[position+1] = 
              pwTarget[position+2] = (byte)gradMag;

The intuition behind the transformation is relatively simple. Since the contour of objects consists of a region in a picture where the color changes rapidly, the Sobel transformation amplifies these differences and simply sets to white those pixels whose changes are too small (and black, the other ones). The horizontal Sobel transformation looks at the left and right neighboring pixels and multiplies these values by 2 or -2, it also looks at the upper right and left, and lower right and left pixels. After summing all these values, you compute the square of the Euclidian distance and use an arbitrary threshold to divide the values.

Points of Interest

I've also included the code for a black and white transformation filter for uncompressed Rgb24 video frames. As mentioned, you'd normally perform this transformation before feeding the result to the Sobel filter. So you can test the performances of a filter graph with two filters written in managed code.

Property pages for filters are handy when used in a utility like GraphEdit. But being able to access custom filter properties from C# (easily) is even more useful. So I've modified Cutler's Wizard-generated C++ code to include a property that can be changed in C#. Similar changes could be done to implement different properties.

Limitations and known issues

The media type accepted by the filters is uncompressed Rgb24 video. Since this is one of the most "natural" media types, it's not a big limitation but the filters were not designed to be very flexible. They are just prototypes to experiment with.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author

daniel049

United States United States
You can read my blog entries at:
http://wwww.informikon.com/blog/

Comments and Discussions

 
GeneralThere Are Several NON-Commercial RTSP Procol Source Filters! Pin
GUI Developer26-Oct-08 5:37
memberGUI Developer26-Oct-08 5:37 
QuestionAbstract Class Error Pin
nickmaggie31-Aug-07 4:56
membernickmaggie31-Aug-07 4:56 
AnswerRe: Abstract Class Error Pin
daniel0491-Sep-07 10:44
memberdaniel0491-Sep-07 10:44 
GeneralCompiling on VS 2003 version 7.1 Framework 1.1 Pin
Russ531-Aug-07 9:01
memberRuss531-Aug-07 9:01 
GeneralRe: Compiling on VS 2003 version 7.1 Framework 1.1 Pin
daniel0492-Aug-07 3:43
memberdaniel0492-Aug-07 3:43 
Generalnot working Pin
yair_coh26-May-07 5:23
memberyair_coh26-May-07 5:23 
GeneralRe: not working Pin
redpirate13-Jun-07 3:40
memberredpirate13-Jun-07 3:40 
GeneralRe: not working Pin
ToolmakerSteve217-Jul-07 21:29
memberToolmakerSteve217-Jul-07 21:29 
GeneralRe: not working Pin
ppaijwar28-Nov-07 18:34
memberppaijwar28-Nov-07 18:34 
GeneralRe: not working Pin
roboticEDAR12-May-09 14:39
memberroboticEDAR12-May-09 14:39 
GeneralRe: not working Pin
roboticEDAR7-May-09 12:23
memberroboticEDAR7-May-09 12:23 
Questiona cry for help from FDaSilva Pin
Sean Ewington9-May-07 14:59
staffSean Ewington9-May-07 14:59 
AnswerRe: a cry for help from FDaSilva Pin
daniel04917-May-07 11:33
memberdaniel04917-May-07 11:33 
GeneralVS Express Solution and Projects file can be downloaded Pin
daniel04917-Nov-06 7:04
memberdaniel04917-Nov-06 7:04 
GeneralRe: VS Express Solution and Projects file can be downloaded Pin
patkoxx11-Jun-09 7:40
memberpatkoxx11-Jun-09 7:40 
GeneralAccesing to a custom interface Pin
burakburak5-Sep-06 12:30
memberburakburak5-Sep-06 12:30 
GeneralRe: Accesing to a custom interface Pin
daniel0495-Sep-06 14:39
memberdaniel0495-Sep-06 14:39 
Generalcannot find streams.h Pin
Nimble993-Sep-06 17:39
memberNimble993-Sep-06 17:39 
GeneralRe: cannot find streams.h Pin
daniel0495-Sep-06 3:07
memberdaniel0495-Sep-06 3:07 
GeneralCutler's DirectShow Filters Wizard Pin
StefanQED31-Jul-06 0:03
memberStefanQED31-Jul-06 0:03 
GeneralRe: Cutler's DirectShow Filters Wizard Pin
daniel04931-Jul-06 11:49
memberdaniel04931-Jul-06 11:49 
GeneralDevStudio 2005 and .Net 2.0 Pin
neticous24-Jul-06 6:46
memberneticous24-Jul-06 6:46 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
daniel04924-Jul-06 12:09
memberdaniel04924-Jul-06 12:09 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
neticous24-Jul-06 16:39
memberneticous24-Jul-06 16:39 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
denils14-Aug-06 5:56
memberdenils14-Aug-06 5:56 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
daniel0494-Aug-06 8:39
memberdaniel0494-Aug-06 8:39 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
Doc_Philip14-Nov-06 1:35
memberDoc_Philip14-Nov-06 1:35 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
roboticEDAR7-May-09 13:04
memberroboticEDAR7-May-09 13:04 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
daniel0497-May-09 13:35
memberdaniel0497-May-09 13:35 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
roboticEDAR7-May-09 14:14
memberroboticEDAR7-May-09 14:14 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
tortona29-Oct-08 7:41
membertortona29-Oct-08 7:41 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
roboticEDAR5-May-09 13:28
memberroboticEDAR5-May-09 13:28 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
roboticEDAR12-May-09 14:36
memberroboticEDAR12-May-09 14:36 
GeneralSecond Output Pin
Eylem Ozbay24-Jul-06 2:30
memberEylem Ozbay24-Jul-06 2:30 
GeneralRe: Second Output Pin
daniel04924-Jul-06 12:14
memberdaniel04924-Jul-06 12:14 
GeneralRe: Second Output Pin
roboticEDAR7-May-09 14:43
memberroboticEDAR7-May-09 14:43 
Generaltype conflict with DirectShowLib v1.4 Pin
Måns Tånneryd3-Jul-06 21:26
memberMåns Tånneryd3-Jul-06 21:26 
GeneralRe: type conflict with DirectShowLib v1.4 Pin
daniel0494-Jul-06 11:40
memberdaniel0494-Jul-06 11:40 
GeneralRe: type conflict with DirectShowLib v1.4 Pin
Måns Tånneryd4-Jul-06 11:59
memberMåns Tånneryd4-Jul-06 11:59 
GeneralRe: type conflict with DirectShowLib v1.4 Pin
daniel0495-Jul-06 4:51
memberdaniel0495-Jul-06 4:51 
GeneralPrototyping DirectShow filters in C# Pin
Eylem Ozbay16-Jun-06 4:08
memberEylem Ozbay16-Jun-06 4:08 
GeneralRe: Prototyping DirectShow filters in C# Pin
daniel04917-Jun-06 9:08
memberdaniel04917-Jun-06 9:08 
GeneralChanging the threshold Pin
metalhammer_116-May-06 9:33
membermetalhammer_116-May-06 9:33 
GeneralRe: Changing the threshold Pin
daniel04916-May-06 15:54
memberdaniel04916-May-06 15:54 
GeneralRe: Changing the threshold Pin
metalhammer_131-May-06 22:47
membermetalhammer_131-May-06 22:47 
GeneralRe: Changing the threshold Pin
daniel0491-Jun-06 3:44
memberdaniel0491-Jun-06 3:44 
GeneralProblem with media types Pin
metalhammer_111-May-06 11:37
membermetalhammer_111-May-06 11:37 
GeneralRe: Problem with media types Pin
daniel04914-May-06 15:07
memberdaniel04914-May-06 15:07 
GeneralCLSID of the filter Pin
metalhammer_18-May-06 13:23
membermetalhammer_18-May-06 13:23 
GeneralRe: CLSID of the filter Pin
daniel0498-May-06 15:52
memberdaniel0498-May-06 15:52 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.150520.1 | Last Updated 14 Dec 2005
Article Copyright 2005 by daniel049
Everything else Copyright © CodeProject, 1999-2015
Layout: fixed | fluid