Click here to Skip to main content
11,566,785 members (51,788 online)
Click here to Skip to main content

Prototyping DirectShow filters in C#

, 14 Dec 2005 205.6K 2.9K 64
Rate this:
Please Sign up or sign in to vote.
This article shows how to implement a DirectShow filter in C#.

Sample Image

Introduction

DirectShow filters are the basic building blocks of multimedia applications on the Windows platform. Normally, they are written in C++. In this article, I'll show how to implement a couple of DirectShow transformation filters in C#. Since Microsoft doesn't recommend using managed code for filters, I'll call them prototypes but they perform quite well in most circumstances. We'll develop Sobel transformation as a DirectShow filter. Sobel transformations are often used for video segmentation which is the process of identifying objects in a scene.

Background

The strategy we use to implement a DirectShow filter in C# relies on having a C++ class that contains a pointer to a managed object which implements our filter. The C++ class just forwards all the processing to the managed object. The C++ class is generated using Cutler's DirectShow Filters Wizard. We just clean up the wizard-generated code to include a pointer to our C# object and implement all the filter processing as methods of the managed object.

I've written this Sobel filter to have an idea of the kind of performance we'd get with C#. If you want to "take your CPU for a spin", this is a good test. As written (i.e., with no optimization effort), there are around 20 integer multiplications per pixel. Usually, you'd transform an image to black and white before applying Sobel transformation to the image. So I've also written a transformation filter that converts a RGB24 bitmap to black and white.

Using the code

If you just want to have a filter that implements Sobel transformations, you can download and install the managed DLL and Ax files (from the download link). Then after registering the Ax file and installing the managed DLL in the Global Assembly Cache (GAC), you can start the GraphEdit utility and add the "CsSobelV2" filter (found in the DirectShow filters category of GraphEdit). Then render a media file to see a black and white "sketch" of the frames in the video. (Note that the open-source DirectShowLib must also be in the GAC.) More information can be found in the "readme.txt" file that is included in the download.

If you want to implement your own filter using these filters as a starting point, you can modify the Transform and ProcessFrame methods of the MySobelV2 class. This is sufficient if you are satisfied with a single media type for the video frame, here Rgb24, and you can live with the default memory allocator of the C++ DirectShow base classes (this covers a lot of situations).

Sobel transformation

The Sobel transformation uses the following two matrices:

  // define the horizontal Sobel mat
  int [,] hx = new  int [,] { {-1, 0, 1}, 
                    {-2, 0, 2}, {-1, 0, 1} };

  // define the vertical Sobel mat
  int[,] hy = new int [,]  { { 1,  2,  1}, 
                  { 0,  0,  0}, {-1, -2, -1} };

And consists of the following code, found in ProcessFrame, which is repeated for every pixel in every frame of a video:

        gradX = 0;
        gradY = 0;
        for (int row = -1; row <= 1; row++)
        {
          for (int col = -1; col <= 1; col++)
          {
            position = GetPosition( x + col, y + row ); 
            intensity = pwSource[position];
            gradX += (intensity * hx[col + 1,row + 1]);
            gradY += (intensity * hy[col + 1,row + 1]); 
          }
        }
         
        // compute the square of the gradient        
        gradMag = gradX * gradX + gradY * gradY;

        // threshold default at 128
        gradMag = (gradMag < threshold*threshold) ? 255 : 0;
               
        // assign the pixel to the destination
        position = GetPosition( x, y );
        pwTarget[position] = pwTarget[position+1] = 
              pwTarget[position+2] = (byte)gradMag;

The intuition behind the transformation is relatively simple. Since the contour of objects consists of a region in a picture where the color changes rapidly, the Sobel transformation amplifies these differences and simply sets to white those pixels whose changes are too small (and black, the other ones). The horizontal Sobel transformation looks at the left and right neighboring pixels and multiplies these values by 2 or -2, it also looks at the upper right and left, and lower right and left pixels. After summing all these values, you compute the square of the Euclidian distance and use an arbitrary threshold to divide the values.

Points of Interest

I've also included the code for a black and white transformation filter for uncompressed Rgb24 video frames. As mentioned, you'd normally perform this transformation before feeding the result to the Sobel filter. So you can test the performances of a filter graph with two filters written in managed code.

Property pages for filters are handy when used in a utility like GraphEdit. But being able to access custom filter properties from C# (easily) is even more useful. So I've modified Cutler's Wizard-generated C++ code to include a property that can be changed in C#. Similar changes could be done to implement different properties.

Limitations and known issues

The media type accepted by the filters is uncompressed Rgb24 video. Since this is one of the most "natural" media types, it's not a big limitation but the filters were not designed to be very flexible. They are just prototypes to experiment with.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author

daniel049
United States United States
You can read my blog entries at:
http://wwww.informikon.com/blog/

You may also be interested in...

Comments and Discussions

 
NewsDirectshow filters completely in .NET Pin
Maxim Kartavenkov15-Aug-12 23:32
memberMaxim Kartavenkov15-Aug-12 23:32 
SuggestionGreat article, but why not do it all in C++? Pin
MikeDC25-Jun-11 13:01
memberMikeDC25-Jun-11 13:01 
QuestionWhere did you get .ax files? Pin
h.ahmadi8619-Dec-09 23:35
memberh.ahmadi8619-Dec-09 23:35 
GeneralGreat idea Pin
Steven Nagy6-Oct-09 16:47
memberSteven Nagy6-Oct-09 16:47 
GeneralMSIL error at compiling CsSobelV2.ax Pin
patkoxx4-Jun-09 3:44
memberpatkoxx4-Jun-09 3:44 
GeneralRe: MSIL error at compiling CsSobelV2.ax Pin
tsylverberg30-Jul-09 20:50
membertsylverberg30-Jul-09 20:50 
GeneralDllRegisterServer in CsSobel.ax Failed. Pin
roboticEDAR11-May-09 12:31
memberroboticEDAR11-May-09 12:31 
QuestionVisual studios 2008 Pin
Member 47309906-May-09 6:45
memberMember 47309906-May-09 6:45 
AnswerRe: Visual studios 2008 Pin
roboticEDAR7-May-09 11:19
memberroboticEDAR7-May-09 11:19 
AnswerRe: Visual studios 2008 Pin
roboticEDAR27-May-09 12:31
memberroboticEDAR27-May-09 12:31 
GeneralThere Are Several NON-Commercial RTSP Procol Source Filters! Pin
GUI Developer26-Oct-08 4:37
memberGUI Developer26-Oct-08 4:37 
QuestionAbstract Class Error Pin
nickmaggie31-Aug-07 3:56
membernickmaggie31-Aug-07 3:56 
AnswerRe: Abstract Class Error Pin
daniel0491-Sep-07 9:44
memberdaniel0491-Sep-07 9:44 
GeneralCompiling on VS 2003 version 7.1 Framework 1.1 Pin
Russ531-Aug-07 8:01
memberRuss531-Aug-07 8:01 
GeneralRe: Compiling on VS 2003 version 7.1 Framework 1.1 Pin
daniel0492-Aug-07 2:43
memberdaniel0492-Aug-07 2:43 
Generalnot working Pin
yair_coh26-May-07 4:23
memberyair_coh26-May-07 4:23 
GeneralRe: not working Pin
redpirate13-Jun-07 2:40
memberredpirate13-Jun-07 2:40 
GeneralRe: not working Pin
ToolmakerSteve217-Jul-07 20:29
memberToolmakerSteve217-Jul-07 20:29 
GeneralRe: not working Pin
ppaijwar28-Nov-07 17:34
memberppaijwar28-Nov-07 17:34 
GeneralRe: not working Pin
roboticEDAR12-May-09 13:39
memberroboticEDAR12-May-09 13:39 
GeneralRe: not working Pin
roboticEDAR7-May-09 11:23
memberroboticEDAR7-May-09 11:23 
Questiona cry for help from FDaSilva Pin
Sean Ewington9-May-07 13:59
staffSean Ewington9-May-07 13:59 
AnswerRe: a cry for help from FDaSilva Pin
daniel04917-May-07 10:33
memberdaniel04917-May-07 10:33 
GeneralVS Express Solution and Projects file can be downloaded Pin
daniel04917-Nov-06 6:04
memberdaniel04917-Nov-06 6:04 
GeneralRe: VS Express Solution and Projects file can be downloaded Pin
patkoxx11-Jun-09 6:40
memberpatkoxx11-Jun-09 6:40 
GeneralAccesing to a custom interface Pin
burakburak5-Sep-06 11:30
memberburakburak5-Sep-06 11:30 
GeneralRe: Accesing to a custom interface Pin
daniel0495-Sep-06 13:39
memberdaniel0495-Sep-06 13:39 
Generalcannot find streams.h Pin
Nimble993-Sep-06 16:39
memberNimble993-Sep-06 16:39 
GeneralRe: cannot find streams.h Pin
daniel0495-Sep-06 2:07
memberdaniel0495-Sep-06 2:07 
GeneralCutler's DirectShow Filters Wizard Pin
StefanQED30-Jul-06 23:03
memberStefanQED30-Jul-06 23:03 
GeneralRe: Cutler's DirectShow Filters Wizard Pin
daniel04931-Jul-06 10:49
memberdaniel04931-Jul-06 10:49 
GeneralDevStudio 2005 and .Net 2.0 Pin
neticous24-Jul-06 5:46
memberneticous24-Jul-06 5:46 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
daniel04924-Jul-06 11:09
memberdaniel04924-Jul-06 11:09 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
neticous24-Jul-06 15:39
memberneticous24-Jul-06 15:39 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
denils14-Aug-06 4:56
memberdenils14-Aug-06 4:56 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
daniel0494-Aug-06 7:39
memberdaniel0494-Aug-06 7:39 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
Doc_Philip14-Nov-06 0:35
memberDoc_Philip14-Nov-06 0:35 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
roboticEDAR7-May-09 12:04
memberroboticEDAR7-May-09 12:04 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
daniel0497-May-09 12:35
memberdaniel0497-May-09 12:35 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
roboticEDAR7-May-09 13:14
memberroboticEDAR7-May-09 13:14 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
tortona29-Oct-08 6:41
membertortona29-Oct-08 6:41 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
roboticEDAR5-May-09 12:28
memberroboticEDAR5-May-09 12:28 
GeneralRe: DevStudio 2005 and .Net 2.0 Pin
roboticEDAR12-May-09 13:36
memberroboticEDAR12-May-09 13:36 
GeneralSecond Output Pin
Eylem Ozbay24-Jul-06 1:30
memberEylem Ozbay24-Jul-06 1:30 
GeneralRe: Second Output Pin
daniel04924-Jul-06 11:14
memberdaniel04924-Jul-06 11:14 
GeneralRe: Second Output Pin
roboticEDAR7-May-09 13:43
memberroboticEDAR7-May-09 13:43 
Generaltype conflict with DirectShowLib v1.4 Pin
Måns Tånneryd3-Jul-06 20:26
memberMåns Tånneryd3-Jul-06 20:26 
GeneralRe: type conflict with DirectShowLib v1.4 Pin
daniel0494-Jul-06 10:40
memberdaniel0494-Jul-06 10:40 
GeneralRe: type conflict with DirectShowLib v1.4 Pin
Måns Tånneryd4-Jul-06 10:59
memberMåns Tånneryd4-Jul-06 10:59 
GeneralRe: type conflict with DirectShowLib v1.4 Pin
daniel0495-Jul-06 3:51
memberdaniel0495-Jul-06 3:51 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web03 | 2.8.150624.2 | Last Updated 14 Dec 2005
Article Copyright 2005 by daniel049
Everything else Copyright © CodeProject, 1999-2015
Layout: fixed | fluid