Click here to Skip to main content
Click here to Skip to main content

Creating Custom DirectShow SampleGrabber Filter for Windows Mobile

By , 29 Nov 2008
Rate this:
Please Sign up or sign in to vote.

Introduction

This article explains how to write your custom DirectShow SampleGrabber filter for Windows Mobile.

Background

In a recent project of mine, I needed to do some real-time video analysis in .NET CF. However the .NET API only allows for taking still images or recording videos, but it offers no way to access the video frame buffer so that I could parse the frames from the camera on the fly. So I decided to use the DirectShow API, which offers much better control of the video stream flow, but it still lacks the ISampleGrabber interface which is available under the complete DirectShow library for Windows. The remaining option was to write a custom DirectShow filter, which implements the ISampleGrabber interface and allows the developer to get access to the video buffer data. In this article, I will try to explain what I learned in the process.

You could use the sample code as a starting point to write your own filter or take the ready SampleGrabber filter and use it in your project. There are multiple references online which I used in the project, but as far as I am aware there is no complete "How to" on how to solve this problem.

Setting Up Your Visual Studio Project

Before your set up your Visual Studio project, you will need to install Windows Mobile SDK, Windows CE 5.0 and Windows CE 5.0 Platform builder which contains the BaseClasses library. In general, you don't necessarily use the BaseClasses, but it definitely makes your job a lot easier so I took advantage.

First create a Smart Devices DLL project with ATL support. You also need to add the header files from the Windows Mobile SDK and Platform Builder to your include path.

Writing the Filter

In this example, we will use a TransInPlaceFilter, which is a simplified version of the TransformFilter. We don't need to alter the data in any way, just to pass it to the client application. So we create a CSampleGrabber class, which also implements our custom interface with any additional functions. I added just the RegisterCallback function, which passes a function pointer from the client app. This function will be called whenever a MediaSample (a video frame) is passed through the filter, so the client can copy the data and do some processing.

// define the filter class
class CSampleGrabber : 
public CTransInPlaceFilter, public ISampleGrabber
{
  private:
  MANAGEDCALLBACKPROC callback;
  long m_Width;
  long m_Height;
  long m_SampleSize;
  long m_Stride;

  public:
  // instantiation
  CSampleGrabber( IUnknown * pOuter, HRESULT * phr, BOOL ModifiesData );
  ~CSampleGrabber();
  static CUnknown *WINAPI CreateInstance(LPUNKNOWN punk, HRESULT *phr);

  // IUnknown
  DECLARE_IUNKNOWN;
  STDMETHODIMP NonDelegatingQueryInterface(REFIID riid, void ** ppv);

  // CTransInPlaceFilter
  HRESULT CheckInputType(const CMediaType *pmt);
  HRESULT SetMediaType(PIN_DIRECTION direction, const CMediaType *pmt);
  HRESULT Transform(IMediaSample *pMediaSample);
  HRESULT DecideBufferSize(IMemAllocator *pAlloc, ALLOCATOR_PROPERTIES *pProperties);
  HRESULT GetMediaType(int iPosition, CMediaType *pMediaType);
  HRESULT CheckTransform(const CMediaType *mtIn, const CMediaType *mtOut) {
    return NOERROR; 
    }

  // ISampleGrabber
  STDMETHODIMP RegisterCallback(MANAGEDCALLBACKPROC mdelegate);
};

You could implement all interfaces like any other filter. The interesting parts are the Transform and RegisterCallback functions:

// Set the client callback
STDMETHODIMP
CSampleGrabber::RegisterCallback( MANAGEDCALLBACKPROC mdelegate ) 
{
    // Set pointer to managed delegate
    callback = mdelegate;
    return S_OK;
}

// Get the pointer to the raw data and pass it to the applications
HRESULT 
CSampleGrabber::Transform(IMediaSample *pMediaSample)
{
    long Size = 0;
    BYTE *pData;

    if ( !pMediaSample )
        return E_FAIL;

    // Get pointer to the video buffer data
    if( FAILED(pMediaSample->GetPointer(&pData)) ) 
        return E_FAIL;
    Size = pMediaSample->GetSize();

    // invoke managed delegate
    if ( callback )
        callback(pData,Size);

    return S_OK;
}

Using the Code

You can use the SampleGrabber filter directly in your DirectShow applications exactly like any other filter. It is important to register the DLL before trying to instantiate it.

// Create and initialize the SampleGrabber filter
CoCreateInstance( CLSID_SampleGrabber, NULL, CLSCTX_INPROC, 
		IID_IBaseFilter, (void**)&pSampleGrabber );
m_pFilterGraph->AddFilter( pSampleGrabber, FILTERNAME );

// Get a pointer to the ISampleGrabber interface
pSampleGrabber->QueryInterface( IID_ISampleGrabber, (void**)&m_pISampleGrabber );

// Register the client callback
if ( m_pISampleGrabber )
  m_pISampleGrabber->RegisterCallback( &CGraphBuilder::OnSampleProcessed );

When the client receives the frame sample, it should copy it to local buffer as soon as possible and return. This allows the filter to carry on and not wait for the client to process the data, which will considerably slow down the whole graph.

You could also make a C++ DLL which creates the filter graph and manages it and calls it through P/Invoke from your .NET CF applications. Creating filter graphs in .NET CF directly should be a bit trickier, as the .NET CF lacks C++. NET support, but maybe possible.

Online References

History

  • 23.08.2008 Article created
  • 26.08.2008 v1.1 Created.
    Fixes:
    • Release now builds OK, definition of NonDelegatingRelease added
    • Included C++ and C# client applications which show how to use the filter
    • The sample code now compiles under WM SDK 5
  • 28.11.2008
    • Added more sample code: CameraCaptureDLL project includes the CameraCapture sample and the builtin SampleGrabber filter. The TestCameraCapture project is a sample C# application that uses the filter to get the real-time frame data.
    • Note: The sample code demonstrates the basic concepts, but there are a couple of issues with it. For example, it might get stuck when trying to stop the filter graph. This is due to the fact that the filter is processing data while you try to do that (sorry but I didn't have time to fix that). Although buggy, you should be able to get the camera data.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Angel Kafazov
Technical Lead CST
Bulgaria Bulgaria
I am software engineering working for CST. My interests cover areas of mobile development, unified communications, VOIP, telecommunications and many more.
 
For further information please contant me at akafazov@cst-bg.net
Follow on   Twitter

Comments and Discussions

 
QuestionCan this project build for WinCE? PinmemberMember 1062955626-Feb-14 23:02 
Questionblurred image LG GM750 smartphone PinmemberLawrence Tsang24-Oct-11 18:35 
Hi Angel,
 
Thanks for sharing your nice works.
 
I have tried your programs on my LG GM750 phone running Windows Mobile 6.5. The camera's output format is YV12 and so I have changed MEDIASUBTYPE_RGB24 to MEDIASUBTYPE_RGB565 in CSampleGrabber::CheckInputType and SampleGrabber::SetMediaType, and Format24bppRgb to Format16bppRgb565 in CreateBitmap and OnMediaTypeSet in the C# program.
 
When deploying the projects into my smartphone, I could see a blurred image on the screen. When I place a finger horizontally in front of the camera, I could see a very blurred and much elongated finger-like shape. When I move my finger up and down, the blurred and elongated shape moves up and down too.
 
I think that the raw data from the camera may be mis-interpreted when they are copied to the screen but I have no idea where should I try to change the parameters. One point I suspected is that the "stride" size may be different when using RGB565 mode, i.e.
 
in SampleGrabber.cpp,
DWORD dwStride = (vih->bmiHeader.biWidth * (vih->bmiHeader.biBitCount / 8) + 3) & ~3;
 
This dwStride may have to be assigned a different value for the programs to work correctly. However, I have no idea how it should be assigned and perhaps there is other parameter that should be changed too.
 
Could I have some advices on how these parameters should be changed. Thanks for your attention.
 
Regards
Lawrence
AnswerRe: blurred image LG GM750 smartphone PinmemberLawrence Tsang26-Oct-11 21:00 
Generalcrash cocreateInstance CSampleGrabeer Pinmemberjoseanaya2-May-11 15:03 
GeneralRegistering the DLL Pinmemberfakepoo3-Mar-11 12:58 
GeneralRe: Registering the DLL PinmemberAngel Kafazov7-Mar-11 0:45 
GeneralISamplegrabber not working on winmo device [modified] PinmemberMember 372985127-Nov-10 20:56 
General[Exception] System.IO.IOException [modified] Pinmemberbubb13t3a10-Oct-10 15:38 
GeneralRe: [Exception] System.IO.IOException [modified] PinmemberY07004016-Oct-10 20:48 
GeneralRe: [Exception] System.IO.IOException Pinmemberbubb13t3a19-Oct-10 22:45 
GeneralRe: [Exception] System.IO.IOException PinmemberY07004019-Oct-10 23:03 
GeneralRe: [Exception] System.IO.IOException Pinmemberbubb13t3a19-Oct-10 23:55 
GeneralRe: [Exception] System.IO.IOException PinmemberY07004020-Oct-10 16:46 
GeneralRe: [Exception] System.IO.IOException Pinmemberbubb13t3a24-Oct-10 17:27 
GeneralRe: [Exception] System.IO.IOException PinmemberY07004025-Oct-10 5:01 
QuestionFrameGrabber only works in Emulator, not on actuall Device Pinmembertrimber9-Aug-10 0:43 
AnswerRe: FrameGrabber only works in Emulator, not on actuall Device PinmemberY07004019-Aug-10 17:58 
AnswerRe: FrameGrabber only works in Emulator, not on actuall Device Pinmemberbubb13t3a10-Oct-10 15:44 
Generalchanging resolution Pinmembers0pHoc4-Aug-10 4:05 
AnswerRe: changing resolution [modified] PinmemberY07004019-Aug-10 16:21 
Generalunresolved external symbol "public: virtual unsigned long __cdecl CBaseFilter::NonDelegatingRelease(void)" Pinmemberneosurya3-Jun-10 12:40 
GeneralRe: unresolved external symbol "public: virtual unsigned long __cdecl CBaseFilter::NonDelegatingRelease(void)" PinmemberLinda Rawson28-Sep-10 6:09 
AnswerRe: unresolved external symbol "public: virtual unsigned long __cdecl CBaseFilter::NonDelegatingRelease(void)" PinmemberY0700402-Oct-10 22:07 
Questionhow to get a samplegrabber instance simply by using method cocreateinstance Pinmemberstonewind2-May-10 3:51 
AnswerRe: how to get a samplegrabber instance simply by using method cocreateinstance PinmemberY0700402-May-10 15:07 
GeneralRe: how to get a samplegrabber instance simply by using method cocreateinstance Pinmemberstonewind2-May-10 20:41 
GeneralRe: how to get a samplegrabber instance simply by using method cocreateinstance PinmemberY0700402-May-10 21:07 
GeneralRe: how to get a samplegrabber instance simply by using method cocreateinstance [modified] Pinmemberstonewind2-May-10 22:28 
GeneralRe: how to get a samplegrabber instance simply by using method cocreateinstance PinmemberY0700403-May-10 15:18 
GeneralRe: how to get a samplegrabber instance simply by using method cocreateinstance Pinmemberstonewind3-May-10 19:43 
GeneralRe: how to get a samplegrabber instance simply by using method cocreateinstance PinmemberY0700403-May-10 22:14 
GeneralRe: how to get a samplegrabber instance simply by using method cocreateinstance Pinmemberstonewind4-May-10 20:38 
GeneralRe: how to get a samplegrabber instance simply by using method cocreateinstance PinmemberY0700404-May-10 21:14 
GeneralRe: how to get a samplegrabber instance simply by using method cocreateinstance Pinmemberstonewind4-May-10 22:15 
GeneralRe: how to get a samplegrabber instance simply by using method cocreateinstance PinmemberY0700404-May-10 23:13 
GeneralRe: how to get a samplegrabber instance simply by using method cocreateinstance Pinmemberstonewind5-May-10 22:54 
QuestionErrors Occured Pinmemberstonewind27-Apr-10 22:34 
AnswerRe: Errors Occured [modified] PinmemberY07004028-Apr-10 15:35 
GeneralRe: Errors Occured Pinmemberstonewind2-May-10 4:17 
GeneralSetting owner PinmemberBig D7-Mar-10 21:22 
AnswerRe: Setting owner PinmemberAngel Kafazov8-Mar-10 7:43 
GeneralRe: Setting owner PinmemberBig D18-Mar-10 4:31 
AnswerRe: Setting owner PinmemberAngel Kafazov18-Mar-10 4:48 
GeneralRe: Setting owner Pinmemberbaltazar0074-Apr-10 3:28 
AnswerRe: Setting owner [modified] PinmemberY07004018-Apr-10 17:53 
GeneralHTC HD2 PinmemberMarten1238-Feb-10 12:19 
AnswerRe: HTC HD2 [modified] PinmemberY07004014-Apr-10 17:35 
GeneralError under WM5 Pinmembersunbite11-Jan-10 23:08 
AnswerRe: Error under WM5 PinmemberAngel Kafazov12-Jan-10 0:41 
GeneralRe: Error under WM5 Pinmembersunbite12-Jan-10 1:19 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web01 | 2.8.140421.2 | Last Updated 29 Nov 2008
Article Copyright 2008 by Angel Kafazov
Everything else Copyright © CodeProject, 1999-2014
Terms of Use
Layout: fixed | fluid