Introduction
This article explains how to write your custom DirectShow SampleGrabber
filter for Windows Mobile.
Background
In a recent project of mine, I needed to do some real-time video analysis in .NET CF. However the .NET API only allows for taking still images or recording videos, but it offers no way to access the video frame buffer so that I could parse the frames from the camera on the fly. So I decided to use the DirectShow
API, which offers much better control of the video stream flow, but it still lacks the ISampleGrabber
interface which is available under the complete DirectShow
library for Windows. The remaining option was to write a custom DirectShow
filter, which implements the ISampleGrabber
interface and allows the developer to get access to the video buffer data. In this article, I will try to explain what I learned in the process.
You could use the sample code as a starting point to write your own filter or take the ready SampleGrabber
filter and use it in your project. There are multiple references online which I used in the project, but as far as I am aware there is no complete "How to" on how to solve this problem.
Setting Up Your Visual Studio Project
Before your set up your Visual Studio project, you will need to install Windows Mobile SDK, Windows CE 5.0 and Windows CE 5.0 Platform builder which contains the BaseClasses
library. In general, you don't necessarily use the BaseClasses
, but it definitely makes your job a lot easier so I took advantage.
First create a Smart Devices DLL project with ATL support. You also need to add the header files from the Windows Mobile SDK and Platform Builder to your include path.
Writing the Filter
In this example, we will use a TransInPlaceFilter
, which is a simplified version of the TransformFilter
. We don't need to alter the data in any way, just to pass it to the client application. So we create a CSampleGrabber
class, which also implements our custom interface with any additional functions. I added just the RegisterCallback
function, which passes a function pointer from the client app. This function will be called whenever a MediaSample
(a video frame) is passed through the filter, so the client can copy the data and do some processing.
class CSampleGrabber :
public CTransInPlaceFilter, public ISampleGrabber
{
private:
MANAGEDCALLBACKPROC callback;
long m_Width;
long m_Height;
long m_SampleSize;
long m_Stride;
public:
CSampleGrabber( IUnknown * pOuter, HRESULT * phr, BOOL ModifiesData );
~CSampleGrabber();
static CUnknown *WINAPI CreateInstance(LPUNKNOWN punk, HRESULT *phr);
DECLARE_IUNKNOWN;
STDMETHODIMP NonDelegatingQueryInterface(REFIID riid, void ** ppv);
HRESULT CheckInputType(const CMediaType *pmt);
HRESULT SetMediaType(PIN_DIRECTION direction, const CMediaType *pmt);
HRESULT Transform(IMediaSample *pMediaSample);
HRESULT DecideBufferSize(IMemAllocator *pAlloc, ALLOCATOR_PROPERTIES *pProperties);
HRESULT GetMediaType(int iPosition, CMediaType *pMediaType);
HRESULT CheckTransform(const CMediaType *mtIn, const CMediaType *mtOut) {
return NOERROR;
}
STDMETHODIMP RegisterCallback(MANAGEDCALLBACKPROC mdelegate);
};
You could implement all interfaces like any other filter. The interesting parts are the Transform
and RegisterCallback
functions:
STDMETHODIMP
CSampleGrabber::RegisterCallback( MANAGEDCALLBACKPROC mdelegate )
{
callback = mdelegate;
return S_OK;
}
HRESULT
CSampleGrabber::Transform(IMediaSample *pMediaSample)
{
long Size = 0;
BYTE *pData;
if ( !pMediaSample )
return E_FAIL;
if( FAILED(pMediaSample->GetPointer(&pData)) )
return E_FAIL;
Size = pMediaSample->GetSize();
if ( callback )
callback(pData,Size);
return S_OK;
}
Using the Code
You can use the SampleGrabber
filter directly in your DirectShow
applications exactly like any other filter. It is important to register the DLL before trying to instantiate it.
CoCreateInstance( CLSID_SampleGrabber, NULL, CLSCTX_INPROC,
IID_IBaseFilter, (void**)&pSampleGrabber );
m_pFilterGraph->AddFilter( pSampleGrabber, FILTERNAME );
pSampleGrabber->QueryInterface( IID_ISampleGrabber, (void**)&m_pISampleGrabber );
if ( m_pISampleGrabber )
m_pISampleGrabber->RegisterCallback( &CGraphBuilder::OnSampleProcessed );
When the client receives the frame sample, it should copy it to local buffer as soon as possible and return. This allows the filter to carry on and not wait for the client to process the data, which will considerably slow down the whole graph.
You could also make a C++ DLL which creates the filter graph and manages it and calls it through P/Invoke from your .NET CF applications. Creating filter graphs in .NET CF directly should be a bit trickier, as the .NET CF lacks C++. NET support, but maybe possible.
Online References
History
- 23.08.2008 Article created
- 26.08.2008 v1.1 Created.
Fixes:
- Release now builds OK, definition of
NonDelegatingRelease
added
- Included C++ and C# client applications which show how to use the filter
- The sample code now compiles under WM SDK 5
- 28.11.2008
- Added more sample code:
CameraCaptureDLL
project includes the CameraCapture
sample and the builtin SampleGrabber
filter. The TestCameraCapture
project is a sample C# application that uses the filter to get the real-time frame data.
- Note: The sample code demonstrates the basic concepts, but there are a couple of issues with it. For example, it might get stuck when trying to stop the filter graph. This is due to the fact that the filter is processing data while you try to do that (sorry but I didn't have time to fix that). Although buggy, you should be able to get the camera data.