Click here to Skip to main content
Click here to Skip to main content
Go to top

DirectShow Filters Development Part 2: Live Source Filter

, 15 Mar 2011
Rate this:
Please Sign up or sign in to vote.
A generic source filter which exposes an interface for pushing downstream RGB samples of predefined size and frame rate, and can be used for any custom frame input scenario.

Introduction

A vast majority of software and hardware vendors working in the digital video field supply a source filter capable of getting frames from some hardware device like video acquisition card or IP camera. So our mission as software developers is as simple as build a filter graph, add this source filter, and render it. However, not all applications and hardware devices are built with DirectShow in mind, and when you need to use them, you will probably build a source filter of your own which will wrap the API of that application or hardware provider.

To make this task easy, I decided to make a generic source filter which exposes an interface for pushing downstream RGB samples of predefined size and frame rate, and can be used for any custom frame input scenario (I hope Smile | :) ).

Before going any further, take a look at part 1 of this series as the filter development prerequisites, filter registration, and debugging are same for all filter types.

Source filters

Source filter is the most important filter in the graph as it dictates the media size, frame rate, and media format. Each filter graph contains at least one source filter which can produce several media streams not necessarily of the same type. For example, a media splitter source filter can produce video, audio, and text (subtitles) streams each of which is pushed to downstream filters through its own pin. Usually, the information regarding the streams is contained inside a media file. When there is no media file, this information resides inside a source filter, or inside its output pins.

When building source filters, you basically have two choices:

  1. Build a source filter which inherits from the CSource base class, and add a nested pin class which inherits from CSourceStream and is responsible for the actual frame creation and delivery. CSourceStream inherits from the CAMThread class which encapsulates the background thread responsible for delivering media samples to the downstream filter through its input pin. These kinds of filters are called push source filters, and they are well documented on MSDN and have numerous examples like the PushSource and Bouncing Ball samples. In addition, there are some articles here on CodeProject describing how to build these filters. Therefore, I will not cover this topic.
  2. Instead, I will write about the less documented Live Source filters. To create a live source filter, use CBaseFilter as the filter base class, and its pin class should inherit from CBaseOutputPin.
class CLiveSourceStream;

class CLiveSource : public CBaseFilter, public IAMFilterMiscFlags
{
public:
   DECLARE_IUNKNOWN;

   CLiveSource(LPUNKNOWN pUnk, HRESULT* phr);
   virtual ~CLiveSource(void);

   static CUnknown* WINAPI CreateInstance(LPUNKNOWN pUnk, HRESULT *phr);
   STDMETHODIMP NonDelegatingQueryInterface(REFIID riid, void ** ppv);

   int GetPinCount();
   CBasePin* GetPin(int n);

   virtual ULONG STDMETHODCALLTYPE GetMiscFlags( void)
   {
          return AM_FILTER_MISC_FLAGS_IS_SOURCE;
   }

private:
   CLiveSourceStream* m_pOutputPin;
   CCritSec m_critSec;
};

CBaseFilter is an abstract class and you need to implement its two pure virtual functions:

int CLiveSource::GetPinCount()
{
   CAutoLock cAutoLock(&m_critSec);

   return 1;
}

CBasePin* CLiveSource::GetPin(int n)
{
   CAutoLock cAutoLock(&m_critSec);

   return m_pOutputPin;
}

GetPinCount returns 1 since there is only one output pin which pushes video samples provided from another context using the ILiveSource interface. GetPin returns a pointer to that only pin.

According to MSDN documentation, a live source filter delivers a media sample at an inconstant rate. The filter is considered as a live source filter if it implements the IAMFilterMiscFlags interface and the filter's output pin implements the IAMPushSource interface.

class CLiveSourceStream : public CBaseOutputPin, 
                 public ILiveSource, public IAMPushSource
{
public:

   DECLARE_IUNKNOWN;

   CLiveSourceStream(CBaseFilter *pFilter, CCritSec *pLock, HRESULT *phr);
   virtual ~CLiveSourceStream();

   // CBaseOutputPin overrides
   virtual HRESULT GetMediaType(int iPosition, CMediaType* pmt);
   virtual HRESULT CheckMediaType(const CMediaType *pmt);
   virtual HRESULT DecideBufferSize(IMemAllocator *pAlloc, 
                   ALLOCATOR_PROPERTIES *ppropInputRequest);

   // ILiveSource members
   virtual HRESULT AddFrame(HBITMAP hBmp);
   virtual HRESULT AddFrame(BYTE* pBuffer, DWORD size);
   virtual HRESULT SetFrameRate(int frameRate);
   virtual HRESULT SetBitmapInfo(BITMAPINFOHEADER& bInfo);

   // IAMPushSource members
   virtual STDMETHODIMP GetPushSourceFlags(ULONG *pFlags);
   virtual STDMETHODIMP SetPushSourceFlags(ULONG Flags);
   virtual STDMETHODIMP SetStreamOffset(REFERENCE_TIME rtOffset);
   virtual STDMETHODIMP GetStreamOffset(REFERENCE_TIME *prtOffset);
   virtual STDMETHODIMP GetMaxStreamOffset(REFERENCE_TIME *prtMaxOffset);
   virtual STDMETHODIMP SetMaxStreamOffset(REFERENCE_TIME rtMaxOffset);
   virtual STDMETHODIMP GetLatency(REFERENCE_TIME *prtLatency);

   virtual STDMETHODIMP Notify(IBaseFilter * pSender, Quality q);

private:
   HRESULT GetPixelData(HBITMAP hBmp, BYTE** ppData, int* pSize);
   HRESULT GetMediaSample(IMediaSample** ppSample);

private:
   BITMAPINFOHEADER m_bmpInfo;
   int m_frameRate;
   REFERENCE_TIME m_rtFrameRate; 
   REFERENCE_TIME m_lastFrame; 
};

Although IAMPushSource is implemented here, it is pretty much useless since there is only one output pin and no synchronization is needed. If, for example, you would want to have a second pin with a different sample rate, then you should implement this interface as described in here.

The ILiveSource filter has four methods:

struct ILiveSource : IUnknown
{
   // Adds bitmap to the video sequence
   virtual HRESULT AddFrame(HBITMAP hBmp) PURE;

   // Adds pixel data buffer to the video sequence
   virtual HRESULT AddFrame(BYTE* pBuffer, DWORD size) PURE;

   // Set the video frame info.
   // Default value is width = 704,
   // height = 576 (4CIF) and 32 bits per pixel
   virtual HRESULT SetBitmapInfo(BITMAPINFOHEADER& bInfo) PURE;

   // Set the expected frame rate of the video.
   // Value should be in range of [0,30]
   // Default value is 0
   virtual HRESULT SetFrameRate(int frameRate) PURE;
};

The first two methods are responsible for passing frames into a running filter graph. The first method uses the GetObject API (described below) to get the pixel data and other bitmap parameters. The second AddFrame override takes a pointer to the pixel data buffer and the size of the buffer to prevent overflows. The size is checked against the biSizeImage member of the BITMAPINFOHEADER structure to determine whether it is valid.

By default, BITMAPINFOHEADER is set to 4CIF frames (704 x 576) and 32 bits per pixel; however, you can set it using the SetBitmapInfo method.

SetFrameRate is used to set the start and end time for each media sample. Using these values, the renderer filter schedules each sample for display and monitors quality control. By default, the frame rate is set to 0, which means no scheduling will take place and each media sample will be rendered immediately after arriving to the input pin of the renderer.

Media samples

IMediaSample is the transport unit used between pins of DirectShow filters. Source filters are responsible for creating them, filling the data, and passing it downstream. A transform filter performs some operations on that data, and renderer filters render them either on screen, network, or another location. Each media sample contains a memory buffer which holds the actual sample data - in the case of a video, it will usually have pixel data either raw or encoded. Each sample also has reference times which govern the playback and quality control messages sent in the reverse direction - from renderer filter to the source filter or whoever can handle such types of messages.

When using the CBaseOutputPin class, it your responsibly to get the media sample from the memory allocator, fill it with data, set time stamps, set the sample size, and deliver the sample to the input pin of the connected downstream filter. Hence, the AddFrame method implementation looks as follows:

HRESULT CLiveSourceStream::AddFrame(HBITMAP hBmp)
{
   CAutoLock cAutoLock(m_pLock);

   IMediaSample* pSample = NULL;
   BYTE* pData = NULL;

   HRESULT hr = GetMediaSample(&pSample);
   if(FAILED(hr))
   {
          return hr;
   }

   hr = pSample->GetPointer(&pData);
   if(FAILED(hr))
   {
          pSample->Release();
          return hr;
   }

   hr = GetPixelData(hBmp, &pData, &iSize);
   if(FAILED(hr))
   {
          pSample->Release();
          return hr;
   }

   hr = pSample->SetActualDataLength(iSize);
   if(FAILED(hr))
   {
          pSample->Release();
          return hr;
   }

   hr = pSample->SetSyncPoint(TRUE);
   if(FAILED(hr))
   {
          pSample->Release();
          return hr;
   }
   
   hr = this->Deliver(pSample);
   pSample->Release();

   return hr;
}

GetMediaSample allocates a sample from the memory allocator in the base CBaseOutputPin class and passes the start and end times which can also be zero:

HRESULT CLiveSourceStream::GetMediaSample(IMediaSample** ppSample)
{
   REFERENCE_TIME rtStart = m_lastFrame;
   m_lastFrame += m_rtFrameRate;

   return this->GetDeliveryBuffer(ppSample, &rtStart, &m_lastFrame, 0);
}

GetPixelData validates the bitmap parameters against the previously set (or default) parameters, and if they are OK, returns the pointer to the pixel data and the data size:

HRESULT CLiveSourceStream::GetPixelData(HBITMAP hBmp, BYTE** ppData, int* pSize)
{
   ASSERT(hBmp);
   
   BITMAP bmp = {0};
   int res = ::GetObject(hBmp, sizeof(BITMAP), &bmp);
   if(res != sizeof(BITMAP))
   {
          return E_FAIL;
   }

   if(bmp.bmBitsPixel != m_bmpInfo.biBitCount ||
      bmp.bmHeight != m_bmpInfo.biHeight ||
      bmp.bmWidth != m_bmpInfo.biWidth)
   {
          return E_INVALIDARG;
   }

   *pSize = bmp.bmWidthBytes * bmp.bmHeight;
   memcpy(*ppData, bmp.bmBits, *pSize);

   return S_OK;
}

After successfully delivering the media sample, you need to release it as it is a simple COM object and uses reference counting for its memory management. Note that downstream filters that need the sample beyond the Deliver method call scope need to call AddRef on that sample.

COM plumbing

As you already know, filters are COM containers, which means you have to implement three basic COM constructs:

  1. Static factory method for creating filter classes:
    CUnknown* WINAPI CLiveSource::CreateInstance(LPUNKNOWN pUnk, HRESULT *phr)
    {
       CUnknown* pNewFilter = new CLiveSource(pUnk, phr);
    
       if (phr)
       {
          if (pNewFilter == NULL) 
            *phr = E_OUTOFMEMORY;
          else
            *phr = S_OK;
       }
    
       return pNewFilter;
    }
  2. When your filter implements some interfaces, you should override the QueryInterface method and increment the reference count by calling the GetInterface method:
    STDMETHODIMP CLiveSource::NonDelegatingQueryInterface(REFIID riid, void **ppv)
    {
       CheckPointer(ppv, E_POINTER);
    
       if(riid == IID_ILiveSource) 
       {
          return GetInterface((ILiveSource*) m_pOutputPin, ppv);
       } 
       else if(riid == IID_IAMPushSource)
       {
          return GetInterface((IAMPushSource*) m_pOutputPin, ppv);
       }
       else if(riid == IID_IAMFilterMiscFlags)
       {
          return GetInterface((IAMFilterMiscFlags*) this, ppv);
       }
       else 
       {
          return CBaseFilter::NonDelegatingQueryInterface(riid, ppv);
       }
    }
  3. When implementing an interface like ILiveSource in this sample code, you should implement IUnknown by using the DECLARE_IUNKNOWN macro.

Using the code

After successfully building and registering the filter, you can use it in your code:

::CoInitialize(NULL);

graph.CoCreateInstance(CLSID_FilterGraph);
graph->QueryInterface(IID_IMediaControl, (void**)&media);
graph->QueryInterface(IID_IFilterGraph2, (void**)&m_filterGraph2);

pSource.CoCreateInstance(CLSID_CLiveSource);
pSource->QueryInterface(IID_ILiveSource, (void**)&pLiveSource);

m_filterGraph2->AddFilter(pSource, LIVE_FILTER_NAME);

IPin* outpin = NULL;
pSource->FindPin(LIVE_OUTPIN_NAME, &outpin);
m_filterGraph2->Render(outpin);
outpin->Release();

RECT area;
m_videoWnd.GetClientRect(&area);

CComPtr<IVideoWindow> pWnd = NULL;
graph->QueryInterface(IID_IVideoWindow, (void**)&pWnd);
pWnd->put_Owner((OAHWND)m_videoWnd.m_hWnd);
pWnd->put_WindowStyle(WS_CHILD | WS_CLIPSIBLINGS);     
pWnd->SetWindowPosition(area.left, area.top, 
         area.right - area.left, area.bottom - area.top);
pWnd.Release();

media->Run();

And at some other place and may be in another thread, as the filter is thread safe, you can call AddFrame:

HBITMAP hBmp = GetFrame();
HRESULT hr = pLiveSource->AddFrame(hBmp);
if(FAILED(hr))
{
   // Handle error here !
}

::DeleteObject(hBmp);

References

  1. Programming DirectShow for Digital Video and TV
  2. Filter Base Classes
  3. Live Source Filters

History

  • 15.2.2011
  • Initial version.

  • 13.3.2011
  • Fixed calling convention in the IAddFrame interface.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Roman Ginzburg
Software Developer (Senior)
Israel Israel
No Biography provided

Comments and Discussions

 
QuestionTrying to write out a video with this filter as source Pinmemberrajas28-May-14 2:31 
Questionbuilding project as dll instead of .ax file PinmemberMember 104988851-Jan-14 23:35 
QuestionFrame rate drop after some time PinmemberMember 99615013-Oct-13 3:49 
AnswerRe: Frame rate drop after some time Pinmemberyunf20-Nov-13 15:06 
QuestionIMediaControl::Run from graphedit Pinmembersmartik1710-Aug-13 8:56 
QuestionLive source filter PinmemberGahmed10-Apr-13 0:41 
QuestionHow to turn it to h264 filter PinmemberRobert Clove3-Mar-13 19:23 
Sir,
 
Really thanks for the sample.
Can you just guide me instead of bitmap i want to give h264 data from buffer.
I have made some changes but that seem so not so perfect.
If you can guide me i will be a great help from your side.
Other wise please give a mail on cloverobert@gmail.com so that i can send you my sample.
 
Thanks
QuestionDirectShow Video Playback from memory Pinmembersdancer7520-Feb-13 4:55 
QuestionSir how to use this in GMF Bridge source code [modified] PinmemberTarun Batra28-Dec-12 2:44 
AnswerRe: Sir how to use this in GMF Bridge source code Pinmemberroman_gin28-Dec-12 7:46 
QuestionHelp with usage Pinmemberladuran21-Dec-12 11:16 
QuestionWould like to use this as a source filter for MPEG Transport Stream Pinmemberladuran19-Dec-12 12:27 
AnswerRe: Would like to use this as a source filter for MPEG Transport Stream Pinmemberroman_gin20-Dec-12 5:57 
GeneralRe: Would like to use this as a source filter for MPEG Transport Stream PinmemberThom_Rex26-Mar-13 20:07 
Questionhow register the filter so Pinmemberpavel636-Oct-12 10:35 
AnswerRe: how register the filter so Pinmemberroman_gin20-Dec-12 5:54 
QuestionOutput to avi file Pinmembermurakoshi24-Sep-12 22:42 
AnswerRe: Output to avi file Pinmemberroman_gin28-Sep-12 23:29 
AnswerRe: Output to avi file PinmemberRobert Clove3-Mar-13 19:24 
AnswerRe: Output to avi file PinmemberRobert Clove3-Mar-13 19:37 
AnswerRe: Output to avi file PinmemberGilad Bauman28-May-13 2:39 
QuestionDear author [modified] Pinmemberiloveyy775852116-Sep-12 2:19 
AnswerRe: Dear author Pinmemberroman_gin16-Sep-12 20:27 
GeneralRe: Dear author Pinmemberiloveyy775852117-Sep-12 3:43 
GeneralRe: Dear author Pinmemberroman_gin17-Sep-12 6:42 
GeneralRe: Dear author Pinmemberiloveyy775852119-Sep-12 4:01 
GeneralRe: Dear author Pinmemberroman_gin19-Sep-12 8:28 
GeneralRe: Dear author Pinmemberiloveyy775852120-Sep-12 3:32 
GeneralRe: Dear author Pinmemberroman_gin20-Sep-12 22:13 
GeneralRe: Dear author Pinmemberiloveyy775852121-Sep-12 16:02 
GeneralRe: Dear author Pinmemberroman_gin21-Sep-12 23:26 
GeneralRe: Dear author Pinmemberiloveyy775852125-Sep-12 12:22 
SuggestionCLiveSourceStream::GetPixelData PinmemberNik Amir20-Jun-12 5:39 
GeneralRe: CLiveSourceStream::GetPixelData Pinmemberroman_gin16-Sep-12 20:22 
QuestionHow can I use this with MP4 File Writer? PinmemberKnDol15-May-12 2:31 
AnswerRe: How can I use this with MP4 File Writer? Pinmemberroman_gin15-May-12 4:37 
GeneralRe: How can I use this with MP4 File Writer? PinmemberKnDol15-May-12 17:52 
GeneralRe: How can I use this with MP4 File Writer? Pinmemberroman_gin16-May-12 1:10 
GeneralRe: How can I use this with MP4 File Writer? PinmemberKnDol16-May-12 2:32 
GeneralI tried to use smart tee filter. PinmemberKnDol16-May-12 21:07 
GeneralRe: I tried to use smart tee filter. Pinmemberroman_gin19-May-12 19:22 
QuestionLive Source Filter - Frame rendered as skewed :( PinmemberAmin Ur Rehman22-Feb-12 7:33 
AnswerRe: Live Source Filter - Frame rendered as skewed :( PinmemberAmin Ur Rehman23-Feb-12 2:42 
GeneralRe: Live Source Filter - Frame rendered as skewed :( Pinmemberroman_gin23-Feb-12 6:00 
GeneralRe: Live Source Filter - Frame rendered as skewed :( PinmemberAmin Ur Rehman23-Feb-12 6:33 
GeneralRe: Live Source Filter - Frame rendered as skewed :( PinmemberAmin Ur Rehman23-Feb-12 7:58 
GeneralRe: Live Source Filter - Frame rendered as skewed :( PinmemberAmin Ur Rehman12-Mar-12 1:10 
QuestionVarying framerate [modified] PinmemberHalloko5-Nov-11 10:28 
AnswerRe: Varying framerate Pinmemberroman_gin6-Nov-11 8:38 
GeneralRe: Varying framerate PinmemberHalloko7-Nov-11 10:44 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140916.1 | Last Updated 15 Mar 2011
Article Copyright 2011 by Roman Ginzburg
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid