Click here to Skip to main content
15,867,568 members
Articles / Multimedia / Video

EVR Presenter in pure C# with Direct3D Video Rendering

Rate me:
Please Sign up or sign in to vote.
4.96/5 (15 votes)
11 Jul 2012CPOL6 min read 78.1K   4K   22   28
Articles describes how to make pure C# rendering video on EVR with custom presenter over Direct3D in .NET

Image 1

Introduction

This is my second article of customizing video output with .NET and in pure C# code. More stuff is similar to my previous post so please review it, as I will not describe common stuff. This tutorial I think for advanced developers.

Before start

First you should read the article how to build EVR Presenter on MSDN especially prerequisites part.

Multimedia threading VS .NET threading

Yes, we should understand that stuff because .NET threading is totally different. MSDN description is good but not enough.

.NET threads and objects are all works and exist in specified execution context. Some objects are not available to be accessed from different threads for example form and controls. Other objects can be accessed from another thread but that require .NET to switch between execution contexts (there object created and there it is accessed). That operation can take a while or even hang the execution. Hanging can appear due different threading model. As in our case ALL multimedia threads are works in same context so you can access object created in other thread without any problem, only don’t forget about synchronization object. Hope I clear the difference and we can proceed.

Tracing, Debugging and Exception

I want to say some words regarding that before reviewing the code. Due different threading stuff I not suggest you to use the things like Trace.Write or Debug.Write in any code which processing multimedia data or accessing unmanaged resources frequently. This is also related to issue with switching threading context and as result degrade the performance. Raising exceptions are not recommended in the same issue, so better to checking returned values, using try catch statements also recommended. The way to solve tracing output is to use of OutputDebugString API.

C#
public static void TRACE(string _message) 
{ 
    if (!string.IsNullOrEmpty(_message)) _message += "\n"; 
    API.OutputDebugString(_message); 
}

Another helper function which will be useful along with above

C#
public static void TRACE_ENTER() 
{ 
    MethodBase _method = (new StackTrace(1,false)).GetFrame(0).GetMethod(); 
    TRACE(string.Format("{0}::{1}", _method.ReflectedType.Name, _method.Name)); 
}

This function print to an output window caller class name and method name.

Application Overview

Demo application shows how to perform video playback using DirectShow with Enhanced Video Renderer (EVR) with custom presenter. Presenter performing allocating the surfaces for playback, performing media type negotiations, synchronization of surfaces time stamps and display frames to the user using Direct3D9. Presenting surfaces done using SlimDX (managed library for Direct3D) similar as in my previous article.

Implementation scene presenting

Presenting the scene is similar to here.

Filter Graph

Filter graph a little different but it also particular graph for playback application, just it used the Enhanced Video Renderer instead of default and look like this:

Image 2

Class declaration and initialization

Same way I use my classes for graph building so the playback class declaration looks:

C#
public class DSFilePlaybackEVR : DSFilePlayback
    , IMFVideoDeviceID 
    , IMFVideoPresenter 
    , IMFGetService 
    , IMFTopologyServiceLookupClient

Here the inherited interfaces are required for the custom EVR presenter. We also have an event delegate and event variable in class to notify the scene that the surface is ready for display. The EVR filters declaration is:

C#
[Guid("FA10746C-9B63-4b6c-BC49-FC300EA5F256")] 
public class EVRRenderer : DSFilter 
{ 
    public EVRRenderer() 
        : base() 
    { 
    } 
}

EVR Class derived from base graph builder class which handles all basic stuff for playback via DirectShow we just need to override methods for initialization filters and connecting them:

C#
protected override HRESULT OnInitInterfaces() 
{ 
    m_evStop.Reset(); 
    HRESULT hr; 
    hr = (HRESULT)MFHelper.DXVA2CreateDirect3DDeviceManager9(out m_DeviceResetToken, out m_DeviceManager); 
    hr.Assert(); 
    if (hr.Succeeded) 
    { 
        hr = (HRESULT)m_DeviceManager.ResetDevice(Marshal.GetObjectForIUnknown(m_Device.ComPointer), m_DeviceResetToken); 
        hr.Assert(); 
    } 
    m_Caller = new Wrapper(this);  
    m_Renderer = new EVRRenderer(); 
    IMFVideoRenderer _renderer = (IMFVideoRenderer)m_Renderer.QueryInterface(typeof(IMFVideoRenderer)); 
    hr = (HRESULT)_renderer.InitializeRenderer(null, (IMFVideoPresenter)this); 
    hr.Assert(); 
    m_Renderer.FilterGraph = m_GraphBuilder; 
    hr = base.OnInitInterfaces(); 
    hr.Assert(); 
    return hr; 
}

Code fairly simple: we initialize DXVA2 device manager, create EVR filter and put it into the graph. After performing initialization EVR filter with our Presenter.

Implementing Presenter Interfaces

Now time for hardest part and you will know why I describe things regarding threading.

Invoker

Some methods of interfaces provided by my class for EVR Presenter is called from different threads, as if there will be one thread we will have no issues. But in there at least 2, commonly 3: user interaction (Play Pause Stop), media streaming (Samples delivering) and Clock (Samples Synchronization). Interfaces which are called at same context are IMFVideoDeviceID and IMFGetService. With other intarfaces we should do something to make it work properly, how to do so? The answer is simple: to make them to be called in same thread so context will be same. Hope you good enough with threading and synchronization to understand following code. Let’s look how implemented IMFTopologyServiceLookupClient:

C#
public int InitServicePointers(IntPtr pLookup)
{
    Wrapper.CCallbackHandler _handler =
                 new Wrapper.InitServicePointersHandler(m_Caller, pLookup);
    _handler.Invoke();
    return _handler.m_Result;
}

public int ReleaseServicePointers()
{
    Wrapper.CCallbackHandler _handler =
                 new Wrapper.ReleaseServicePointersHandler(m_Caller);
    _handler.Invoke();
    return _handler.m_Result;
}

Here we are simplify creates the specified pre-defined async invoker, wait for it result and return it. The invoker callback base class looks next:

C#
public class CCallbackHandler
{
    public bool m_bAsync = false;
    public CallType m_Type = CallType.Unknown;
    public EventWaitHandle m_Notify = new ManualResetEvent(false);
    public int m_Result = S_OK;
    private Wrapper m_Invoker = null;

    #region Constructor

    public CCallbackHandler(Wrapper _Invoker)
    {
         m_Invoker = _Invoker;
    }

    #endregion

    #region Methods

    public void Invoke()
    {
         m_Invoker.InvokeThread(this);
         WaitHandle.WaitAny(new WaitHandle[] { this.m_Notify, m_Invoker.m_Quit });
    }

    #endregion
}

And the actual invokers thread methods:

C#
private void InvokeThread(object _param)
{
	lock (m_LockThread)
	{
         		m_Parameter = _param;
                  m_Notify.Set();
	}
	WaitHandle.WaitAny(new WaitHandle[] { m_Quit, m_Ready });
}

private void ThreadProc(object _state)
{
    while (true)
    {
                 int nWait = WaitHandle.WaitAny(new WaitHandle[] { m_Quit, m_Notify });
                 if (nWait == 1)
                 {
                         object _param;
                         lock (m_LockThread)
                         {
                                _param = m_Parameter;
                         }
                         m_Ready.Set();
                         AsyncInvokerProc(_param);
                 }
                 else
                 {
                          break;
                 }
    }
}

We put the caller object as parameter after set the notify event which wakes up the thread to process the parameter and waits until the parameter will be retrieved. The thread got the parameter and executes passed callback; so all access to managed resources become from single thread.

Advanced Marshaling

Hope you still here; as previous part was not last hard code. Once the method is called with the invoker in same thread we can without any problems hold COM interfaces in our class. But someone who try to make EVR Presenter in .NET I’m sure had an issue with the InitServicePointers method of IMFTopologyServiceLookupClient interface, right? The issue appear, as I remember (I wrote that code year or two ago) was with query IMFTopologyServiceLookUp from pLookUp. Issue happened because of COM have aggregation and the returned interface may not be the interface of an actual object and the .NET doesn’t handle that it just call the QueryInterface and fail if it not in there, but we are know it is here. To solve this we just can access the vtable (table of virtual methods) of interface we interested in. All entry in that table are the pointers to the functions in order of interface inheritance and interface methods (actually interface it is structure with the specified methods entries and nothing else). As an example first 3 entries in interface are always implementation of IUnknown in order QueryInterface, AddRef and Release, That is true for all managed interfaces too, plus also for managed objects, as all managed objects are COM objects by default (that just hidden from developers). So I made the helper class which allows accessing COM object by it vtable:

C#
public class VTableInterface : COMHelper,IDisposable
{
    #region Delegates

    private delegate int QueryInterfaceProc(
        IntPtr pUnk,
        ref Guid riid,
        out IntPtr ppvObject
        );

    #endregion

    #region Variables

    protected IntPtr m_pUnknown = IntPtr.Zero;

    #endregion

    #region Constructor

    protected VTableInterface(IntPtr pUnknown)
    {
        if (pUnknown != IntPtr.Zero)
        {
            m_pUnknown = pUnknown;
            Marshal.AddRef(m_pUnknown);
        }
    }
    ~VTableInterface()
    {
        Dispose();
    }

    #endregion

    #region Methods

    public int QueryInterface(ref Guid riid, out IntPtr ppvObject)
    {
        ppvObject = IntPtr.Zero;
        if (m_pUnknown == IntPtr.Zero) return E_NOINTERFACE;
        QueryInterfaceProc _Proc = GetProcDelegate<QueryInterfaceProc>(0);
        if (_Proc == null) return E_UNEXPECTED;
        return (HRESULT)_Proc(m_pUnknown,ref riid,out ppvObject);
    }
    #endregion

    #region Helper Methods

    protected T GetProcDelegate<T>(int nIndex) where T : class
    {
        IntPtr pVtable = Marshal.ReadIntPtr(m_pUnknown);
        IntPtr pFunc = Marshal.ReadIntPtr(pVtable, nIndex * IntPtr.Size);
        return (Marshal.GetDelegateForFunctionPointer(pFunc, typeof(T))) as T;
    }

    #endregion

    #region IDisposable Members

    public void Dispose()
    {
        if (m_pUnknown != IntPtr.Zero)
        {
            Marshal.Release(m_pUnknown);
            m_pUnknown = IntPtr.Zero;
        }
    }
    #endregion
}

The main interesting method here is GetProcDelegate which allows getting method from vtable by it index. How it works you can see in QueryInterface implementation. So to implement IMFTopologyServiceLookUp without any problems we make next code:

C#
public class MFTopologyServiceLookup : VTableInterface, IMFTopologyServiceLookup
…
private delegate int LookupServiceProc(
            IntPtr pUnk,
            MFServiceLookUpType Type,
            uint dwIndex,
            [In, MarshalAs(UnmanagedType.LPStruct)] Guid guidService,
            [In, MarshalAs(UnmanagedType.LPStruct)] Guid riid,
            [Out, MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.SysInt)] IntPtr[] ppvObjects,
            [In, Out] ref uint pnObjects);
….
public int LookupService(MFServiceLookUpType _type, uint dwIndex, Guid guidService, Guid riid, IntPtr[] ppvObjects, ref uint pnObjects)
{
            if (m_pUnknown == IntPtr.Zero) return E_NOINTERFACE;
            LookupServiceProc _lookUpProc = GetProcDelegate<LookupServiceProc>(3);
            if (_lookUpProc == null) return E_UNEXPECTED;
            return (HRESULT)_lookUpProc(
                        m_pUnknown,
                        _type,
                        dwIndex,
                        guidService,
                        riid,
                        ppvObjects,
                        ref pnObjects
                        );
}

Not so hard I think. Forgot to mention the interface delegate function differ from method declaration in interface in one additional argument. First argument should be the pointer to the vtable object or our IntPtr, why that necessary you can find over web I think.

Samples Scheduler and notify of free sample

If you look at the EVR Presenter C++ example from Microsoft you can find that it define free samples while it released, I mean called Release with specified notification set on that. .NET doesn’t allow us to use that method as we have no access to the IUnknown directly. So I solve that with usage of events (probably for someone better to use semaphores, but this is just an example).

Main Application

Implementation of main form is very easy, most interesting methods I describe here. Variable declaration for scene and playback:

C#
private Scene m_Scene = null; 
private DSFilePlayback m_Playback = null;

Creating scene object:

C#
m_Scene = new Scene(this.pbView);

Here pbView control on which we’ll do presenting the video. Starting playback code:

C#
m_Playback = new DSFilePlaybackEVR(m_Scene.Direct3DDevice); 
m_Playback.OnPlaybackStop += new EventHandler(btnStart_Click); 
((DSFilePlaybackEVR)m_Playback).OnSurfaceReady += new EVR.SurfaceReadyHandler(m_Scene.OnSurfaceReady); 
m_Playback.FileName = this.tbFileName.Text; 
if (m_Playback.Start().Succeeded) 
{ 
    btnStart.Text = "Stop"; 
    btnBrowse.Enabled = false; 
}

In this code we create EVR rendering graph with specified Scene device. After we provide event handler for surface ready notify, and starting playback. Stopping code is fairly simple – just Dispose the playback:

C#
m_Playback.Dispose(); 
m_Playback = null;

History

Initial Version 11-07-2012

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Software Developer (Senior)
Russian Federation Russian Federation
I'm a professional multimedia developer (more than 10 years) in any kind of applications and technologies related to it, such as DirectShow, Direct3D, WinMM, OpenGL, MediaFoundation, WASAPI, Windows Media and other including drivers development of Kernel Streaming, Audio/Video capture drivers and audio effects. Have experience in following languages: C, C++, C#, delphi, C++ builder, VB and VB.NET. Strong knowledge in math and networking.

Comments and Discussions

 
QuestionVideo from memory, MemoryStream Pin
Member 1094791220-Aug-15 6:33
Member 1094791220-Aug-15 6:33 
AnswerRe: Video from memory, MemoryStream Pin
Maxim Kartavenkov23-Aug-15 19:15
Maxim Kartavenkov23-Aug-15 19:15 
QuestionAbout aspect ratio Pin
k.sin22-Mar-15 23:50
k.sin22-Mar-15 23:50 
AnswerRe: About aspect ratio Pin
Maxim Kartavenkov31-Mar-15 8:30
Maxim Kartavenkov31-Mar-15 8:30 
QuestionRe: About aspect ratio Pin
k.sin9-Jun-15 17:21
k.sin9-Jun-15 17:21 
AnswerRe: About aspect ratio Pin
Maxim Kartavenkov30-Jun-15 23:32
Maxim Kartavenkov30-Jun-15 23:32 
QuestionDirectX Memory Leak Pin
Wayne Work 2117-Nov-14 1:59
Wayne Work 2117-Nov-14 1:59 
Hello Maxim,

Thank you for this great piece of code it has really helped me a lot.

I'm experiencing a small memory leak.
When I run your application using the debug mode of DirectX I get the following errors reported back:
I can reproduce the issue by playing a video, stopping it and then clicking Exit.

Object of type SlimDX.Direct3D9.Direct3DEx was not disposed. Stack trace of object creation:
Object of type SlimDX.Direct3D9.Surface was not disposed. Stack trace of object creation:
c:\Users\user\Desktop\EVRPresenterSrc\Sources\EVRPlayback\Scene.cs(77,17): Void OnSurfaceReady(SlimDX.Direct3D9.Surface ByRef)
c:\Users\user\Desktop\EVRPresenterSrc\Sources\EVRPlayback\EvrGraph.cs(2489,21): Sonic.HRESULT PresentSample(System.Object, Int64)
c:\Users\user\Desktop\EVRPresenterSrc\Sources\EVRPlayback\EvrGraph.cs(367,21): Sonic.HRESULT ScheduleSample(MFSample, Boolean)
c:\Users\user\Desktop\EVRPresenterSrc\Sources\EVRPlayback\EvrGraph.cs(2289,13): Sonic.HRESULT DeliverSample(MFSample, Boolean)
c:\Users\user\Desktop\EVRPresenterSrc\Sources\EVRPlayback\EvrGraph.cs(2252,29): Sonic.HRESULT ProcessOutput()
c:\Users\user\Desktop\EVRPresenterSrc\Sources\EVRPlayback\EvrGraph.cs(2113,17): Void ProcessOutputLoop()
c:\Users\user\Desktop\EVRPresenterSrc\Sources\EVRPlayback\EvrGraph.cs(2055,17): Sonic.HRESULT ProcessInputNotify()
c:\Users\user\Desktop\EVRPresenterSrc\Sources\EVRPlayback\EvrGraph.cs(1520,29): Int32 ProcessMessageImpl(EVR.MFVPMessageType, IntPtr)
c:\Users\user\Desktop\EVRPresenterSrc\Sources\EVRPlayback\EvrGraph.cs(889,29): Void AsyncInvokerProc(System.Object)
c:\Users\user\Desktop\EVRPresenterSrc\Sources\EVRPlayback\EvrGraph.cs(872,25): Void ThreadProc(System.Object)
Total of 2 objects still alive.
AnswerRe: DirectX Memory Leak Pin
Maxim Kartavenkov7-Nov-14 2:29
Maxim Kartavenkov7-Nov-14 2:29 
GeneralRe: DirectX Memory Leak Pin
Wayne Work 21112-Nov-14 3:06
Wayne Work 21112-Nov-14 3:06 
GeneralRe: DirectX Memory Leak Pin
Maxim Kartavenkov12-Nov-14 4:37
Maxim Kartavenkov12-Nov-14 4:37 
QuestionSeamless video loop Pin
wana9-Sep-14 4:44
wana9-Sep-14 4:44 
QuestionHow the EVR.cs had been created? Pin
Pirox25-Jul-14 3:06
Pirox25-Jul-14 3:06 
AnswerRe: How the EVR.cs had been created? Pin
Maxim Kartavenkov25-Jul-14 3:09
Maxim Kartavenkov25-Jul-14 3:09 
QuestionDXVA Hardware Decoding Pin
damikez27-Jan-13 15:31
damikez27-Jan-13 15:31 
AnswerRe: DXVA Hardware Decoding Pin
Maxim Kartavenkov6-Feb-13 20:40
Maxim Kartavenkov6-Feb-13 20:40 
QuestionMemory Leak Pin
2fast4all16-Nov-12 1:32
2fast4all16-Nov-12 1:32 
AnswerRe: Memory Leak Pin
Maxim Kartavenkov16-Nov-12 4:32
Maxim Kartavenkov16-Nov-12 4:32 
GeneralRe: Memory Leak Pin
2fast4all16-Nov-12 4:54
2fast4all16-Nov-12 4:54 
GeneralRe: Memory Leak Pin
Maxim Kartavenkov16-Nov-12 5:02
Maxim Kartavenkov16-Nov-12 5:02 
GeneralMy vote of 5 Pin
Bavarian12-Nov-12 8:14
Bavarian12-Nov-12 8:14 
QuestionDXVA2 decoder Pin
ghiaccio19-Oct-12 5:16
ghiaccio19-Oct-12 5:16 
AnswerRe: DXVA2 decoder Pin
Maxim Kartavenkov19-Oct-12 5:38
Maxim Kartavenkov19-Oct-12 5:38 
QuestionRe: DXVA2 decoder Pin
ghiaccio22-Oct-12 2:23
ghiaccio22-Oct-12 2:23 
AnswerRe: DXVA2 decoder Pin
Maxim Kartavenkov22-Oct-12 4:23
Maxim Kartavenkov22-Oct-12 4:23 
QuestionEVR hangs while playing large .ts file Pin
revengeoffallen10-Aug-12 3:17
revengeoffallen10-Aug-12 3:17 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.