Mahhala - WPF MediaKit

Mahhala (isiZulu word for free or without charge) is a new series that showcases some of the cool free WPF/Silverlight controls available…

“You can learn a lot by reading other people's source code. That's the idea behind this series, "The Weekly Source Code." You can certainly become a better programmer by writing code but I think good writers become better by reading as much as they can.” - Scott Hanselman

Not a lot of people know this but when I started working, I worked for some companies that produced security software… One of the projects I worked on was a DVR (Digital Video Recorder) written in C++! Thankfully times have changed and there are easier ways of doing this now! Today I chat with Jeremiah Morrill, the creator of the WPF MediaKit! A collection of controls that enable us to create DVR-like software!!!

• Source: http://wpfmediakit.codeplex.com/
• License: New BSD License (BSD)

RG - Welcome Jer, please tell us a little about yourself?
JM - In school, I thought I would become a graphic designer and spent four years working hard to do just that! I've always been interested in computer graphics so I felt this would be the best place for me. As fate would have it, I ended up writing code and people ended up paying me for it. I have worked on many platforms, in many languages, but I usually find myself spending the most time writing multimedia type applications. Performance in these applications are usually of utmost concern, which is why Direct3D and WPF are my favourite tools of choice.

RG - What is WPF MediaKit and how did it all start?
JM - When WPF was first released with .NET 3.0, I was totally excited about the multimedia abilities. It has always been quite a chore to even composite text over video, so when I saw this could be accomplished with a few XAML tags, I was in love. Like most love at first sight stories, it didn't last. I had hit a brick wall almost instantly when I attempted to create a control that rendered a webcam. MediaElement, WPF's only way of playing video, was not extensible. At the time, WriteableBitmap was ill suited for rendering video as it would internally allocate a new bitmap with every pixel update. This led me to develop a series of hacks to get beyond the limitations imposed by WPF. Notably the WPF MediaBridge and the VideoRendererElement. Both these projects leverage how the MediaElement works behind the scenes to access the high performance internals WPF was using. The solution worked very well, but many (including myself) felt the level of hackiness was too much.

.NET 3.5 SP1 features were announced and it had some great new additions to WPF. The features that really piqued my interest were the new WriteableBitmap and D3DImage. Both these could potentially be a solution for custom video sources in WPF. I was less excited about WriteableBitmap as rendering raw pixels is important, but it surely isn't the only thing involved in high performance video. After some experimentation and hearing about all the CPU bound operations that go along with WriteableBitmap, I decided it just wasn't well suited for the task. D3DImage looked very attractive for several reasons. The first is that it is fast! The developer has full control how the pixels on the Direct3D surface get written. Also the GPU -> GPU blit is fairly inexpensive. The second reason is that Window's multimedia subsystems, DirectShow and MediaFoundation directly support Direct3D, so interoperation would be pretty straight forward. Thirdly, maximum hardware acceleration. GPU decoding of certain media types and hardware accelerated color-space conversions and deinterlacing were possible.

Once I got a hot-off-the-compiler beta .NET with D3DImage (TAP access! Thanks Laurent!), I made a very rough proof of concept video player using D3DImage. After getting some feedback, I really only set out to make a better MediaElement. This is where MediaKit was born.

WPF Mediakit is an open source project that allows for high performance integration of Microsoft's multimedia subsystems (mostly DirectShow at the moment). Depending on your requirements, WPF MediaKit is a set of controls for video playback. It is an SDK if you wish to make your own custom players. It is an example of how to get the best (IMHO) video playback performance in WPF.

The controls MediaKit comes with cover what I felt were the most common requests:

• VideoCaptureElement - This element allows a developer to show a webcam or capture card in WPF.
• DvdPlayerElement - Allows you to play a DVD in WPF. It even supports interactive DVD menus. One must have his/her own audio codec installed as the one that comes with Windows has license restrictions (and locked to Microsoft players).
• MediaUriElement - This is the a MediaElement replacement. It can play all files DirectShow can, where MediaElement was a hit-or-miss. It supports output to any audio device, bindable Position property and more...

RG - I am a little jealous of all the cool features Silverlight is getting, should WPF also get a simplified way of detecting and using Webcams/Mics like Silverlight?
JM - The jealousy is totally understandable. Silverlight got the VSM first. At the moment, it exclusively has DeepZoom and the Pivot controls. As far as Webcam/Mics, I'm not particularly jealous...yet. The Silverlight API for webcam/mic is very well thought out and has an extremely low barrier to entry. One downside is, in the context of motion video, it's relatively CPU intensive in rendering the pixels. The other downside is encoding. It is very impractical to encode video in managed code and this is why you will not see any useful motion video encoders (audio is much less complex) in managed Silverlight. Even though I do not supply any encoding features within WPF MediaKit, the opportunity is still there. It is when Silverlight gets media encoding features that I will turn green with envy and beg for parity with Silverlight in this area!

RG - Where can readers learn more about you and your projects?
JM - I usually write (see ramble) about my projects and miscellaneous commentary on my blog.

My newest projects include:

• The SilverlightViewport which allows running Silverlight applications in WPF/XNA without airspace restrictions and without a web browser.
• The SilverlightEncoder, which will encode/store/stream audio and video from a Silverlight OOB+E application. Oddly enough, this is more of an example of how to "hack" Silverlight to run native code without COM registration than a useful utility.

Let’s give it a go… First, let's get a list of available video capture devices:

<ComboBox x:Name="videoCapDevices"
     ItemsSource="{Binding Source={x:Static Controls:MultimediaUtil.VideoInputDevices}}"
     DisplayMemberPath="Name" />

And then create a VideoCaptureElement:

<WPFMediaKit:VideoCaptureElement x:Name="videoCapElement"
                                 LoadedBehavior="Play"
                                 DesiredPixelWidth="320"
                                 DesiredPixelHeight="240"
                                 Stretch="Fill"
                                 VideoCaptureSource="{Binding Path=CaptureDeviceName}"
                                 FPS="30" />

That’s it!!!

• Mahhala – AvalonDock
• Mahhala – Fluent Ribbon Control Suite
• Mahhala – Bag o' Tricks
• Mahhala - WPF NotifyIcon
• Mahhala - FluidKit

CodeProject