Silverlight 4 did not have P/Invoke support for calling native functions; Silverlight 5 does (see Pete Brown's blog for more details). Our objectives for this HowTo article is no different from my previous article with almost exactly the same Title. That article dealt with COM support. It is important to go through that article first - most of the steps covered there won't be covered here.
So in short, just as the previous article, this HowTo article tries to demonstrate the power of the
MediaStreamSource classes with P/Invoke support to call native code to play an AVI video.
Please Note the second download (
CMS_S5_plusWriteableBitmap) includes displaying the same Video Samples using the
WriteableBitmap. Code not explained here.
MediaElement (MediaStreamSource) as well as WriteableBitmap Implementation
As of 1st of September 2011, Silverlight 5 RC has been made available for download (see Pete's blog - link above). Silverlight 5 comes with the Platform Invoke support for calling native code (among many other great features).
To recap from the previous article, we need to first derive our custom class from
System.Windows.Media.MediaStreamSource. This will require us to override a number of methods. Without going into too much detail, the methods are
SwitchMediaStreamAsync. I will not dig deep into defining these methods, but the ones we shall use in our example code are:
OpenMediaAsync: We override this method, and in it, we initialize and report some metadata about the media by calling the
GetSampleAsync: We override this method and retrieve the next requested sample by the
MediaElement will call this method every time it needs a sample. To report back to
MediaElement that the sample is ready, we call the
Our main objective in this article is to write a simple Silverlight application that plays back an AVI video. Well, for the video (.avi) to play, you must have the relevant codec on your machine first.
We shall use the following simple steps to achieve our goal:
- Prepare a simple UI with a
MediaElement control, two
buttons, and a
- Write our custom class that is derived from
System.Windows.Media.MediaStreamSource and override all the required methods.
- Set our custom class as the source stream to the
MediaElement control in UI, behind-code.
This sample was tested using Silverlight 5 RC?
- Create a new Silverlight 5 Project (VB.NET) and give it any name you wish.
- Make sure you set the Project Properties as above enabling the out-of-browser and ensuring "Require elevated trust ..." is checked.
- Open the default created
UserControl named MainPage.xaml.
- Change the XAML code to resemble that shown below, which includes the
mediaPlayer) and two
<Grid x:Name="LayoutRoot" Background="Black">
<Button Content="Open AVI File" Height="23" Margin="29,12,0,0"
Name="OpenStream" HorizontalAlignment="Left" Width="123"
<Button Content="Close AVI File" Height="23" Margin="175,12,0,0"
Name="CloseStream" HorizontalAlignment="Left" Width="123"
mediaPlayer will be used to display our video. The button
OpenStream will be used to initialize our custom
MediaStreamSource object and assign it as a media stream to
mediaPlayer. The button
CloseStream will be used to close and stop the stream.
We will come back to the UI behind code and connect the remaining code.
Please follow Step 2 of my previous article as it is almost the same as most of the code in this project.
Let us modify the overridden stub and call the method (
retrieveSample) that retrieves a
Sample everytime the
mediaElement calls for one. Of course, this is not the best implementation - you may want to spawn a thread and immediately return and let the thread signal to
mediaElement when it has a
Protected Overrides Sub GetSampleAsync
(mediaStreamType As System.Windows.Media.MediaStreamType)
If mediaStreamType = mediaStreamType.Video Then
Again, as in the previous article, I decided to create a
stream that contains only one sample at a time.
Let us now look at the method
Private Sub retrieveSample()
' I CANNOT REMEMBER WHERE I GOT THIS CALCULATION FROM ONLINE
' it basically calculates the frame number based on time passed
' since start of play
If initcapture = 0 Then
timeexpended = timeGetTime() - initialtime
frametime = 0
timeexpended = 0
initialtime = timeGetTime()
If initcapture = 0 Then
If timelapse > 1000 Then
frametime = frametime + 1
frametime = (timeexpended / 1000) * (1000 / _speed) 'timelapse)
If frametime >= numFrames Then
initcapture = 0
' Get the requested Frame from the Stream - frame number "frametime" provided.
' Return a Pointer to the DIB
pDIB = AVIStreamGetFrame(pGetFrameObj, frametime)
If pDIB <> 0 Then
'Copy the Frame bits into RGB_Sample
'Call CopyMemory(RGB_Sample(0), pDIB + Marshal.SizeOf(bih), bih.biSizeImage)
Marshal.Copy(pDIB + Marshal.SizeOf(bih), RGB_Sample, 0, bih.biSizeImage)
If Not RGB_Sample Is Nothing Then
For verticalCount As Integer = RGB_Sample.Length - 1 _
To 0 Step _frameWidth * RGBByteCount * -1
For horizontalCount As Integer = 0 To _frameWidth - 1
' Calculate the next pixel position from the original Sample
' based on the outer loop, it is calculated from bottom-right
pixelPos = verticalCount - (_frameWidth * RGBByteCount) + _
(horizontalCount * RGBByteCount) + 1
RGBA_Sample(j) = RGB_Sample(pixelPos)
RGBA_Sample(j + 1) = RGB_Sample(pixelPos + 1)
RGBA_Sample(j + 2) = RGB_Sample(pixelPos + 2)
' Assign 1 byte for the Alpha Channel
RGBA_Sample(j + 3) = 255 '&FF
'jump 4 bytes for the RGBA byte counter
j += RGBAByteCount
j = 0
' Instantiate a Sample
' The Sample has two members (Time & Buffer)
Dim _sample As Sample = New Sample()
_sample.sampleBuffer = RGBA_Sample
_sample.sampleTime = DateTime.Now
' We always start at the beginning of the stream
' this is because we always reset the stream with one sample at a time
' if you decide to add more than one sample into the stream then you
' can modify the logic to increment this offset by the size of the sample
' everytime there is a call to return a sample
_offset = 0
' write the retrieved Sample into the stream
' remember our stream is just one Sample
_stream.Write(_sample.sampleBuffer, 0, _count)
Dim mediaSample As MediaStreamSample = New MediaStreamSample(
_timeStamp += TimeSpan.FromSeconds(1 / _speed).Ticks
' report back a successful Sample
The method above is what gets called everytime a
Sample is requested by the
mediaElement. In order to know what Frame to extract from the video stream, we calculate based on the current time since the time the playback started. so, in short, frametime is the frame number in the video.
We use the native function
AVIStreamGetFrame to return a
Pointer to a Packed DIB by passing it the
Frames object and the frame number to extract. If all goes well, we ignore the Bitmap Info Header and start copying the frame Bytes just after the Info Header the size of the frame in Bytes into
Now we have raw Bytes representing our uncompressed Sample that is three bytes per pixel. THe Sample that we shall pass back to the
MediaElement stream is also uncompressed but of type RGBA which is 4 bytes per pixel. We therefore need to convert from RGB to RGBA by adding an extra byte to represent the Alpha channel.
If you directly assign the first byte from
RGB_Sample to the first byte in
RGBA_Sample, the image will turn out to be upside-down - at least which is what I get. To flip the image to its true orientation, we start assigning the last byte from
RGB_Sample to the first byte of
RGBA_Sample, and for every fourth byte of
RGBA_Sample, we set the Alpha channel to 0xFF or
' assume N number of pixels in the original RGB Sample
' and M pixels in RGBA Sample
RGBA_Sample[Pixel 1] = frameBytes[Pixel N]
RGBA_Sample[Pixel 2] = frameBytes[Pixel N-1]
RGBA_Sample[Pixel 3] = frameBytes[Pixel N-2]
RGBA_Sample[Pixel 4] = 255
RGBA_Sample[Pixel M - 3] = frameBytes[Pixel 3]
RGBA_Sample[Pixel M - 2] = frameBytes[Pixel 2]
RGBA_Sample[Pixel M - 1] = frameBytes[Pixel 1]
RGBA_Sample[Pixel M] = 255
The above will not be possible without the code for opening the AVI file and getting the memory Pointers to the file itself and the
Stream Information. This is made possible by using the native functions
Public Function startVideo(fname As String) As Boolean
' Return a Pointer of the File in Memory assigned to pAVIFile
Dim res As Int32 = AVIFileOpen(pAVIFile, fname, 32, 0)
' Return a Pointer of the Video Stream in Memory assigned to pAVIStream
res = AVIFileGetStream(pAVIFile, pAVIStream, streamtypeVIDEO, 0)
' Return the First Video Frame Number
firstFrame = AVIStreamStart(pAVIStream)
' Return the Total Number of Frames in the Stream
numFrames = AVIStreamLength(pAVIStream)
' Return the Video Stream Information
res = AVIStreamInfo(pAVIStream, streamInfo, Marshal.SizeOf(streamInfo))
_speed = streamInfo.dwRate 'fps
.biBitCount = 24
.biClrImportant = 0
.biClrUsed = 0
.biCompression = BI_RGB
.biHeight = streamInfo.rcFrame.bottom - streamInfo.rcFrame.top
.biPlanes = 1
.biSize = 40
.biWidth = streamInfo.rcFrame.right - streamInfo.rcFrame.left
.biXPelsPerMeter = 0
.biYPelsPerMeter = 0
.biSizeImage = (((.biWidth * 3) + 3) And &HFFFC) * .biHeight
_frameWidth = bih.biWidth
_frameHeight = bih.biHeight
_count = _frameHeight * _frameWidth * _framePixelSize
_frameStreamSize = _count
numScans = IIf(_frameHeight > _frameWidth, _frameHeight, _frameWidth)
' Retrieve the Frames Object from the stream byte by supplying
' the Stream and the format we expect
pGetFrameObj = AVIStreamGetFrameOpen(pAVIStream, bih)
If pGetFrameObj = 0 Then Return False 'ERROR
' Resize our actual Samples in Bytes
ReDim RGB_Sample(bih.biSizeImage - 1)
' Resize our final modified sample in Bytes
ReDim RGBA_Sample(_count - 1)
Now, let's write some code that hooks up our UI to our custom derived class to complete our objective.
In our MainPage.xaml code-behind, we first instantiate our custom derived
MediaStreamSource class. To bring everything into action, initialize our custom
MediaStreamSource and call its
startVideo(_filename) to open our video and start buffering such that when our
MediaElement requests for the first sample (and subsequent ones), our derived object will be ready to satisfy those requests. Finally, set our custom
MediaStreamSource object as the source of our media to the
MediaElement and voila, our application is ready to render .avi videos.
Partial Public Class MainPage
Dim _mediaSource As MyDerivedMediaStreamSource
Dim mediaOpen As Boolean = False
Dim _filename As String = "Video_File_Full_Path_Here.avi"
Public Sub New()
Private Sub OpenStream_Click(sender As System.Object, _
e As System.Windows.RoutedEventArgs) Handles OpenStream.Click
_mediaSource = New MyDerivedMediaStreamSource()
If _mediaSource.startVideo(_filename) Then
mediaOpen = True
Private Sub CloseStream_Click(sender As System.Object, _
e As System.Windows.RoutedEventArgs) Handles CloseStream.Click
If mediaOpen Then
_mediaSource = Nothing
mediaOpen = False
Please change "Video_File_Full_Path_Here.avi" to your own file. It must be a .avi file and you must have the relevant codec installed on your computer for the media to be decompressed.
I hope this simple HowTo article was helpful to all.