Click here to Skip to main content
13,295,616 members (37,457 online)
Click here to Skip to main content
Add your own
alternative version

Stats

172.2K views
31.9K downloads
80 bookmarked
Posted 18 Mar 2015

Stream Player control

, 17 Oct 2017
Rate this:
Please Sign up or sign in to vote.
In this article you will find an implementation of a stream player control.

Introduction

This article is a sort of continuation of my previous article, which shows an implementation of a web camera control. Recently I created another control and would like to share my experience with community. It is a FFmpeg-based stream player control, which can do the following:

  1. Play a RTSP/RTMP video stream or local video file
  2. Retrieve the current frame being displayed by the control

The control has no additional dependencies and a minimalistic interface.

Requirements

  1. The WinForms version of the control is implemented using .NET Framework 2.0
  2. The WPF version of the control is implemented using .NET Framework 4 Client Profile

The control supports both x86 and x64 platform targets.

Background

Streaming audio, video and data over the Internet is a very usual thing these days. However, when I tried to find a .NET control to play a video stream sent over the network, I found almost nothing. This project tries to fill up that gap.

Implementation details

If you are not interested in implementation details, then you can skip this section.

The implementation is divided into three layers.

  1. The bottom layer is implemented as a native DLL module, which forwards our calls to the FFmpeg framework.
  2. For distribution convenience, the native DLL module is embedded into the control’s assembly as a resource. On the runtime stage, the DLL module will be extracted to a temporary file on disk and used via late binding technique. Once the control is disposed, the temporary file will be deleted. In other words, the control is distributed as a single file. All those operations are implemented by the middle layer.
  3. The top layer implements the control class itself.

The following diagram shows a logical structure of the implementation.

Only the top layer is supposed to be used by clients.

The Bottom Layer

The bottom layer uses the facade pattern to provide a simplified interface to the FFmpeg framework. The facade consists of three classes: the StreamPlayer class, which implements a stream playback functionality

/// <summary>
/// The StreamPlayer class implements a stream playback functionality.
/// </summary>
class StreamPlayer : private boost::noncopyable
{
public:

    /// <summary>
    /// Initializes a new instance of the StreamPlayer class.
    /// </summary>
    StreamPlayer();

    /// <summary>
    /// Initializes the player.
    /// </summary>
    /// <param name="playerParams">The StreamPlayerParams object that contains the information that is used to initialize the player.</param>
    void Initialize(StreamPlayerParams playerParams);

    /// <summary>
    /// Asynchronously plays a stream.
    /// </summary>
    /// <param name="streamUrl">The url of a stream to play.</param>
    void StartPlay(std::string const& streamUrl);
    
    /// <summary>
    /// Retrieves the current frame being displayed by the player.
    /// </summary>
    /// <param name="bmpPtr">Address of a pointer to a byte that will receive the DIB.</param>
    void GetCurrentFrame(uint8_t **bmpPtr);

    /// <summary>
    /// Retrieves the unstretched frame size, in pixels.
    /// </summary>
    /// <param name="widthPtr">A pointer to an int that will receive the width.</param>
    /// <param name="heightPtr">A pointer to an int that will receive the height.</param>
    void GetFrameSize(uint32_t *widthPtr, uint32_t *heightPtr);

    /// <summary>
    /// Uninitializes the player.
    /// </summary>
    void Uninitialize();
};

the Stream class, which converts a video stream into series of frames

/// <summary>
/// A Stream class converts a stream into series of frames. 
/// </summary>
class Stream : private boost::noncopyable
{
public:
    /// <summary>
    /// Initializes a new instance of the Stream class.
    /// </summary>
    /// <param name="streamUrl">The url of a stream to decode.</param>
    Stream(std::string const& streamUrl);

    /// <summary>
    /// Gets the next frame in the stream.
    /// </summary>
    /// <returns>The next frame in the stream.</returns>
    std::unique_ptr<Frame> GetNextFrame();

    /// <summary>
    /// Gets an interframe delay, in milliseconds.
    /// </summary>
    int32_t InterframeDelayInMilliseconds() const;

    /// <summary>
    /// Releases all resources used by the stream.
    /// </summary>
    ~Stream();
};

and the Frame class, which is a set of frame related utilities.

/// <summary>
/// The Frame class implements a set of frame-related utilities. 
/// </summary>
class Frame : private boost::noncopyable
{
public:
    /// <summary>
    /// Initializes a new instance of the Frame class.
    /// </summary>
    Frame(uint32_t width, uint32_t height, AVPicture &avPicture);

    /// <summary>
    /// Gets the width, in pixels, of the frame.
    /// </summary>
    uint32_t Width() const { return width_; }

    /// <summary>
    /// Gets the height, in pixels, of the frame.
    /// </summary>
    uint32_t Height() const { return height_; }

    /// <summary>
    /// Draws the frame.
    /// </summary>
    /// <param name="window">A container window that frame should be drawn on.</param>
    void Draw(HWND window);

    /// <summary>
    /// Converts the frame to a bitmap.
    /// </summary>
    /// <param name="bmpPtr">Address of a pointer to a byte that will receive the DIB.</param>
    void ToBmp(uint8_t **bmpPtr);

    /// <summary>
    /// Releases all resources used by the frame.
    /// </summary>
    ~Frame();
};

These tree classes form a heart of the FFmpeg Facade DLL module.

The Middle Layer 

The middle layer is implemented by the StreamPlayerProxy class, which serves as a proxy to the FFmpeg Facade DLL module.

First, what we should do is extract the FFmpeg Facade DLL module from the resources and save it to a temporary file.

_dllFile = Path.GetTempFileName();
using (FileStream stream = new FileStream(_dllFile, FileMode.Create, FileAccess.Write))
{
    using (BinaryWriter writer = new BinaryWriter(stream))
    {
        writer.Write(Resources.StreamPlayer);
    }
}

Then we load our DLL module into the address space of the calling process.

_hDll = LoadLibrary(_dllFile);
if (_hDll == IntPtr.Zero)
{
    throw new Win32Exception(Marshal.GetLastWin32Error());
}

And bind the DLL module functions to the class instance methods.

private delegate Int32 StopDelegate();
private StopDelegate _stop;

// ...

IntPtr procPtr = GetProcAddress(_hDll, "Stop");
_stop =
    (StopDelegate)Marshal.GetDelegateForFunctionPointer(procPtr, 
     typeof(StopDelegate));

When the control is being disposed, we unload the DLL module and delete it.

private void Dispose()
{    
    if (_hDll != IntPtr.Zero)
    {
        FreeLibrary(_hDll);
        _hDll = IntPtr.Zero;
    }

    if (File.Exists(_dllFile))
    {
        File.Delete(_dllFile);
    }    
}

The Top Layer

The top layer is implemented by the StreamPlayerControl class with the following interface.

/// <summary>
/// Asynchronously plays a stream.
/// </summary>
/// <param name="uri">The url of a stream to play.</param>
/// <exception cref="ArgumentException">An invalid string is passed as an argument.</exception>
/// <exception cref="Win32Exception">Failed to load the FFmpeg facade dll.</exception>
/// <exception cref="StreamPlayerException">Failed to play the stream.</exception>
public void StartPlay(Uri uri)

/// <summary>
/// Retrieves the image being played.
/// </summary>
/// <returns>The current image.</returns>
/// <exception cref="InvalidOperationException">The control is not playing a video stream.</exception>
/// <exception cref="StreamPlayerException">Failed to get the current image.</exception>
public Bitmap GetCurrentFrame();

/// <summary>
/// Stops a stream.
/// </summary>
/// <exception cref="InvalidOperationException">The control is not playing a stream.</exception>
/// <exception cref="StreamPlayerException">Failed to stop a stream.</exception>
public void Stop();

/// <summary>
/// Gets a value indicating whether the control is playing a video stream.
/// </summary>
public Boolean IsPlaying { get; }

/// <summary>
/// Gets the unstretched frame size, in pixels.
/// </summary>
public Size VideoSize  { get; }

/// <summary>
/// Occurs when the first frame is read from a stream.
/// </summary>
public event EventHandler StreamStarted;

/// <summary>
/// Occurs when there are no more frames to read from a stream.
/// </summary>
public event EventHandler StreamStopped;

/// <summary>
/// Occurs when the player fails to play a stream.
/// </summary>
public event EventHandler StreamFailed;

Usage

Open the Package Manager Console and add a nuget package to your project:

Install-Package WebEye.Controls.WinForms.StreamPlayerControl

First, we need to add the control to the Visual Studio Designer Toolbox, using a right-click and then the "Choose Items..." menu item. Then we place the control on a form at the desired location and with the desired size. The default name of the control instance variable will be streamPlayerControl1.

The following code asynchronously plays a stream using the supplied address.

streamPlayerControl1.StartPlay(new Uri("rtsp://184.72.239.149/vod/mp4:BigBuckBunny_115k.mov"));

There is also an option to specify a connection timeout and underlying transport protocol.

streamPlayerControl1.StartPlay(new Uri("rtsp://184.72.239.149/vod/mp4:BigBuckBunny_115k.mov"),
    TimeSpan.FromSeconds(15), RtspTransport.UdpMulticast);

To get a frame being played just call the GetCurrentFrame() method. The resolution and quality of the frame depend on the stream quality.

using (Bitmap image = streamPlayerControl1.GetCurrentFrame())
{
    // image processing...
}

To stop the stream the Stop() method is used.

streamPlayerControl1.Stop();

You can always check the playing state using the following code.

if (streamPlayerControl1.IsPlaying)
{
    streamPlayerControl1.Stop();
}

Also, the StreamStarted, StreamStopped and StreamFailed events can be used to monitor the playback state.

To report errors, exceptions are used, so do not forget to wrap your code in a try/catch block. That is all about using it. To see a complete example please check the demo application sources.

WPF version

The FFmpeg facade expects a WinAPI window handle (HWND) in order to use it as a render target. The issue is that in the WPF world windows do not have handles anymore. The VideoWindow class workarounds this issue.

<UserControl x:Class="WebEye.StreamPlayerControl"

             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"

             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"

             xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" 

             xmlns:d="http://schemas.microsoft.com/expression/blend/2008" 

             mc:Ignorable="d" 

             d:DesignHeight="300" d:DesignWidth="300"

             xmlns:local="clr-namespace:WebEye">
    <local:VideoWindow x:Name="_videoWindow"

                       HorizontalAlignment="Stretch" VerticalAlignment="Stretch"/>
</UserControl>

To add a WPF version of the control to your project use the following nuget command:

Install-Package WebEye.Controls.Wpf.StreamPlayerControl

GitHub

The project has a GitHub repository available on the following page.

https://github.com/jacobbo/WebEye/tree/master/StreamPlayerControl

Any questions, remarks, and comments are welcome.

Licensing

  1. The FFmpeg facade sources, the same as the FFmpeg framework, are licensed under The LGPL license
  2. The .NET controls' sources and demos' sources are licensed under The Code Project Open License (CPOL).

You can use the control in your commercial product, the only thing is that you should mention that your product uses the FFmpeg library, here are the details.

History

  • March 19th, 2015 - The initial version.
  • August 22nd, 2015 - Added the x64 platform support.
  • October 25th, 2015 - Added asyncronous stream start and stream status events.
  • November 8th, 2015 - Added support for local files playback.
  • November 30th, 2015 - Added stream connection timeout.
  • October 17th, 2017 - Use new FFmpeg decoding API.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Alexander Iacobciuc
Software Developer
Russian Federation Russian Federation
Niko Bellic (Serbian: Niko Belić) is the main protagonist and playable character in the video game Grand Theft Auto IV. He is a 30 year old former soldier who moved to Liberty City to escape his troubled past and pursue the American Dream.

You may also be interested in...

Pro

Comments and Discussions

 
QuestionAudio is not working on stream player Pin
bhavinshah441-Jun-17 0:46
memberbhavinshah441-Jun-17 0:46 
AnswerRe: Audio is not working on stream player Pin
Paolo Ceccanti12-Jul-17 1:29
memberPaolo Ceccanti12-Jul-17 1:29 
GeneralRe: Audio is not working on stream player Pin
Alexander Iacobciuc17-Oct-17 13:45
memberAlexander Iacobciuc17-Oct-17 13:45 
QuestionRTMP stream stops Pin
JATla9-Feb-17 7:04
memberJATla9-Feb-17 7:04 
QuestionBuild Instructions Pin
Member 1288533930-Jan-17 6:49
memberMember 1288533930-Jan-17 6:49 
Questionnot have any song when play stream????!!!!!!! Pin
moha12127-Jan-17 12:26
membermoha12127-Jan-17 12:26 
QuestionNice Work. Capture from DirectShow device Pin
John24242424-Jan-17 5:04
memberJohn24242424-Jan-17 5:04 
QuestionNot able to stream Video through Stream Player Control Pin
Member 1079399010-Jan-17 3:04
memberMember 1079399010-Jan-17 3:04 
QuestionAbout player Pin
Member 1242836922-Dec-16 5:12
memberMember 1242836922-Dec-16 5:12 
QuestionVideo pause Pin
Member 1242836922-Dec-16 5:12
memberMember 1242836922-Dec-16 5:12 
AnswerRe: Video pause Pin
Alexander Iacobciuc22-Dec-16 14:36
memberAlexander Iacobciuc22-Dec-16 14:36 
Questionc# publish (click once?) Pin
Member 1291207821-Dec-16 6:53
memberMember 1291207821-Dec-16 6:53 
QuestionPlay/Pause Pin
Member 973583927-Nov-16 14:51
memberMember 973583927-Nov-16 14:51 
BugMemory leak Pin
Wouterru8-Nov-16 1:25
memberWouterru8-Nov-16 1:25 
Hi, first of all thanks for the great code. But I've noticed streaming an rtsp stream slowly leaks some memory.

When reading a frem using ac_read_frame you allocate an AVPacket by calling the following code:

AVPacket* packet = static_cast<AVPacket*>(av_malloc(sizeof(AVPacket)));

But the av_free(packet) is missing. By adding the call, the memory leak was resolved.
GeneralRe: Memory leak Pin
Alexander Iacobciuc8-Nov-16 13:06
memberAlexander Iacobciuc8-Nov-16 13:06 
GeneralRe: Memory leak Pin
Wouterru10-Nov-16 0:16
memberWouterru10-Nov-16 0:16 
GeneralRe: Memory leak Pin
Alexander Iacobciuc15-Nov-16 14:34
memberAlexander Iacobciuc15-Nov-16 14:34 
GeneralRe: Memory leak Pin
Member 946553622-Dec-16 4:30
memberMember 946553622-Dec-16 4:30 
GeneralRe: Memory leak Pin
Member 945191015-Mar-17 9:36
memberMember 945191015-Mar-17 9:36 
GeneralRe: Memory leak Pin
THOAI NGUYEN ANH26-Jul-17 21:45
professionalTHOAI NGUYEN ANH26-Jul-17 21:45 
GeneralRe: Memory leak Pin
Daniel Smolka8-Sep-17 0:34
memberDaniel Smolka8-Sep-17 0:34 
GeneralRe: Memory leak Pin
E30Tomas21-Sep-17 4:07
memberE30Tomas21-Sep-17 4:07 
GeneralRe: Memory leak Pin
Daniel Smolka25-Sep-17 9:00
memberDaniel Smolka25-Sep-17 9:00 
GeneralRe: Memory leak Pin
THOAI NGUYEN ANH4-Oct-17 0:03
professionalTHOAI NGUYEN ANH4-Oct-17 0:03 
GeneralRe: Memory leak Pin
Alexander Iacobciuc17-Oct-17 10:54
memberAlexander Iacobciuc17-Oct-17 10:54 
Questionworks slowly. time lags. help please. for example: in VLC time 14:42 and in your programm 14:38. :(( Pin
Вадим Алексеев27-Oct-16 3:55
memberВадим Алексеев27-Oct-16 3:55 
AnswerRe: works slowly. time lags. help please. for example: in VLC time 14:42 and in your programm 14:38. :(( Pin
Alexander Iacobciuc27-Oct-16 13:20
memberAlexander Iacobciuc27-Oct-16 13:20 
GeneralRe: works slowly. time lags. help please. for example: in VLC time 14:42 and in your programm 14:38. :(( Pin
Вадим Алексеев9-Nov-16 22:57
memberВадим Алексеев9-Nov-16 22:57 
GeneralRe: works slowly. time lags. help please. for example: in VLC time 14:42 and in your programm 14:38. :(( Pin
patelpradip200331-Mar-17 0:52
memberpatelpradip200331-Mar-17 0:52 
GeneralRe: works slowly. time lags. help please. for example: in VLC time 14:42 and in your programm 14:38. :(( Pin
patelpradip200331-Mar-17 0:53
memberpatelpradip200331-Mar-17 0:53 
GeneralRe: works slowly. time lags. help please. for example: in VLC time 14:42 and in your programm 14:38. :(( Pin
John2424247-Apr-17 2:21
memberJohn2424247-Apr-17 2:21 
Bugworks slowly. time lags. Pin
Вадим Алексеев27-Oct-16 3:56
memberВадим Алексеев27-Oct-16 3:56 
GeneralRe: works slowly. time lags. Pin
gopiitit19-Jan-17 20:21
membergopiitit19-Jan-17 20:21 
QuestionProlem with properties AllowsTransparency=true Pin
quainhan_pro16-Oct-16 18:01
memberquainhan_pro16-Oct-16 18:01 
QuestionPerformance Pin
Member 127267831-Oct-16 22:28
memberMember 127267831-Oct-16 22:28 
QuestionAbout Video StreamPlayer Pin
Member 1272678329-Sep-16 2:01
memberMember 1272678329-Sep-16 2:01 
QuestionThnak you Pin
Member 367756822-Sep-16 21:11
memberMember 367756822-Sep-16 21:11 
Questionerror LNK1104: cannot open file 'libavformat.a' Pin
Member 1245551015-Sep-16 5:26
memberMember 1245551015-Sep-16 5:26 
AnswerRe: error LNK1104: cannot open file 'libavformat.a' Pin
Alexander Iacobciuc15-Sep-16 11:47
memberAlexander Iacobciuc15-Sep-16 11:47 
AnswerRe: error LNK1104: cannot open file 'libavformat.a' Pin
Member 946553620-Dec-16 9:21
memberMember 946553620-Dec-16 9:21 
GeneralRe: error LNK1104: cannot open file 'libavformat.a' Pin
Alexander Iacobciuc22-Dec-16 14:07
memberAlexander Iacobciuc22-Dec-16 14:07 
QuestionGetting AccessViolation if stopping play of local file Pin
AndyP19658-Sep-16 6:53
memberAndyP19658-Sep-16 6:53 
AnswerRe: Getting AccessViolation if stopping play of local file Pin
Alexander Iacobciuc12-Sep-16 15:03
memberAlexander Iacobciuc12-Sep-16 15:03 
PraiseJust what I was looking for, useful .NET rtsp cade Pin
Tranzmetro4-Sep-16 10:44
memberTranzmetro4-Sep-16 10:44 
Questionadding control Pin
Member 1212997720-Jun-16 6:32
memberMember 1212997720-Jun-16 6:32 
AnswerRe: adding control Pin
Alexander Iacobciuc20-Jun-16 13:37
memberAlexander Iacobciuc20-Jun-16 13:37 
GeneralRe: adding control Pin
Member 1212997721-Jun-16 3:07
memberMember 1212997721-Jun-16 3:07 
Questionlocal video file Pin
Manirishi12-Jun-16 8:25
memberManirishi12-Jun-16 8:25 
AnswerRe: local video file Pin
Alexander Iacobciuc13-Jun-16 13:25
memberAlexander Iacobciuc13-Jun-16 13:25 
GeneralRe: local video file Pin
Manirishi1-Aug-16 7:10
memberManirishi1-Aug-16 7:10 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web03 | 2.8.171207.1 | Last Updated 17 Oct 2017
Article Copyright 2015 by Alexander Iacobciuc
Everything else Copyright © CodeProject, 1999-2017
Layout: fixed | fluid