Click here to Skip to main content
13,252,065 members (57,071 online)
Click here to Skip to main content
Add your own
alternative version

Stats

8.7K views
26 bookmarked
Posted 6 Aug 2017

cam2web - Streaming camera to web as MJPEG stream

, 29 Aug 2017
Rate this:
Please Sign up or sign in to vote.
The article describes cam2web project - an open source application aimed for streaming cameras as MJPEG streams.
cam2web 

Introduction

Some time ago I was working on a robotics related project based on Raspberry Pi board. One of the set requirements was to have an ability to view Pi's camera remotely, from either a web browser or from whatever software which fits the task best. Doing a quick search leads to a number of tutorials on the topic, which are all based on the MJPG-Streamer application. However, using that was not really an option in the project I had in mind. First, it was due to some limitations of that software, which would not allow achieving my goal without substantial rework. Second, just viewing a camera remotely was only the first step in the project. And so, a decision was made to create some code of my own, which would fit the needs of the projects I had and serve the base for some further ideas in mind. As the result, a quick list of requirements was set:

  • To stream camera as MJPEG stream over HTTP, so it can be consumed by any application supporting it;
  • Provide JPEG snapshots as well, so applications, which are not MJPEG-friendly, could grab a single image;
  • Provide a way to control options like brightness, contrast, saturation, sharpness, etc., if a camera supports those;
  • Come with simple built-in web UI, so camera could be viewed directly from a web browser, as well as its options set;
  • Support authentication, so viewing/configuring a camera could be restricted

While the work was progressing smoothly, it was decided to grow the project a bit bigger and make it available not only for Raspberry Pi, but for generic Linux and Windows as well. As the result, a new open source project was set on GitHub - cam2web, streaming camera to web.

The application overview

Before diving into the technical details of the implementation and the code, let's have a look at the end result first. The way cam2web looks on different platforms is quite different. The Windows version comes with graphical user interface, which allows users to start camera streaming and configure different settings. While the Linux and Pi versions are provided as command line tools. The web UI on the opposite - it looks almost identical for all platforms. Only the cameras' options look different in it, since different platforms provide different APIs to access cameras. However, we'll get there.

Windows version

The main form of the Windows version is very simple and self-explanatory - it provides the list of detected cameras (devices supporting DirectShow API) and their supported resolutions. Once the required camera is chosen and the "Start streaming" button is clicked, the camera goes live to web.

Note: The resolution box shows default average frame rate for all resolutions supported by a camera. However, some cameras support a range of frames rates - minimum/maximum rate. For such cameras it is possible to override default frame rate and set the one needed. But, don't expect all frame rate values to work from the provided range. Due to limitations of DirectShow API and badly written drivers of some cameras, many frame rate values may no work.

Windows - Main form

If you have a preferred application of your choice for viewing cameras, you may access your camera's MJPEG stream with an URL like http://ip:port/camera/mjpeg. In case single JPEG snapshots are required, use URL like http://ip:port/camera/jpeg. Simple, isn't it!

However, one of the set requirements says we should not really need any special software to view cameras; just a web browser. To try it - just click the "Streaming on ..."; link on the main form.

Web view

Many cameras provide different configuration options to set brightness, contrast, saturation, etc., which are available when clicking "Settings" button in the web UI. Those are persisted by the application, so the next time it starts, all the settings should be set to whatever configuration user did.

The Windows version of cam2web also provides user interface to change various application’s options, like default port to listen for incoming connections, quality level of provided JPEG images, frame rate of MJPEG stream (in case it must be limited to certain value), etc.

Windows - Settings form

Finally, we get to the last requirement - user authentication. By default, when the application starts first time, it allows anyone to do anything - view camera and change its settings. However, this does not look like desired in most cases. And so, the "Access rights" form allows to specify who can do what and populate the list of users allowed to access the camera

Windows - Access Rights form

One thing to note about changing configuration – camera streaming must be restarted for the new settings to take an effect. Just "Stop streaming" / "Start streaming" – should not be a big deal.

There are few more options available for the Windows version, but those are command line options this time:

  • /start - Automatically start camera streaming on application start.
  • /minimize - Minimize application’s window on its start.
  • /fcfg:file_name - Name of configuration file to store application's settings. By default, the application stores all settings (including last run camera/resolution) in app.cfg stored in cam2web folder within user's home directory. However, the name of configuration file can be set different, so several instances of the application could run, having different settings and streaming different cameras.

Linux and Raspberry Pi versions

Unlike Windows version, Linux versions don't provide any graphical user interface and so all configuration is done by using command line options. Most of them are very similar to those found in GUI of Windows version and are quite self-explanatory. To see the list of available options, just run the application with -? option.

When the application starts on Linux/Pi, it starts camera streaming automatically (provided no errors happened). And so, it can be accessed same way from either a web browser or whatever application in preference.

Unlike Windows version, the Linux/Pi version does not provide means for editing users' list who can access camera. Instead, the Apache htdigest tool is used to manage users’ file, which name can then be specified as one of the command line options. This creates a limitation though – only one user with administrator role can be created, which is admin. All other names get user role.

Getting the code and building it

The cam2web code is published on GitHub and so its latest version can be either cloned from the git repository or downloaded as ZIP archive. Alternatively, a tagged released version can be downloaded as archive as well.

Before building cam2web itself, it is required to build web2h tool provided with it, which translates some of the common web files (HTML, CSS, JS, JPEG and PNG) into header files. Those are then compiled and linked into the cam2web executable, so it could provide default web interface without relying on external files.

If building in debug configuration however, the web2h is not required – all web content is served from the files located in ./web folder.

Building on Windows

Microsoft Visual Studio solution files are provided for both web2h and cam2web applications (Express 2013 can be used, for example). First build src/tools/web2h/make/msvc/web2h.sln and then src/apps/win/cam2web.sln. On success, it will produce build/msvc/[configuration]/bin folder, which contains applications’ executables.

Building on Linux and Raspberry Pi

Makefiles for GNU make are provided for both web2h and cam2web. Running bellow commands from the project’s root folder, will produce the required executables in build/gcc/release/bin folder.

pushd .
cd src/tools/web2h/make/gcc/
make
popd

pushd .
cd src/apps/linux/
# or cd src/apps/pi/
make
popd

Note: libjpeg development library must be installed for cam2web build to succeed (which may not be installed by default):

sudo apt-get install libjpeg-dev

Accessing camera from other applications (WEB API)

As it was already mentioned, the streamed camera can be accessed not only from a web browser, but also from any other application supporting MJPEG streams (like VLC media player, for example, or different applications for IP cameras monitoring). The URL format to access MJPEG stream is:
http://ip:port/camera/mjpeg

In the case an individual image is required, the next URL provides the latest camera snapshot:
http://ip:port/camera/jpeg

Camera information

To get some camera information, like device name, width, height, etc., an HTTP GET request should be sent the next URL:
http://ip:port/camera/info

It provides reply in JSON format, which may look like the one below:

{
  "status":"OK",
  "config":
  {
    "device":"RaspberryPi Camera",
    "title":"My home camera",
    "width":"640"
    "height":"480",
  }
}

Changing camera's settings

Cameras settings/configuration is available using the next URL:
http://ip:port/camera/config

To get current camera' settings, an HTTP GET request is sent to the above URL, which provides all values as JSON reply:

{
  "status":"OK",
  "config":
  {
    "awb":"Auto",
    "brightness":"50",
    "contrast":"15",
    "effect":"None",
    "expmeteringmode":"Average",
    "expmode":"Auto",
    "hflip":"1",
    "saturation":"25",
    "sharpness":"100",
    "vflip":"1",
    "videostabilisation":"0"
  }
}

In the case if only some values are required, their names (separated with coma) can be passed as vars variable. For example:
http://ip:port/camera/config?vars=brightness,contrast

For setting camera's properties, the same URL is used, but HTTP POST request must be used. The posted data must contain collection of variables to set encoded in JSON format. For example, posting below JSON will set both camera's brightness and contrast options:

{
  "brightness":"50",
  "contrast":"15"
}

On success, the reply JSON will have status variable set to "OK". Or it will contain failure reason otherwise.

Getting description of camera properties

Starting from version 1.1.0, the cam2web application provides description of all properties camera provides. This allows, for example, to have single WebUI code, which queries the list of available properties first and then does unified rendering. The properties description can be optained using the below URL:
http://ip:port/camera/properties

The JSON response provides name of all available properties, their default value, type, display name and order. For integer type properties, it also includes allowed minimum and maximum values. And for selection type properties, it provides all possible choices for the property.

{
  "status":"OK",
  "config":
  {
    "hflip":
    {
      "def":0,
      "type":"bool",
      "order":8,
      "name":"Horizontal Flip"
    },
    "brightness":
    {
      "min":0,
      "max":100,
      "def":50,
      "type":"int",
      "order":0,
      "name":"Brightness"
    },
    "awb":
    {
      "def":"Auto",
      "type":"select",
      "order":4,
      "name":"White Balance",
      "choices":
      [
        ["Off","Off"],
        ["Auto","Auto"],
        ["Sunlight","Sunlight"],
        ...
      ]
    },
    ...
  }
}

Getting version information

To get information about version of the cam2web application streaming the camera, the next URL is used:
http://ip:port/version
, which provides information in the format below:

{
  "status":"OK",
  "config":
  {
    "platform":"RaspberryPi",
    "product":"cam2web",
    "version":"1.0.0"
  }
}

Access rights

Accessing JPEG, MJPEG and camera information URLs is available to those, who can view the camera. Access to camera configuration URL is available to those, who can configure it. The version URL is accessible to anyone.

Customizing Web UI

All release builds of cam2web come with embedded web resources, so the application can provide default web UI without relying on any extra files. However, it is possible to override default user interface even without diving into build tools.

Using configuration UI on Windows or command line options on Linux, it is possible to specify folder name to use for serving custom web content. When the option is not set, embedded UI is provided. And when it is set, whatever is found in the specified folder will be provided instead.

Getting default Web files

The easiest way to start customizing Web UI is to get all default content first as a starting point. All the required files can be found in the two folders: src/web and externals/jquery. Those files must be put into a single folder, so the basic directory tree should look like this:

index.html
cameraproperties.html
styles.css
cam2web.png
cam2web_white.png
camera.js
cameraproperties.js
cameraproperty.js
jquery.js
jquery.mobile.css
jquery.mobile.js

Once all files are in place, the application can be configured to serve web content from the folder containing them. To make sure it is all working, alter some styles in the styles.css or change some HTML in the index.html. If it does, you are ready to go further customizing the web UI.

Source code overview

Now it is time to go through the code a bit. It will be a very high-level overview without going too deep into implementation details. So more of an architecture review.

Most of the code is written in C++ with some plain C here and there. In some rare places, few C++ 11 features are used – getting compiler supporting it should not be an issue these days.

The cam2web project relies on few dependencies. First, mongoose embedded web server is used as a lightweight HTTP server, which is easy to integrate. Second, libjpeg-turbo is used to do JPEG compression (or libjpeg can be used if performance is not a concern). As for the rest – all the code is there. Even a very simple JSON parser was done instead of pulling extra dependencies.

The graphical user interface of Windows version is done using plain old Win32 API. No MFC, no Qt, just doing KISS. In fact, the Windows version prefers static linking for everything possible, which results in a single executable not requiring even MSVC redistributables. However, because of the some UI controls being used, the application requires Windows Vista as a minimum supported OS.

As for web UI, a bit jQuery is in use. But, since it is all customizable as it was already mentioned, it all can be ruined and replaced with whatever web technology in preference.

Cameras

Camera's API is one of the few things, which is different between the number of supported platforms.

  • DirectShow API is used to access cameras on Windows (USB/integrated cameras, capture boards, etc.). It is COM based API, which is well documented on MSDN and has plenty of samples here on Code Project.
  • Video for Linux API (V4L or V4L2 for version 2) is used to access cameras on Linux (again USB/integrated cameras or anything else supporting this interface).
  • Multi-Media Abstraction Layer (MMAL) is the API to access camera module on Raspberry PI.

All these APIs are very different, same as the code used to access cameras on different platforms. Since most of the rest code is platform agnostic, the IVideoSource abstract class (an interface) is defined to set the common way of talking to different cameras and then 3 implementations of it are provided for different platforms.

// Interface for video source events' listener
class IVideoSourceListener
{
public:
    virtual ~IVideoSourceListener( ) { }

    // New video frame notification
    virtual void OnNewImage( const std::shared_ptr<const XImage>& image ) = 0;

    // Video source error notification
    virtual void OnError( const std::string& errorMessage, bool fatal ) = 0;
};    
    
// Interface for video sources continuously providing frames to display/process
class IVideoSource
{
public:
    virtual ~IVideoSource( ) { }

    // Start video source so it initializes and begins providing video frames
    virtual bool Start( ) = 0;
    // Signal video source to stop, so it could finalize and clean-up
    virtual void SignalToStop( ) = 0;
    // Wait till video source (its thread) stops
    virtual void WaitForStop( ) = 0;
    // Check if video source is still running
    virtual bool IsRunning( ) = 0;

    // Get number of frames received since the start of the video source
    virtual uint32_t FramesReceived( ) = 0;

    // Set video source listener returning the old one
    virtual IVideoSourceListener* SetListener( IVideoSourceListener* listener ) = 0;
};    

Although we are not going into details of implementation (source code is available for that), it is still worth mentioning about image data provided by the cameras on each platform. The implementation on Windows provides images in RGB24 format, which is easy to process if we wish to, and which is well accepted by libjpeg(-turbo).

The MMAL implementation can also provide RGB24 data, however this might not be the preference on Raspberry Pi. Since Pi’s CPU is not as powerful as desktops may have, it’s better to avoid software JPEG compression there. Luckily MMAL API allows requesting already compressed images, which is hardware accelerated. And so, the MMAL implementation provides both options, defaulting to the compressed one.

Finally, the Video for Linux implementation. As V4L documentation suggests, the API does support RGB24 format. However, I failed to find a camera which would agree supporting that. Instead, all the cameras I managed to test did support YUYV format. So, it is required to do a bit of decoding if RGB data are needed. However, same as with MMAL, V4L also allows requesting already compressed JPEGs, which again saves on CPU usage. As the result, the V4L implementation mimics the MMAL one – provides an option to request either uncompressed data or compressed, defaulting to the latter.

IVideoSource

Camera configuration

The above defined IVideoSource interface only provides APIs to start/stop cameras and get images from them. Obviously, it is not enough and so each particular implementation provides additional methods to configure cameras and set/get different options like brightness, contrast, etc. The initial configuration is done by the main application depending on what user chooses. However, the rest of configuration, which needs to be persisted and changed from web UI (REST API), would be better to abstract as well.

And so, a simple IObjectConfigurator interface is defined, which allows configuring different objects using string key/value pairs (remember, KISS).

// Interface to configure objects with string key/value pairs
class IObjectConfigurator
{
public:
    virtual ~IObjectConfigurator( ) { }

    // Set/Get specified property
    virtual XError SetProperty( const std::string& propertyName,
                                const std::string& value ) = 0;
    virtual XError GetProperty( const std::string& propertyName,
                                std::string& value ) const = 0;

    // Get values of all properties provided by an object
    virtual std::map<std::string, std::string> GetAllProperties( ) const = 0;
};

The above interface could be implemented by the 3 classes implementing IVideoSource. However, the camera classes were left to provide specific methods for configuration means (in case they are reused in other projects where configuration abstraction is not a concern). Instead, another 3 classes where implemented to provide abstract configuration for the camera classes they bound to.

IObjectConfigurator

Embedded web server

As it was already mentioned, cam2web uses mongoose embedded web server to handle HTTP requests. It is small and easy to integrate. Comes in just two files - source and a header file. And builds with no issues on a variety of platforms. Yes, its licence is quite restrictive. Since it is released under GPL, it does require providing combined work under the same licence. Well, for cam2web it is not an issue.

The mongoose web server is implemented in plain C. It does make it small, but less friendly for C++ users. And so, a small wrapper was made around it, to give it a bit more object-oriented design and even hide all the mongoose internals from the end user.

// All implementation details will be hidden, so user does
// not even know we are using mongoose
namespace Private
{
    class XWebServerData;
}

// A class to provide simple embedded web server API
class XWebServer : private Uncopyable
{
public:
    XWebServer( const std::string& documentRoot = "", uint16_t port = 8000 );
    ~XWebServer( );

    // Get/Set document root
    std::string DocumentRoot( ) const;
    XWebServer& SetDocumentRoot( const std::string& documentRoot );

    // Get/Set authentication domain
    std::string AuthDomain( ) const;
    XWebServer& SetAuthDomain( const std::string& authDomain );

    // Get/Set port
    uint16_t Port( ) const;
    XWebServer& SetPort( uint16_t port );

    // Add/Remove web handler
    XWebServer& AddHandler( const std::shared_ptr<IWebRequestHandler>& handler,
                            UserGroup allowedUserGroup = UserGroup::Anyone );
    void RemoveHandler( const std::shared_ptr<IWebRequestHandler>& handler );

    // Remove all handlers
    void ClearHandlers( );
    
    // Start/Stop the Web server
    bool Start( );
    void Stop( );

    // ... some more methods ...
    
private:
    Private::XWebServerData* mData;
};

The above XWebServer class provides basic configuration of a web server and managing its life time. It allows serving either static content providing files from the specified folder or handle requests dynamically by invoking provided web request handlers inheriting IWebRequestHandler abstract class. Each handler needs to provide implementation of HandleHttpRequest() methods, which purpose is to provide web response (by using IWebResponse interface) based on the received web request (described using IWebRequest interface).

// Interface for web request details
class IWebRequest
{
public:
    virtual ~IWebRequest( ) { }

    virtual std::string Uri( )    const = 0;
    virtual std::string Method( ) const = 0;
    virtual std::string Proto( )  const = 0;
    virtual std::string Query( )  const = 0;
    virtual std::string Body( )   const = 0;

    virtual std::string GetVariable( const std::string& name ) const = 0;

    virtual std::map<std::string, std::string> Headers( ) const = 0;
};

// Interface used to provide web response
class IWebResponse
{
public:
    virtual ~IWebResponse( ) { }

    // Length of data, which is still enqueued for sending
    virtual size_t ToSendDataLength( ) const = 0;

    virtual void Send( const uint8_t* buffer, size_t length ) = 0;
    virtual void Printf( const char* fmt, ... ) = 0;

    virtual void SendChunk( const uint8_t* buffer, size_t length ) = 0;
    virtual void PrintfChunk( const char* fmt, ... ) = 0;

    virtual void SendError( int errorCode, const char* reason = nullptr ) = 0;

    virtual void CloseConnection( ) = 0;

    // Generate timer event for the connection associated with the response
    // after the specified number of milliseconds
    virtual void SetTimer( uint32_t msec ) = 0;
};

// Base class for web request handlers
class IWebRequestHandler
{
protected:
    IWebRequestHandler( const std::string& uri, bool canHandleSubContent );

public:
    virtual ~IWebRequestHandler( ) { }

    const std::string& Uri( ) const { return mUri;  }
    bool CanHandleSubContent( ) const { return mCanHandleSubContent; }

    // Handle the specified request
    virtual void HandleHttpRequest( const IWebRequest& request, IWebResponse& response ) = 0;

    // Handle timer event
    virtual void HandleTimer( IWebResponse& ) { };

private:
    std::string mUri;
    bool        mCanHandleSubContent;
};

As the result, the public API of the XWebServer does not have any dependency on the chosen implementation of a particular embedded web server. It can be mongoose or it can be anything else - all implementation details are hidden from user.

XWebServer

When implementing new requests handlers, it is required to specify two of their properties. First is the URI they are going to serve. For example, one web request handler can serve "/camera/config" URI, while another - "/camera/info" URI. When HTTP request comes into the web server, it checks the list of registered handlers and invokes the one matching the requested URI. If no handler was found, the server will either try finding static content from the specified document root, or it will say 404 – page not found.

The second property of a request handler is whether it can handle sub content or not. If not, the handler is used only to serve the URI it was assigned, "/camera/config" for example. But if it can serve sub content, then it will be used for anything which starts from the given URI, like "/camera/config/brightness". So, it is like serving a folder. Obviously, it is up to the handler's implementation then to check the requested URI and handle it appropriately.

Handling embedded content

So we talked about the idea of abstract web request handlers. And now it is time to discuss some particular implementations serving specific roles. To get started, let’s have a look at the way embedded web resources are provided.

As it was already mentioned, the API of XWebServer allows specifying document root folder, which is used for serving static content. This is fine to use when user wants to provide his own custom web UI. However, as it was stated from the very beginning, all versions of cam2web don't rely on externals and provide default web UI from embedded resources. This is achieved using XEmbeddedContentHandler class:

// Structure defining embedded content
typedef struct
{
    uint32_t       Length;
    const char*    Type;
    const uint8_t* Body;
}
XEmbeddedContent;

// Web request handler serving embedded content
class XEmbeddedContentHandler : public IWebRequestHandler
{
public:
    XEmbeddedContentHandler( const std::string& uri, const XEmbeddedContent* content ) :
        IWebRequestHandler( uri, false ), mContent( content )
    { }

    // Handle request providing given embedded content
    void HandleHttpRequest( const IWebRequest& /* request */, IWebResponse& response )
    {
        response.Printf( "HTTP/1.1 200 OK\r\n"
                         "Content-Type: %s\r\n"
                         "Content-Length: %u\r\n"
                         "\r\n", mContent->Type, mContent->Length );
        response.Send( mContent->Body, mContent->Length );
    }

private:
    const XEmbeddedContent* mContent;
};

All this web handler does is simply sending out the content of the specified embedded resource along with some HTML headers. It does not check URI, since it is already done by the web server when looking for the appropriate request handler. It could check HTTP method though, to make sure it is a GET request, but it was kept simple here.

And here is a sample of defining some embedded content and registering it with the web server:

// Define simple test page
XEmbeddedContent testHtml =
{
    35, "text/html",
    (const uint8_t*) "<html><body>Test page</body></html>"
};

// ...
XWebServer server;

server.AddHandler( make_shared<XEmbeddedContentHandler>( "test.html", &testHtml ) );

That is it - our web server is ready to provide some HTML content from embedded resource. Same is done with anything else – styles, Java Script, images, etc. Of course, it is just a sample (which still does work). In reality, I would not spend time myself hardcoding all the web content required for cam2web. That is why the web2h tools was provided (mentioned before), which generates header files containing XEmbeddedContent definitions from provided web files.

Handling camera configuration

Another web request handler to mention is XObjectConfigurationRequestHandler – a web request handler, which allows getting/setting object's configuration (properties) through the IObjectConfigurator interface mentioned earlier. This way the request handler does not even know what it configures – it is all hidden from it.

// Request handler used to configure objects (camera objects)
class XObjectConfigurationRequestHandler : public IWebRequestHandler
{
public:
    XObjectConfigurationRequestHandler( const std::string& uri, 
                        const std::shared_ptr<IObjectConfigurator>& objectToConfig );

    void HandleHttpRequest( const IWebRequest& request, IWebResponse& response );

private:
    void HandleGet( const std::string& varsToGet, IWebResponse& response );
    void HandlePost( const std::string& body, IWebResponse& response );

private:
    std::shared_ptr<IObjectConfigurator> ObjectToConfig;
};

Implementation of this request handler checks the type of HTTP method used with it. If it is a GET requests, it retrieves object’s configuration and provides it as JSON response. If it is a POST request, it parses the posted JSON and tries setting any properties found there. For any other request type is simple says "Sorry, method is not allowed".

Handling JPEG and MJPEG requests

Finally we get to the handlers, which provide JPEG snapshots and MJEPS streams. Those are managed by the XVideoSourceToWeb class, which can work with any arbitrary video source implementing IVideoSource interface and then provide both JPEG and MJPEG handlers through the IWebRequestHandler interface.

namespace Private
{
    class XVideoSourceToWebData;
}

// Class providing a bridge between IVideoSource and IWebRequestHandler
class XVideoSourceToWeb : private Uncopyable
{
public:
    XVideoSourceToWeb( uint16_t jpegQuality = 85 );
    ~XVideoSourceToWeb( );
    
    // Get video source listener, which could be fed to some video source
    IVideoSourceListener* VideoSourceListener( ) const;

    // Create web request handler to provide camera images as JPEGs
    std::shared_ptr<IWebRequestHandler> CreateJpegHandler( const std::string& uri ) const;

    // Create web request handler to provide camera images as MJPEG stream
    std::shared_ptr<IWebRequestHandler> CreateMjpegHandler( const std::string& uri,
                                                            uint32_t frameRate ) const;

    // Get/Set JPEG quality (valid only if camera provides uncompressed images)
    uint16_t JpegQuality( ) const;
    void SetJpegQuality( uint16_t quality );

private:
    Private::XVideoSourceToWebData* mData;
};

The XVideoSourceToWeb::VideoSourceListener() method provides a listener, which is fed to whatever video source is in use. Remember the IVideoSource::SetListener() method? This allows the XVideoSourceToWeb to get video frames from a video source. Then the XVideoSourceToWeb::CreateJpegHandler() provides a web request handler, which serves JPEG snapshots – if a video source provides already compressed image data, it simply outputs that into web response; otherwise it does software JPEG compression first and then does the web response.

The XVideoSourceToWeb::CreateMjpegHandler() method does similar, but provides web request handler to serve MJPEG streams. The first image is provided in a similar way to JPEG request handler (except that HTTP headers are a bit different), and then the other image are provided through IWebRequestHandler::HandleTimer() call back, which is requested through IWebResponse::SetTimer().

For example, here is how it may look for Raspberry Pi camera:

XWebServer               server;
XVideoSourceToWeb        video2web;
shared_ptr<XRaspiCamera> xcamera = XRaspiCamera::Create( );

// subscribe XVideoSourceToWeb object for video frame events
xcamera->SetListener( video2web.VideoSourceListener( ) );

// add JPEG and MJPEG handlers to the web server
server.AddHandler( video2web.CreateJpegHandler( "/camera/jpeg" ) ).
       AddHandler( video2web.CreateMjpegHandler( "/camera/mjpeg", 30 ) );

With the above described architecture, it become extremally easy to add support for new type of cameras or whatever video sources. All we need to do, is to provide an implementation of IVideoSource interface for a particular camera and then the XVideoSourceToWeb will do the rest of the job (provided we can get RGB24 or compressed data from the camera).

XVideoSourceToWeb

Access control

So far we've seen how to configure XWebServer to serve either static content from the specified folder or dynamic content through different implementations of the IWebRequestHandler interface. However, nothing was said about controlling who can access what. We don't want to leave our camera exposed to public, do we?

The access control is implemented through user groups. When adding a web request handler, it is required to specify which group of users can access it. If it is UserGroup::Anyone, then the handler can be accessed by everyone. However, if it is UserGroup::User or UserGroup::Admin, then user must be authenticated and belong to the required group. This is all checked by the XWebServer. When it finds a request handler for certain URI, it first checks if the current user is allowed to access it. If yes, then the handler is invoked and does its job. If not, the web server sends out request for digest authentication.

Here is a quick sample, which shows how to allow everyone to access some embedded web content, users to view camera and administrators to change its settings:

webServer.AddHandler( make_shared<XEmbeddedContentHandler>( "index.html", &web_index_html ),
                      UserGroup::Anyone ).
          AddHandler( make_shared<XEmbeddedContentHandler>( "styles.css", &web_styles_css ),
                      UserGroup::Anyone ).
          AddHandler( video2web.CreateJpegHandler( "/camera/jpeg" ),
                      UserGroup::User ).
          AddHandler( video2web.CreateMjpegHandler( "/camera/mjpeg", 30 ),
                      UserGroup::User ).
          AddHandler( make_shared<XObjectConfigurationRequestHandler>( "/camera/config",
                      cameraConfig ), UserGroup::Admin );

Of course, to get it working it is required to add some users. The XWebServer API provides methods for adding either individual users or load the from a file having htdigest format.

Web UI

The web part of cam2web is kept quite simple - very basic HTML page and some Java Script using jQuery for dynamic loading of camera's configuration page and then sending GET/POST requests for obtaining/setting camera's properties. jQuery Mobile is also in use to get some user controls, which look consistent across different web browsers.

By default, the web UI tries to display MJPEG stream directly through image element. However, not all web browsers support this. Firefox and Chrome do, IE – does not. So, if MJPEG stream fails to start, the web UI switches into JPEG mode, when image element continuously refreshes JPEG snapshots. The details of this can be found in the camera.js.

Conclusion

Well, starting as a small project aimed for some hobby robotics with Raspberry Pi, the project grew into something bigger, which may serve a good purpose and applied in different applications. Someone, for example, may decide to build his own video surveillance system at home using conventional USB cameras or Raspberries scattered around a house. Or just monitoring a single camera from work, to make sure your house is still there.

Parts of the project can be easily reused in many different applications. For example, the code to access cameras can be easily integrated into any application dealing with video processing. Similar about the mongoose web server wrapper – the C++ API provides an easier way to handle HTTP requests from within an application requiring support of embedded web server.

As it was already mentioned, the project is published on GitHub. So its code is available for study or reuse (provided the licence is respected). Any code comments, bug reports, fixes, etc. are all welcome.

History

30.08.2017 - Version 1.1.0

  • The REST API is updated to provide description of camera's properties (GET request to /camera/properties). This description allows now to have common WebUI, which renders camera's properties for all supported APIs (DirectShow, V4L, MMAL).
  • New configuration option is added to specify name of a streamed camera, which is displayed in WebUI.
  • Windows: Added configuration option to override default camera's frame rate. It allows to choose from a range of supported frame rates. Note: not all frame rate values will work. This is caused by limitations of DirectShow API and simply some bad camera drivers, which don't care about what they report.
  • Windows: Main application's window now shows actual camera's frame rate.
  • Windows: Optimized BGR to RGB conversion using SSSE3 instructions if CPU supports those.
  • Windows: Main window's icon and system tray's icon (if minimized) show an indication of web activity - when the streamed camera is accessed.
  • Windows: Added an option to specify folder for custom WebUI using folder selection form.
  • Windows: Added configuration option to specify color of window/tray icon. This makes it easier to distinguish multiple instances of the application streaming different cameras.

30.07.2017 - The first public release of the project

License

This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)

Share

About the Author

Andrew Kirillov
Software Developer IBM
United Kingdom United Kingdom
Started software development at about 15 years old and it seems like now it lasts most part of my life. Fortunately did not spend too much time with Z80 and BK0010 and switched to 8086 and further. Similar with programming languages – luckily managed to get away from BASIC and Pascal to things like Assembler, C, C++ and then C#. Apart from daily programming for food, do it also for hobby, where mostly enjoy areas like Computer Vision, Robotics and AI. This led to some open source stuff like AForge.NET and not so open Computer Vision Sandbox.

Going out of computers I am just a man loving his family, enjoying travelling, a bit of books, a bit of movies and a mixture of everything else. Always wanted to learn playing guitar, but it seems like 6 strings are much harder than few dozens of keyboard’s keys. Will keep progressing ...

You may also be interested in...

Comments and Discussions

 
PraiseMy vote of 5 Pin
César de Souza15-Sep-17 11:19
professionalCésar de Souza15-Sep-17 11:19 
GeneralRe: My vote of 5 Pin
Andrew Kirillov15-Sep-17 12:06
memberAndrew Kirillov15-Sep-17 12:06 
QuestionHow to autostart? Pin
eslipak7-Sep-17 7:06
professionaleslipak7-Sep-17 7:06 
AnswerRe: How to autostart? Pin
Andrew Kirillov7-Sep-17 13:14
memberAndrew Kirillov7-Sep-17 13:14 
GeneralRe: How to autostart? Pin
eslipak8-Sep-17 15:37
professionaleslipak8-Sep-17 15:37 
GeneralRe: How to autostart? Pin
eslipak4-Oct-17 15:55
professionaleslipak4-Oct-17 15:55 
GeneralRe: How to autostart? Pin
Andrew Kirillov5-Oct-17 1:13
memberAndrew Kirillov5-Oct-17 1:13 
GeneralMy vote of 5 Pin
eslipak7-Sep-17 6:52
professionaleslipak7-Sep-17 6:52 
GeneralRe: My vote of 5 Pin
Andrew Kirillov7-Sep-17 13:01
memberAndrew Kirillov7-Sep-17 13:01 
QuestionVery Nice Job - You Got My 5 Pin
Bill SerGio Jr.17-Aug-17 6:28
professionalBill SerGio Jr.17-Aug-17 6:28 
AnswerRe: Very Nice Job - You Got My 5 Pin
Andrew Kirillov17-Aug-17 23:00
memberAndrew Kirillov17-Aug-17 23:00 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web02 | 2.8.171114.1 | Last Updated 30 Aug 2017
Article Copyright 2017 by Andrew Kirillov
Everything else Copyright © CodeProject, 1999-2017
Layout: fixed | fluid