Click here to Skip to main content
Click here to Skip to main content
Go to top

Video Manipulation with the Canvas Element

, 17 Aug 2011
Rate this:
Please Sign up or sign in to vote.
This post will explain how to combine the video and canvas elements in order to create transparency inside a video during runtime using JavaScript.

Video Manipulation with the Canvas Element

Video Manipulation with the Canvas Element

One of the designers in my company asked me to help him with the creation of a transparent video which runs on top of an image in HTML. So I did some digging and found a very good resource to do that - Manipulating video using canvas. This post will explain how to combine the video and canvas elements in order to create transparency inside a video during runtime using JavaScript. I’ve based the code on the previous link so they deserve all the credit. You can download the demo code from here.

The Demands

The designer has created a simple movie file with a red square that is floating on top of a black background. He wanted that during runtime, the black background will be transparent so the image that he wants to show underneath will be visible. Here is a figure that shows how the movie looks like:

MoviePicture

The HTML File

<!DOCTYPE HTML>
<html>
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <title>Working with Canvas</title>
    <script type="text/javascript" src="main.js"></script>
    <link href="style.css" rel="stylesheet" />
</head>
<body onload="handlePageLoad()">
    <video id="video" controls="controls" width="997px" height="587px" autoplay>
        <source src="vid.mp4" type="video/mp4" />
    </video>
    <div>
        <canvas id="c1" width="997px" height="587px" />
        <canvas id="c2" width="997px" height="587px" />
    </div>
</body>
</html>

The HTML includes two canvas elements and the video element. I use two canvas elements because one will hold the buffer that I’m working on (and make the black background transparent) and the other will show the movie after the adjustments are finished. I also use the following style.css file:

#c1
<span class="code-none">{
    background-image<span class="code-none">: url(Bg.png)<span class="code-none">;
    background-repeat<span class="code-none">: no-repeat<span class="code-none">;
<span class="code-none">}
#c2
<span class="code-none">{
    display<span class="code-none">: none<span class="code-none">;
<span class="code-none">}
#video
<span class="code-none">{
    display<span class="code-none">: none<span class="code-none">;
<span class="code-none">}</span></span></span></span></span></span></span></span></span></span></span></span></span></span>

In the CSS, I’m making sure not to display the buffer canvas and the video element and to show the background image in the first canvas element.

The Real Thing – The JavaScript Code

Here is the implementation of the JavaScript file that will help you do the job:

var video;
var ctx1;
var ctx2;
var width;
var height;

function timerCallback() {
    if (video.paused || video.ended) {
        return;
    }
    generateFrame();
    setTimeout(function () {
        timerCallback();
    }, 0);
}

function handlePageLoad() {
    video = document.getElementById("video");
    var c1 = document.getElementById("c1");
    ctx1 = this.c1.getContext("2d");
    var c2 = document.getElementById("c2");
    ctx2 = this.c2.getContext("2d");
    video.addEventListener("play", function () {
        width = this.videoWidth;
        height = this.videoHeight;
        timerCallback();
    }, false);
}

function generateFrame() {
    ctx2.drawImage(video, 0, 0, width, height);
    var frame = ctx2.getImageData(0, 0, width, height);
    var len = frame.data.length / 4;

    for (var i = 0; i < len; i++) {
        var r = frame.data[i * 4 + 0];
        var g = frame.data[i * 4 + 1];
        var b = frame.data[i * 4 + 2];
        if (g < 10 && r < 10 && b < 10) {
            frame.data[i * 4 + 3] = 0;
        }
    }
    ctx1.putImageData(frame, 0, 0);
    return;
}

Let's deep dive into the things that the functions are doing:

  • The handlePageLoad function is responsible to initialize all the canvas contexts and to add a listener to the play event of the video. When it occurs, it will start the timerCallback function.
  • The timerCallback function is responsible to stop the generation of frames when the video stops and to generate the current frame using the generateFrame function.
  • The generateFrame function is the function that does the heavy lifting of drawing the transparent video on the canvas elements. It draws the video frame to the buffer canvas and then extracts that frame. After extracting the frame, it passes on every pixel and checks whether it is black. If so, it sets its opacity to transparent.

The effect will look like:

TheFullExample

Summary

In the post, I showed you how to use a canvas in order to make changes to a running video on the fly. This is only an experiment that I made to show that it is possible to do that. The performance of this implementation isn’t so good and I would have thought about other solutions such as a video that includes the background image and the movie.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Gil Fink
Technical Lead sparXys
Israel Israel
Gil Fink is a web development expert and ASP.Net/IIS Microsoft MVP. He is the founder and owner of sparXys. He is currently consulting for various enterprises and companies, where he helps to develop Web and RIA-based solutions. He conducts lectures and workshops for individuals and enterprises who want to specialize in infrastructure and web development. He is also co-author of several Microsoft Official Courses (MOCs) and training kits, co-author of "Pro Single Page Application Development" book (Apress) and the founder of Front-End.IL Meetup. You can read his publications at his website: http://www.gilfink.net
Follow on   Twitter   Google+   LinkedIn

Comments and Discussions

 
Questioncan't replace your test file Pinmemberkware25-Aug-11 23:12 
AnswerRe: can't replace your test file PinmemberGil Fink26-Aug-11 3:37 
AnswerRe: can't replace your test file Pinmemberkware28-Aug-11 21:43 
GeneralRe: can't replace your test file PinmemberGil Fink29-Aug-11 3:31 
GeneralMy vote of 5 Pinmemberkware23-Aug-11 1:39 
GeneralRe: My vote of 5 PinmemberGil Fink24-Aug-11 4:42 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140916.1 | Last Updated 17 Aug 2011
Article Copyright 2011 by Gil Fink
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid