Minimalist Example of Video Capture Using Java

Introduction

Last year I wrote an article documenting Java class creation and distribution, using a video display and picture capture class as an example. The article concentrated on the mechanics of creating redistributable Java classes rather than video capture, and did not delve deeply into the nuts and bolts of Java Media Framework audio and video capture. So, I wanted to get to know how to use the JMF better. Specifically, I wanted to be able to capture both sight and sound into a file using Java. There was not much information that could be found (vague allusions to clones), so when I finally found a solid explanation and got the code to work, it seemed necessary to share the example.

Background

Java and the Java Media Framework (JMF) offer a powerful yet relatively quick and easy way of developing multimedia applications. But, there is a lot of subtlety in making the platform work. The example presented here shows the basics of finding audio and video devices, creating data sources from these devices, cloning a data source so that it can be used in the player to be displayed and in the processor so it can be captured, and capturing the data as an AVI video file.

Using the Code

The code example is a very basic illustration of capturing sight and sound into a file using Java. To use the code, the JMF has to be installed on the system running it. NetBeans IDE 6.7.1 was used to build the application, but any IDE will work as well.

Merging Video with Audio

In order to capture the video with sound, two data sources are required, but the objects that use data sources only take one data source as an argument. A new data source has to be created using the two original sources. This is done with an array of the data sources to be merged and the javax.media.Manager.createMergingDataSource function:

// merge audio and video data sources
// --------------------------
DataSource mixedDataSource = null;
DataSource dsArray[] = new DataSource[2];
// this is a cloned datasource and is controlled
// by the master clonable data source
dsArray[0] = CaptureVidDS;
dsArray[1] = audioDataSource;
try {
    mixedDataSource = 
      javax.media.Manager.createMergingDataSource(dsArray);
} catch (Exception e) {
    // your exception handling here
}

The Processor

A processor object is similar to a player. The processor processes data and creates an output in the destination format required. The processor is created using Manager.createRealizedProcessor from the processor model, which itself is created using the data source, data format, and the output type as arguments.

// ----------------------
// create a new processor
// ----------------------
ProcessorModel processorModel = 
  new ProcessorModel(mixedDataSource, outputFormat, outputType);
try {
    processor = Manager.createRealizedProcessor(processorModel);
} catch (Exception e) {
    // your exception handling here
}

The Data Sink

The DataSink object takes a DataSource as input, and renders the output to a specified destination. In this case, the input is the data output of the processor which controls the data sources and the output format. The data sink will control capture to a specific media destination.

// get the output of the processor to be used as the datasink input
DataSource source = processor.getDataOutput();

// create a File protocol MediaLocator with the location
// of the file to which bits are to be written
MediaLocator mediadestination = new MediaLocator("file:.\\vidcap.avi");

// create a datasink to create the video file
dataSink = Manager.createDataSink(source, mediadestination);
dataSinkListener = new TheDataSinkListener();

// create a listener to control the datasink
dataSink.addDataSinkListener(dataSinkListener);
dataSink.open();

The Data Sink Listener

The DataSinkListener is used just like a window listener to control the data sink based on internal events or user actions. In this program's case, the closing of the frame created to display the video causes the processor to stop, which cuts off the data stream to the data sink. The data sink listener allows the program thread to sleep just before the program completely closes, to make sure the JMF threads are complete prior to the program ending. In this case, the sleep is 10 milliseconds.

public void windowClosing(WindowEvent event) {
Frame f = (Frame) event.getSource();

// Stop the processor doing the movie capture first
processor.stop();
processor.close();

// Closing the processor will end the data stream to the data sink.
// Wait for the end of stream to occur before closing the datasink
dataSinkListener.waitEndOfStream(10);
dataSink.close();

// stop and close the player which closes the video data source
player.stop();
player.close();

//------------------------------------------------
// dispose of the frame and close the application
//------------------------------------------------
f.dispose();
System.exit(0);

....


/**
 * Control the ending of the program prior to closing the data sink
 */
class TheDataSinkListener implements DataSinkListener {

    boolean endOfStream = false;

    // Flag the ending of the data stream
    public void dataSinkUpdate(DataSinkEvent event) {
        if (event instanceof javax.media.datasink.EndOfStreamEvent) {
            endOfStream = true;
        }
    }

    /*Cause the current thread to sleep if the data stream is still available.
     * This makes certain that JMF threads are done prior
     * to closing the data sink and finalizing the output file
     */
    public void waitEndOfStream(long checkTimeMs) {
        while (!endOfStream) {
            try {
                //Thread.currentThread().sleep(checkTimeMs);
                Thread.sleep(checkTimeMs);
            } catch (InterruptedException ie) {
                // your exception handling here
            }
        }
    }

The Complete Source Code

//Java SE 1.6.0_13, Java Media Framework 2.1.1e

package vidcapminimal;

import java.util.Vector;
import java.awt.*;
import java.awt.event.*;
import javax.media.*;
import javax.media.format.*;
import javax.media.protocol.*;
import javax.media.datasink.*;

/**
 *
 * @author TW Burger from code by pawan_sin99
 */
public class Main {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {

        // run a video capture object
        new VidCap();
    }
}

class VidCap {

    CaptureDeviceInfo device = null;
    MediaLocator ml = null;
    Player player = null;
    Component videoScreen = null;
    Processor processor = null;
    DataSink dataSink = null;
    TheDataSinkListener dataSinkListener = null;

    VidCap() {
        try { //gets a list of devices how support the given video format
            Vector deviceList = CaptureDeviceManager.getDeviceList(new YUVFormat());

            // get video device - the first one is almost always the only available camera
            device = (CaptureDeviceInfo) deviceList.firstElement();
            ml = device.getLocator();

            //create a source from the device
            DataSource ods = null;
            ods = Manager.createDataSource(ml);

            // Clone the video source so it can be displayed and used to capture
            // the video at the same time. Trying to use the same source for two
            // purposes would cause a "source is in use" error
            DataSource cloneableDS = Manager.createCloneableDataSource(ods);
            DataSource PlayerVidDS = cloneableDS;

            // The video capture code will use the clone which is controlled by the player
            DataSource CaptureVidDS = 
              ((javax.media.protocol.SourceCloneable) cloneableDS).createClone();

            // -----------------------------------------------------------------
            // Display video
            // by starting the player on the source
            // clonable data source the clones are fed data
            // stopping the player will stop the video flow to the clone data source
            // -----------------------------------------------------------------

            player = Manager.createRealizedPlayer(PlayerVidDS);
            player.start();

            // get an audio device and create an audio data source
            deviceList = CaptureDeviceManager.getDeviceList(
                            new javax.media.format.AudioFormat(null));
            device = (CaptureDeviceInfo) deviceList.firstElement();
            ml = device.getLocator();
            DataSource audioDataSource = Manager.createDataSource(ml);

            // merge audio and video data sources
            // --------------------------
            DataSource mixedDataSource = null;
            DataSource dsArray[] = new DataSource[2];
            dsArray[0] = CaptureVidDS; // this is a cloned datasource
                                 // and is controlled by the master clonable data source
            dsArray[1] = audioDataSource;
            try {
                mixedDataSource = javax.media.Manager.createMergingDataSource(dsArray);
            } catch (Exception e) {
                // your exception handling here
            }

            // setup output file format to msvideo
            FileTypeDescriptor outputType = 
                new FileTypeDescriptor(FileTypeDescriptor.MSVIDEO);

            // setup output video and audio data format
            Format outputFormat[] = new Format[2];
            //outputFormat[0] = new VideoFormat(VideoFormat.RGB);
            outputFormat[0] = new VideoFormat(VideoFormat.YUV);
            outputFormat[1] = new AudioFormat(AudioFormat.LINEAR);

            // ----------------------
            // create a new processor
            // ----------------------
            ProcessorModel processorModel = 
              new ProcessorModel(mixedDataSource, outputFormat, outputType);
            try {
                processor = Manager.createRealizedProcessor(processorModel);
            } catch (Exception e) {
                // your exception handling here
            }

            try {
                // get the output of the processor to be used as the datasink input 
                DataSource source = processor.getDataOutput();

                // create a File protocol MediaLocator with the location
                // of the file to which bits are to be written
                MediaLocator mediadestination = new MediaLocator("file:.\\vidcap.avi");

                // create a datasink to create the video file
                dataSink = Manager.createDataSink(source, mediadestination);
                
                // create a listener to control the datasink
                dataSinkListener = new TheDataSinkListener();
                dataSink.addDataSinkListener(dataSinkListener);
                dataSink.open();

                // now start the datasink and processor
                dataSink.start();

                processor.start();
            } catch (Exception e) {
                // your exception handling here
            }

            videoScreen = player.getVisualComponent();

            Frame frm = new Frame("Display of Webcam");
            frm.addWindowListener(new WindowAdapter() {

                @Override
                public void windowClosing(WindowEvent event) {
                    Frame f = (Frame) event.getSource();

                    // Stop the processor doing the movie capture first
                    processor.stop();
                    processor.close();

                    // Closing the processor will end the data stream to the data sink.
                    // Wait for the end of stream to occur before closing the datasink
                    dataSinkListener.waitEndOfStream(10);
                    dataSink.close();

                    // stop and close the player which closes the video data source
                    player.stop();
                    player.close();

                    //------------------------------------------------
                    // dispose of the frame and close the application
                    //------------------------------------------------
                    f.dispose();
                    System.exit(0);
                    //------------------------------------------------
                }
            });

            frm.setBounds(10, 10, 300, 300);
            frm.add(videoScreen);
            frm.setVisible(true);

        } catch (Exception e) {
            System.out.println(e);
        }
    }
}

/**
 * Control the ending of the program prior to closing the data sink
 *
 */
class TheDataSinkListener implements DataSinkListener {

    boolean endOfStream = false;

    // Flag the ending of the data stream
    public void dataSinkUpdate(DataSinkEvent event) {
        if (event instanceof javax.media.datasink.EndOfStreamEvent) {
            endOfStream = true;
        }
    }

    /*Cause the current thread to sleep if the data stream is still available.
     * This makes certain that JMF threads are done prior
     * to closing the data sink and finalizing the output file
     */
    public void waitEndOfStream(long checkTimeMs) {
        while (!endOfStream) {
            try {
                //Thread.currentThread().sleep(checkTimeMs);
                Thread.sleep(checkTimeMs);
            } catch (InterruptedException ie) {
                // your exception handling here
            }
        }
    }
}

Points of Interest

The code was developed and tested using Windows XP. The code was also tested using Ubuntu 9.1, and in Ubuntu, the code runs as is (with changes to the file name path possibly needed), but the JMF would not recognize the camera in my HP 6830s laptop.

History

• 1.0: This is the first version of the code simplified for use as an instructive example. If there is interest expressed in this material, I will expand the code to include a snapshot (video frame), capture and control of device, and format selections.
• 1.1: May 26, 2010: I found that the setting of the output file video format to RGB did not work for an older (comes with Windows 98 only software) Philips PCA646VC camera I tested. Changing the code to:

outputFormat[0] = new VideoFormat(VideoFormat.RGB);
outputFormat[0] = new VideoFormat(VideoFormat.YUV);

Also allowed the camera to create a usable video file.

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)