Kinect and WPF: Painting with Kinect using OpenNI

• Download binaries and source code

It's time for the second blog post considering Kinect and WPF. In the previous one, I showed you how to obtain the raw and depth images from the Kinect cameras. This time, I developed a simple painting application in Windows Presentation Foundation (I implemented a "punch" gesture, too). Here is a video demonstrating the way it is used:

How It Works

Wave on camera and Kinect will start tracking the waving hand. While the hand is moving, color is drawn in the corresponding screen points. Punch forward to stop painting. Punch again to restart!

Prerequisites

• Kinect sensor or other similar device
• Windows 7 / Vista operating system
• .NET Framework 4.0 (version 3.5 should work, too)
• OpenNI library (installation guide)

Implementing the Demo in C# and WPF

Step 0

Ensure that OpenNI is properly installed in your Windows operating system.

Step 1

• Open Visual Studio and create a new WPF application (I named it "KinectPaint") and select .NET 4 as the target framework (you can use either version 4 or 3.5).
• Add a reference to OpenNI.net.dll. OpenNI.net is found under C:\Program Files\OpenNI\Bin.
• Add an existing item and load SamplesConfig.xml to your project. Remember to copy this file in your output folder (Debug / Release). SamplesConfig.xml is found under C:\Program Files\OpenNI\Data. You need to have the default XML file replaced with something like the one I provided in my "how-to" post.

Step 2

Download my NuiHandTracker & NuiPositionEventArgs classes. These classes use OpenNI internally to track a hand. The first one also implements a custom "punch" gesture!

Step 3

Navigate to MainWindow.xaml and add a canvas in your XAML code (yes, it is both a painting and a WPF one):

<Canvas Name="canvasPaint" Width="640" Height="480" />  

Step 4

Navigate to MainWindow.xaml.cs and create a new instance of NuiHandTracker providing the SamplesConfig.xml file path as a parameter. Also handle its Hover and Push events.

_handTracker = new NuiHandTracker("SamplesConfig.xml");
_handTracker.Hover += new NuiHandTracker.HoverHandler(Hands_Hover);
_handTracker.Push += new NuiHandTracker.PushHandler(Hands_Push);

Step 5

We need a boolean member indicating whether the user is painting or not. Declare one and initialize it to true:

bool _isPainting = true;

Step 6

It's time to paint! Fill-in the Hands_Hover and Hands_Push event handlers:

void Hands_Hover(object sender, NuiPositionEventArgs e)
{
    // Hover event.
    if (_isPainting)
    {
        // Coordinates range from -320 to 320 and -240 to 240.
        DrawPixels(e.X + (IMAGE_WIDTH / 2), e.Y + (IMAGE_HEIGHT / 2));
    }
}

void Hands_Push(object sender, NuiPositionEventArgs e)
{
    // Push event!
    _isPainting = !_isPainting;
}

IMAGE_WIDTH and IMAGE_HEIGHT represent Kinect camera's resolution (640x480 respectively). DrawPixels method simply creates a new Ellipse object and adds it to the canvas element. X and Y coordinates, provided by OpenNI, range from -320 to +320 and -240 to +240. Canvas' coordinates start from 0, so we needed to convert the negative coordinates into positive ones properly. Here is the DrawPixels method:

private void DrawPixels(float x, float y)
{
    Dispatcher.BeginInvoke((Action)delegate
    {
        Ellipse ellipse = new Ellipse
        {
            Fill = new SolidColorBrush(Colors.Blue),
            Width = 25,
            Height = 25
        };

        Canvas.SetLeft(ellipse, x);
        Canvas.SetBottom(ellipse, y);
        
        canvasPaint.Children.Add(ellipse);
    });
}

You are done! KinectPaint is ready! I have added extra functionality in the demo I provide (such as random color generation and the ability to view the raw image from the camera).

Download the source code and have fun!