Click here to Skip to main content
Click here to Skip to main content
Technical Blog

Tagged as

Kinect for Windows version 2: Hand tracking

, 11 Apr 2014 CPOL
Rate this:
Please Sign up or sign in to vote.
NOTE: This is preliminary software and/or hardware and APIs are preliminary and subject to change. Kinect is awesome for body tracking, but Kinect version 2 is more than finding the human joints positions. Today we’ll see how we can track hands and thumbs and specify the hands’ state. In

NOTE: This is preliminary software and/or hardware and APIs are preliminary and subject to change.

Kinect is awesome for body tracking, but Kinect version 2 is more than finding the human joints positions. Today we’ll see how we can track hands and thumbs and specify the hands’ state. In this article, we’ll extend the functionality we built in the previous blog posts, so you might want to revise them before going on.

Download the source code of this example

Requirements

Video

Here is a brief video I created demonstrating the functionality we are going to examine (cudos to my young sister).

Hand tracking

Finding the position of a hand or thumb is just like finding the position of every other joint. Remember the snippet we used before? Here’s how you can get the position of each hand:

<code class="language-csharp">
void Reader_MultiSourceFrameArrived(object sender, MultiSourceFrameArrivedEventArgs e)
{
    var reference = e.FrameReference.AcquireFrame();

    // Color
    // Display the color stream...

    // Body
    using (var frame = reference.BodyFrameReference.AcquireFrame())
    {
        if (frame != null)
        {
            canvas.Children.Clear();

            _bodies = new Body[frame.BodyFrameSource.BodyCount];

            frame.GetAndRefreshBodyData(_bodies);

            foreach (var body in _bodies)
            {
                if (body != null)
                {
                    if (body.IsTracked)
                    {
                        // Find the joints
                        Joint handRight = body.Joints[JointType.HandRight];
                        Joint thumbRight = body.Joints[JointType.ThumbRight];

                        Joint handLeft = body.Joints[JointType.HandLeft];
                        Joint thumbLeft = body.Joints[JointType.ThumbLeft];
                    }
                }
            }
        }
    }
}

Displaying these joints is easy. Simply call the DrawPoint method we created before. In today’s example, I have slightly modified the source code to draw different colors:

<code class="language-csharp">
// Find the joints
Joint handRight = body.Joints[JointType.HandRight];
Joint thumbRight = body.Joints[JointType.ThumbRight];
Joint handLeft = body.Joints[JointType.HandLeft];
Joint thumbLeft = body.Joints[JointType.ThumbLeft];

// Draw hands and thumbs
canvas.DrawPoint(handRight);
canvas.DrawPoint(handLeft);
canvas.DrawPoint(thumbRight);
canvas.DrawPoint(thumbLeft);

Hand states

Currently, Kinect supports the following hand states. They are parts of the HandState enum:

  • Open
  • Closed
  • Lasso
  • Unknown
  • NotTracked

Note that the hand states are members of the body object, regardless of the corresponding hand joints.. Here’s the way to access the states:

<code class="language-csharp">
if (body.IsTracked)
{
    // Find and draw the joints

    // Find the right hand state
    switch (body.HandRightState)
    {
        case HandState.Open:
            break;
        case HandState.Closed:
            break;
        case HandState.Lasso:
            break;
        case HandState.Unknown:
            break;
        case HandState.NotTracked:
            break;
        default:
            break;
    }

    // Find the left hand state
    switch (body.HandLeftState)
    {
        case HandState.Open:
            break;
        case HandState.Closed:
            break;
        case HandState.Lasso:
            break;
        case HandState.Unknown:
            break;
        case HandState.NotTracked:
            break;
        default:
            break;
    }
}

Can’t be any easier!

After that, I simply created some <TextBlock> elements (named tblRightHandState and tblLeftHandState respectively) and updated each hand state. Here’s the complete source code:

<code class="language-csharp">
using (var frame = reference.BodyFrameReference.AcquireFrame())
{
    if (frame != null)
    {
        canvas.Children.Clear();

        _bodies = new Body[frame.BodyFrameSource.BodyCount];

        frame.GetAndRefreshBodyData(_bodies);

        foreach (var body in _bodies)
        {
            if (body != null)
            {
                if (body.IsTracked)
                {
                    // Find the joints
                    Joint handRight = body.Joints[JointType.HandRight];
                    Joint thumbRight = body.Joints[JointType.ThumbRight];

                    Joint handLeft = body.Joints[JointType.HandLeft];
                    Joint thumbLeft = body.Joints[JointType.ThumbLeft];

                    // Draw hands and thumbs
                    canvas.DrawPoint(handRight);
                    canvas.DrawPoint(handLeft);
                    canvas.DrawPoint(thumbRight);
                    canvas.DrawPoint(thumbLeft);

                    // Find the hand states
                    string rightHandState = "-";
                    string leftHandState = "-";

                    switch (body.HandRightState)
                    {
                        case HandState.Open:
                            rightHandState = "Open";
                            break;
                        case HandState.Closed:
                            rightHandState = "Closed";
                            break;
                        case HandState.Lasso:
                            rightHandState = "Lasso";
                            break;
                        case HandState.Unknown:
                            rightHandState = "Unknown...";
                            break;
                        case HandState.NotTracked:
                            rightHandState = "Not tracked";
                            break;
                        default:
                            break;
                    }

                    switch (body.HandLeftState)
                    {
                        case HandState.Open:
                            leftHandState = "Open";
                            break;
                        case HandState.Closed:
                            leftHandState = "Closed";
                            break;
                        case HandState.Lasso:
                            leftHandState = "Lasso";
                            break;
                        case HandState.Unknown:
                            leftHandState = "Unknown...";
                            break;
                        case HandState.NotTracked:
                            leftHandState = "Not tracked";
                            break;
                        default:
                            break;
                    }

                    tblRightHandState.Text = rightHandState;
                    tblLeftHandState.Text = leftHandState;
                }
            }
        }
    }
}

What’s next?

We are expecting a new SDK update from Microsoft in the next few days, so stay tuned for more great staff like facial expressions.

PS 2: New Kinect book – 20% off

Well, I am publishing a new ebook about Kinect development in a couple months. It is an in-depth guide about Kinect, using simple language and step-by-step examples. You’ll learn usability tips, performance tricks and best practices for implementing robust Kinect apps. Please meet Kinect Essentials, the essence of my 3 years of teaching, writing and developing for the Kinect platform. Oh, did I mention that you’ll get a 20% discount if you simply subscribe now? Hurry up ;-)

Subscribe here for 20% off

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Vangos Pterneas
Product Manager LightBuzz
United Kingdom United Kingdom
Vangos Pterneas is a Microsoft Most Valuable Professional in the Kinect technology. He helps companies from all over the world grow their revenue by creating profitable software products. Vangos is the owner of LightBuzz Software agency and author of two technical books.
Follow on   Twitter   Google+   LinkedIn

Comments and Discussions

 
QuestionMy kinect cannot recognize the hand states PinmemberMember 1104175212-Oct-14 22:19 
AnswerRe: My kinect cannot recognize the hand states PinprofessionalVangos Pterneas13-Oct-14 1:16 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.141223.1 | Last Updated 11 Apr 2014
Article Copyright 2014 by Vangos Pterneas
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid