12,830,941 members (41,801 online)
Technical Blog
alternative version

#### Stats

12.5K views
4 bookmarked
Posted 22 Jul 2011

# Kinect – Calculator – Adjust Skeleton Movements To Mouse

, 22 Jul 2011 Ms-PL
 Rate this:
CodeProjectIn my previous post Kinect – Create Buttons I’ve showed one approach how to create Kinect Buttons for Windows, over the next posts I’ll show more ways to accomplish that by moving windows Cursor based on Kinect Skeleton Right Hand Position.Why To Create Kinect Button?Why not using wi

In my previous post Kinect – Create Buttons I’ve showed one approach how to create Kinect Buttons for Windows, over the next posts I’ll show more ways to accomplish that by moving windows Cursor based on Kinect Skeleton Right Hand Position.

### Why To Create Kinect Button?

Why not using windows cursor and create brilliant hand or head () movement to simulate Click, so we don’t need to create designated Kinect Buttons.

1. You had to stand at least 1 meter from the computer screen and even so I’m young I can’t see very well from that distance…
2. Kinect precision is still not perfect and it’s hard to hit a 48x48 button from that distance.
3. Even if you increase desktop resolution it’s not easy and not practical to work with both hands to perform Click.
4. But, I’m going to do that anyway! Magnifying glass, Cool Head or Hand movements etc…

In this post I’ll take those Kinect Button(working on Timer) implement on a simple Calculator I’ve built and hook the Windows mouse to the Skeleton movement.

The only major issue with that is to adjust the Skeleton Position to your Screen Resolution, as you know the Skeleton coordinates are expressed in meters and comes from 640x480 screen, so I’ve create new class called Positions that will help me to do it.

The AdjustToScreen method gets the Joint (and based on the current screen resolution) and another gets the Joint and specific screen width and height.

```public static class Positions
{

//For - Resolution640x480
private const float SkeletonMaxX = 0.6f;

private const float SkeletonMaxY = 0.4f;

float maxJointPosition, float jointPosition)
{

var value = (((((float)primaryScreenResolution) / maxJointPosition) / 2f) *
jointPosition)
+ (primaryScreenResolution / 2);

if (value > primaryScreenResolution || value < 0f) return 0f;

return value;

}

/// <span class="code-SummaryComment"><summary>
</span>```

Now I’ve create another helper class called – `NativeMethods` to move the set the Cursor  Position based on Skeleton Joint position.

```public static class NativeMethods
{

public partial class MouseOperations
{

[DllImport("user32.dll")]
public static extern bool SetCursorPos(int x, int y);

[DllImport("user32.dll")]

public static extern bool GetCursorPos(out Point pt);
}

}```

In the `SkeletonFrameReady` I’ve added another quality check, also check the Joint TrakingState and not just the Data `TrakingState`, also make sure the joint position quality (W) is high enough before moving the mouse.

The quality check will prevent that mouse from jumping around.

```void SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{

foreach (SkeletonData data in e.SkeletonFrame.Skeletons)
{

//Tracked that defines whether a skeleton is 'tracked' or not.
//The untracked skeletons only give their position.
if (data.TrackingState != SkeletonTrackingState.Tracked) continue;

//Each joint has a Position property that is defined by a Vector4: (x, y, z, w).

//The first three attributes define the position in camera space.
//The last attribute (w)
//gives the quality level (between 0 and 1) of the

foreach (Joint joint in data.Joints)
{

switch (joint.ID)
{

case JointID.HandRight:
if (joint.Position.W < 0.6f || joint.TrackingState !=
JointTrackingState.Tracked) return;// Quality check

//Based on Skeleton Right Hand Position adjust the location
//to the screen

if (newPos.X == 0f || newPos.Y == 0f) return;

NativeMethods.MouseOperations.SetCursorPos(
Convert.ToInt32(newPos.X), Convert.ToInt32(newPos.Y));

break;
}

}
}

}```

Make sure you enable `TransformSmooth`, you don’t want the mouse jumping around. (Kinect–How to Apply Smooths Frame Display Using TransformSmoothParameters)

```_kinectNui.SkeletonEngine.TransformSmooth = true;
var parameters = new TransformSmoothParameters

{
Smoothing = 1.0f,

Correction = 0.1f,
Prediction = 0.1f,

};
_kinectNui.SkeletonEngine.SmoothParameters = parameters;```

Live Demo

### Demo Project

This project includes a Common class with this post main methods, from Moving and mouse and adjusting Kinect Skeleton to desktop resolution.

Enjoy

## Share

 Architect Sela Israel
Shai Raiten is VS ALM MVP, currently working for Sela Group as a ALM senior consultant and trainer specializes in Microsoft technologies especially Team System and .NET technology. He is currently consulting in various enterprises in Israel, planning and analysis Load and performance problems using Team System, building Team System customizations and adjusts ALM processes for enterprises. Shai is known as one of the top Team System experts in Israel. He conducts lectures and workshops for developers\QA and enterprises who want to specialize in Team System.

My Blog: http://blogs.microsoft.co.il/blogs/shair/