In my previous post Kinect – Create Buttons I’ve showed one approach how to create Kinect Buttons for Windows, over the next posts I’ll show more ways to accomplish that by moving windows Cursor based on Kinect Skeleton Right Hand Position.
Why To Create Kinect Button?
Why not using windows cursor and create brilliant hand or head () movement to simulate Click, so we don’t need to create designated Kinect Buttons.
- You had to stand at least 1 meter from the computer screen and even so I’m young I can’t see very well from that distance…
- Kinect precision is still not perfect and it’s hard to hit a 48x48 button from that distance.
- Even if you increase desktop resolution it’s not easy and not practical to work with both hands to perform Click.
- But, I’m going to do that anyway! Magnifying glass, Cool Head or Hand movements etc…
In this post I’ll take those Kinect Button(working on Timer) implement on a simple Calculator I’ve built and hook the Windows mouse to the Skeleton movement.
The only major issue with that is to adjust the Skeleton Position to your Screen Resolution, as you know the Skeleton coordinates are expressed in meters and comes from 640x480 screen, so I’ve create new class called Positions that will help me to do it.
The AdjustToScreen method gets the Joint (and based on the current screen resolution) and another gets the Joint and specific screen width and height.
public static class Positions
private const float SkeletonMaxX = 0.6f;
private const float SkeletonMaxY = 0.4f;
private static float Adjust(int primaryScreenResolution,
float maxJointPosition, float jointPosition)
var value = (((((float)primaryScreenResolution) / maxJointPosition) / 2f) *
+ (primaryScreenResolution / 2);
if (value > primaryScreenResolution || value < 0f) return 0f;
Now I’ve create another helper class called –
NativeMethods to move the set the Cursor Position based on Skeleton Joint position.
public static class NativeMethods
public partial class MouseOperations
public static extern bool SetCursorPos(int x, int y);
public static extern bool GetCursorPos(out Point pt);
SkeletonFrameReady I’ve added another quality check, also check the Joint TrakingState and not just the Data
TrakingState, also make sure the joint position quality (W) is high enough before moving the mouse.
The quality check will prevent that mouse from jumping around.
void SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
foreach (SkeletonData data in e.SkeletonFrame.Skeletons)
if (data.TrackingState != SkeletonTrackingState.Tracked) continue;
foreach (Joint joint in data.Joints)
if (joint.Position.W < 0.6f || joint.TrackingState !=
var newPos = Positions.AdjustToScreen(joint);
if (newPos.X == 0f || newPos.Y == 0f) return;
Make sure you enable
TransformSmooth, you don’t want the mouse jumping around. (Kinect–How to Apply Smooths Frame Display Using TransformSmoothParameters)
_kinectNui.SkeletonEngine.TransformSmooth = true;
var parameters = new TransformSmoothParameters
Smoothing = 1.0f,
Correction = 0.1f,
Prediction = 0.1f,
JitterRadius = 0.05f,
MaxDeviationRadius = 0.05f
_kinectNui.SkeletonEngine.SmoothParameters = parameters;
This project includes a Common class with this post main methods, from Moving and mouse and adjusting Kinect Skeleton to desktop resolution.