Click here to Skip to main content
Click here to Skip to main content
Go to top

The Kinect Mouse Controller C#

, 2 Nov 2012
Rate this:
Please Sign up or sign in to vote.
In this tip, I will describe how to implement a cursor controller in your project that use the Kinect to control the mouse of your PC.

The zip file provided with this article contains a DLL file KinectMouseController.DLL that could be used in all kinds of C# projects to control the cursor of Windows OS system from a Kinect device.

This schema explains how the provided DLL could be useful to send position (x,y) from the Kinect device to the Windows cursor:

Introduction

Controlling the cursor using the Kinect is a common question that I have noticed in many web sites, This feature could help a lot of people to make something more interactive, Designer, Architect, exploring maps, exploring 3D building ,... etc.

So in this quick description, I will explain how to add this feature to your project and also provide an application that could help people who aren't familiar with User32.dll and Kinect SDK.

Background

Having an experience with Kinect SDK isn't a must to benefit from this DLL,you should have some basics in coding a Kinect application to implement cursor controller in your Kinect project and use my code.

You should have a Kinect device, install the Kinect SDK and have hardware requirements to benefit from Kinect device in your computer.

Learn my first article about the Kinect for more details, and take a look at this.

What is User32 ?

User32 is one of the DLLs cached on system32 of Windows OS.

It's a Microsoft DLL that provides a lower-level control of Windows, menu, mouse, keyboard,... etc, it's based on many native methods; MOUSE_EVENT etc. that allows a control of a Windows user interface.

Using the Code

To use the DLL KinectMouseController in your project, you should first add a reference to this DLL in your project.

So after creating a project that will use the Kinect SDK, right click the reference folder of your project:

Choose the Browse tab, and select the KinectMouseController.DLL:

In your .cs file where you need to use the mouse controller, add a using KinectMouseController:

Now, you can use the following method:

SendMouseInput(int CursorX, int CursorY,int ScreenWidth, int ScreenHeight, bool isClicked)

In this method, we give the mouse position, the screen dimension and a boolean that represents a mouse click, true if down and false if up.

KinectMouseController.KinectMouseMethods.SendMouseInput(cursorX, cursorY, 
 (int)SystemInformation.PrimaryMonitorSize.Width, 
 (int)SystemInformation.PrimaryMonitorSize.Height, true); 

This code could be used in nui_SkeletonFrameReady() of Kinect SDK, in this code I'll use it to control the mouse using my right hand ,e.g :

void nui_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
       { 
            SkeletonFrame allSkeletons = e.SkeletonFrame;

            //get the first tracked skeleton
            SkeletonData skeleton = (from s in allSkeletons.Skeletons 
                                     where s.TrackingState == SkeletonTrackingState.Tracked
                                     select s).FirstOrDefault();
            //Test if skelton exist and tracked
            if (skeleton != null && skeleton.TrackingState == SkeletonTrackingState.Tracked)
            {
                SwG.Add(skeleton.Joints[JointID.HandRight].Position, nui.SkeletonEngine);

                // scale those Joints to the primary screen width and height 
                Joint scaledRight = skeleton.Joints[JointID.HandRight].ScaleTo((
                    int)SystemInformation.PrimaryMonitorSize.Width, 
                    (int)SystemInformation.PrimaryMonitorSize.Height, 
                    SkeletonMaxX, SkeletonMaxY);

                int cursorX = (int)scaledRight.Position.X;
                int cursorY = (int)scaledRight.Position.Y;
KinectMouseController.KinectMouseMethods.SendMouseInput
	(cursorX, cursorY, (int)SystemInformation.PrimaryMonitorSize.Width, 
	(int)SystemInformation.PrimaryMonitorSize.Height, false); 
          }

In the nui_SkeletonFrameReady(), we detect the first Skelton, after that we get the right hand portion of this skelton cursorX and cursorY, those coordinates are passed to the SendMouseInput() method of the KinectMouseController to take control of the mouse using the right hand of the first detected Skelton.

Points of Interest

I also wrote a related article:

Note

This code is tested with Windows 7, Kinect SDK beta 1 and Visual Studio 2010.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Saad Mousliki
Software Developer IWS
Morocco Morocco
currently an IT Support and Evangelist Professional
specialized in Web and software: JavaScript; Wakanda; ASP.NET, C #, WPF, JAVA, C... (SQL Server, jQuery, AJAX, ...)
I am also interested to mobile: Kinect, iOS, PhoneGap,Windows Phone and Android
To contact me : saadmos06@gmail.com
Follow on   Google+

Comments and Discussions

 
Questiongood day! PinmemberRalph Jason Austin Fonte15-Sep-14 22:20 
AnswerRe: good day! PinmemberSaad Mousliki15-Sep-14 22:29 
GeneralRe: good day! PinmemberRalph Jason Austin Fonte16-Sep-14 15:33 
QuestionCode PinmemberMinh Map Vu12-Feb-14 17:17 
AnswerRe: Code PinmemberSaad Mousliki12-Feb-14 17:56 
QuestionThe Kinect mouse controller C# Pinmembermiguel061222-Nov-12 19:37 
AnswerRe: The Kinect mouse controller C# Pinmembersaadmos22-Nov-12 22:36 
Generalhihi Pinmemberhoangquybao4-Sep-12 22:02 
GeneralRe: hihi Pinmembersaadmos4-Sep-12 23:54 
GeneralRe: hihi Pinmemberhoangquybao5-Sep-12 19:20 
GeneralRe: hihi Pinmembersaadmos14-Sep-12 0:32 
GeneralRe: hihi Pinmemberhoangquybao16-Sep-12 13:55 
QuestionCould you help me please PinmemberChinnawat Heranya11-Aug-12 6:14 
AnswerRe: Could you help me please Pinmembersaadmos14-Aug-12 3:07 
QuestionControl Windows 7 with Kinect SDK with Source Code Pinmemberalhazmi112-May-12 17:53 
AnswerRe: Control Windows 7 with Kinect SDK with Source Code Pinmembersaadmos2-May-12 22:05 
GeneralRe: Control Windows 7 with Kinect SDK with Source Code Pinmemberalhazmi112-May-12 22:33 
Questionthe kinect Pinmemberjimbobwillarby2-May-12 16:02 
QuestionThanks I see PinmemberMember 888696029-Apr-12 4:21 
Thank you for your code.
But you have used SDK beta1.
I am using SDK v1.0, So your code didn't work.
Can you help for SDK v 1.0??
AnswerRe: Thanks I see Pinmembersaadmos29-Apr-12 5:31 
GeneralRe: Thanks I see PinmemberMember 888696029-Apr-12 15:20 
GeneralRe: Thanks I see Pinmembersaadmos30-Apr-12 0:56 
Questionimages Pinmembersaadmos25-Apr-12 6:33 
QuestionKinect SDK Beta 1 or RTW v1? Pinmembergduncan41123-Apr-12 6:21 
AnswerRe: Kinect SDK Beta 1 or RTW v1? Pinmembersaadmos23-Apr-12 7:38 
QuestionYour images don't appear to be working. PinmemberMarcus Kramer23-Apr-12 4:30 
AnswerRe: Your images don't appear to be working. Pinmembersaadmos23-Apr-12 6:13 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web04 | 2.8.140916.1 | Last Updated 2 Nov 2012
Article Copyright 2012 by Saad Mousliki
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid