The zip file provided with this article contains a DLL file KinectMouseController.DLL that could be used in all kinds of C# projects to control the cursor of Windows OS system from a Kinect device.
This schema explains how the provided DLL could be useful to send position (x,y) from the Kinect device to the Windows cursor:
Controlling the cursor using the Kinect is a common question that I have noticed in many web sites, This feature could help a lot of people to make something more interactive, Designer, Architect, exploring maps, exploring 3D building ,... etc.
So in this quick description, I will explain how to add this feature to your project and also provide an application that could help people who aren't familiar with User32.dll and Kinect SDK.
Having an experience with Kinect SDK isn't a must to benefit from this DLL,you should have some basics in coding a Kinect application to implement cursor controller in your Kinect project and use my code.
You should have a Kinect device, install the Kinect SDK and have hardware requirements to benefit from Kinect device in your computer.
Learn my first article about the Kinect for more details, and take a look at this.
What is User32 ?
User32 is one of the DLLs cached on system32 of Windows OS.
It's a Microsoft DLL that provides a lower-level control of Windows, menu, mouse, keyboard,... etc, it's based on many native methods;
MOUSE_EVENT etc. that allows a control of a Windows user interface.
Using the Code
To use the DLL
KinectMouseController in your project, you should first add a reference to this DLL in your project.
So after creating a project that will use the Kinect SDK, right click the reference folder of your project:
Choose the Browse tab, and select the KinectMouseController.DLL:
In your .cs file where you need to use the mouse controller, add a
Now, you can use the following method:
SendMouseInput(int CursorX, int CursorY,int ScreenWidth, int ScreenHeight, bool isClicked)
In this method, we give the mouse position, the screen dimension and a boolean that represents a mouse click,
true if down and
false if up.
This code could be used in
nui_SkeletonFrameReady() of Kinect SDK, in this code I'll use it to control the mouse using my right hand ,e.g :
void nui_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
SkeletonFrame allSkeletons = e.SkeletonFrame;
SkeletonData skeleton = (from s in allSkeletons.Skeletons
where s.TrackingState == SkeletonTrackingState.Tracked
if (skeleton != null && skeleton.TrackingState == SkeletonTrackingState.Tracked)
Joint scaledRight = skeleton.Joints[JointID.HandRight].ScaleTo((
int cursorX = (int)scaledRight.Position.X;
int cursorY = (int)scaledRight.Position.Y;
(cursorX, cursorY, (int)SystemInformation.PrimaryMonitorSize.Width,
nui_SkeletonFrameReady(), we detect the first
Skelton, after that we get the right hand portion of this
skelton cursorX and
cursorY, those coordinates are passed to the
SendMouseInput() method of the
KinectMouseController to take control of the mouse using the right hand of the first detected
Points of Interest
I also wrote a related article:
This code is tested with Windows 7, Kinect SDK beta 1 and Visual Studio 2010.