Click here to Skip to main content
Click here to Skip to main content

The Poor Man's Mouse Gesture

By , 15 Jun 2005
 

Introduction

In a project I am currently working on I had the need for the ability to handle simple mouse gestures. I wanted to be able to do stuff like change the current selection in a combo box by simply moving the mouse up or down, and not having to switch the input focus from where I was working to the combo box and back again. I found a C++ article here[^] on The Code Project about mouse gestures, but it seemed to be more of an intellectual treatise on using neural networks to recognize predetermined mouse gestures rather than a ready to use class library that could be plugged into any project others might be working on (or maybe I am just too dense to figure it out ). As I am not one who likes to spend hours searching all over the web trying to find something that may or may not be there, I decided that the best way to get exactly what I wanted was to write it myself. This class and the article are the results of my efforts. Enjoy .

The idea (or how it works)

My idea on how to get this thing to work is actually quite simple. First of all, it involves setting up a mouse hook to track what the user is doing with the mouse. When the user presses one of the mouse buttons down, I start the mouse gesture by setting up a bounding square around the point where the mouse is located. If the user moves the mouse over the edge of the square I have a mouse gesture. The idea is to move the square along with the mouse, keeping track of which edge the mouse moves over. Every time the mouse moves over a different edge I get a new direction in the gesture.

Sequence for mouse gesture 'right and down'

You can get visual feedback of the mouse tracks and the bounding square in action if you download and build the demonstration application in DEBUG mode. You can turn the visual feedback off by commenting out line number 55 in the MouseGesture.cpp file.

//////////////////////////////////////////////////////////////////////
// Uncomment the #define _SHOW_GESTURE line to draw mouse points and
// bounding squares on screen when the gesture is being made. 
//////////////////////////////////////////////////////////////////////

#   define _SHOW_GESTURE  // this line controls visual feedback

Using the CMouseGesture class

The CMouseGesture class is a simple class that you can plug into any C++ Windows code so that you too can have mouse gesture recognition in your applications. It is really quite simple. Just add the class files, MouseGesture.cpp and MouseGesture.h, to your project. Declare a CMouseGesture object, tell it which window to watch for mouse gestures, and what gestures to watch for. When the CMouseGesture object recognizes a mouse gesture it posts a registered message to the window it's watching, passing the ID of the gesture. To create the ID of the registered message pass the MOUSE_GESTURE_MESSAGE_STRING to the RegisterWindowMessage() function. The WPARAM parameter of the registered message will be the identification number of the gesture recognized, and the LPARAM parameter will be a pointer to the CMouseGesture object that posted the message.

The following code snippet sets up the CMouseGesture object to watch for a mouse gesture where the user presses the right mouse button, then moves the mouse to the right, then down, and then lets up the right mouse button again:

#include "MouseGesture.h"
#define ID_GESTURE_RIGHT_DOWN 1001

const UINT WMU_MOUSEGESTURE = 
       RegisterWindowMessage(MOUSE_GESTURE_MESSAGE_STRING);
.
.
.
    // Setup the CMouseGesture object
    CMouseGesture MyMouseGesture;
    MyMouseGesture.Attach(GetSafeHwnd());
 
    CMouseGesture::Motion WatchFor[] = { CMouseGesture::RightButton,
                                         CMouseGesture::Right,
                                         CMouseGesture::Down };
                                         
    MyMouseGesture.AddGesture(ID_GESTURE_RIGHT_DOWN, WatchFor, 3);
.
.
.
// Handle WMU_MOUSEGESTURE message
LRESULT CMyWindowClass::OnMouseGesture(WPARAM wp, LPARAM lp)
{
    UINT GestureID = (UINT)wp;
    CMouseGesture *pMouseGesture = (CMouseGesture *)lp;
    if (GestureID == ID_GESTURE_RIGHT_DOWN)
    {
        // Do whatever
    }
}

CMouseGesture class members

CMouseGesture::CMouseGesture()

CMouseGesture class constructor

Initializes all the internal variables.

Parameters:

None

Returns:

Nothing

CMouseGesture::~CMouseGesture()

CMouseGesture class destructor

Cleans up the class and removes the mouse hook if it is no longer needed.

Parameters:

None

Returns:

Nothing

bool CMouseGesture::Attach(HWND hWnd,
                  UINT Distance /* = 25 */)

CMouseGesture::Attach

Initializes the CMouseGesture object, attaching it to the supplied window handle and starting the mouse hook.

Parameters:

  • hWnd

    The window handle of the window that this CMouseGesture is to watch. The CMouseGesture object will post a registered message to this window whenever it recognizes a mouse gesture.

  • Distance

    The minimum distance in pixels that the mouse has to move before a mouse motion is registered. This parameter is optional and defaults to 25 pixels.

Returns:

  • true, if successful.
  • false, if not successful.
void CMouseGesture::Detach()

CMouseGesture::Detach

Detaches the CMouseGesture object from the window it was attached to in the Attach function. Stops the mouse hook if it is no longer needed.

Parameters:

None

Returns:

Nothing

bool CMouseGesture::RemoveGesture(UINT nID)

CMouseGesture::RemoveGesture

Removes the gesture with the supplied nID from the CMouseGesture object.

Parameters:

  • nID

    The identification number of the gesture to remove.

Returns:

  • true, if the gesture with the identification number was found and removed.
  • false, if the gesture with the identification number was not found.
int CMouseGesture::AddGesture(UINT nID,
               const CMouseGesture::Motion* Motions,
               size_t count)
int CMouseGesture::AddGesture(UINT nID,
             const CMouseGesture::Gesture& rGesture)

CMouseGesture::AddGesture

Use these overloaded functions to add mouse gesture patterns to the CMouseGesture object.

Parameters:

  • nID

    The identification number of the gesture pattern. This number has to be unique to this CMouseGesture object. (A different CMouseGesture object can use the same ID.) This is the number that is passed back to the window when the gesture is recognized.

  • Motions

    A pointer to an array of Motions. The Motion type is an enum defined in the CMouseGesture class.

  • count

    The number of Motion elements in the Motions array.

  • rGesture

    A reference to a Gesture vector containing an array of Motions.

Returns:

  • -2

    The supplied mouse gesture has already been added.

  • -1

    The supplied ID number is already in use.

  • 0

    The supplied mouse gesture has an invalid format or nID is zero.

  • >0

    The gesture was successfully added, and the return value is the number of gestures in the CMouseGesture object.

Note: The CMouseGesture::Motion type is an enum, and the CMouseGesture::Gesture type is a typedef std::vector<Motion>.

    enum Motion { None         = 0x0000,

                  // These are the possible directions
                  Up           = 0x0001,
                  Down         = 0x0002,
                  Left         = 0x0003,
                  Right        = 0x0004,

                  // These are for the shift and control keys
                  Shift        = 0x0005,
                  Control      = 0x0006,

                  // These are bit flags for the mouse buttons
                  LeftButton   = 0x0100,
                  MiddleButton = 0x0200,
                  RightButton  = 0x0400
                };

    typedef std::vector<Motion> Gesture;

When you build the array (or vector) of Motions to pass to the AddGesture function you must make sure that the array has the following format:

MouseButton [Shift] [Control] Motion ...

The mouse button that is pressed down to start the mouse gesture must be the first element in the array. It can be one of LeftButton, MiddleButton, or RightButton. Following the mouse button can be the optional Shift and/or Control flags. Use Shift if the gesture requires the SHIFT key on the keyboard to be pressed. Use Control if the CTRL key is supposed to be pressed. If you want to use both, then Shift has to be first, and Control has to be second. After that comes one or more direction (Up, Down, Left, or Right) motions. You can not use the same direction motion twice immediately together.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

PJ Arends
President
Canada Canada
No Biography provided

Sign Up to vote   Poor Excellent
Add a reason or comment to your vote: x
Votes of 3 or less require a comment

Comments and Discussions

 
You must Sign In to use this message board.
Search this forum  
    Spacing  Noise  Layout  Per page   
QuestionVB version?memberRobert Dudley23-Apr-12 9:49 
Have you got a Visual Basic version of these? They're great but I don't know C++ yet
Generaltrack the gesture anywherememberYuval Twig30-Jan-06 11:38 
how would i to use this class in order to detect gestures anywhere on the screen and not only under the main window ??
GeneralRe: track the gesture anywherememberPJ Arends30-Jan-06 21:02 
One solution that comes to mind is to modify the GetGesturePointer() function to return the CMouseGesture* pointer attached to your main window.
CMouseGesture* CMouseGesture::GetGesturePointer(HWND hWnd, POINT pt)
{
...
    if (!PtInRect(&rc, pt))
    {
        // return NULL;
        return ((CMyMainWindow *)AfxGetMainWnd())->m_MouseGesturePointer;
    }
...
}
This code is untested, but it should give you an idea.
 
The problem with this approach is if you have more than one instance of the app running, or more than one app using this class, the class will get confused as to which app is supposed to recieve the mouse gesture messages.
 


"You're obviously a superstar." - Christian Graus about me - 12 Feb '03
 
"Obviously ???  You're definitely a superstar!!!" - mYkel - 21 Jun '04
 
"There's not enough blatant self-congratulatory backslapping in the world today..." - HumblePie - 21 Jun '05
 
Within you lies the power for good - Use it!
GeneralA different detectionmembertmangan22-Jun-05 5:53 
Thanks for the article.
 
It reminds me of a gesture recognition we did back at a CAD/CAM company around '83 or '84. The difference was that rather than detect direction, we used "cell order".
 
In other words, we record where the mouse went in the gesture, build a box big enough to hold it, then divide it into smaller cells. A 3x3 grid worked great, a 4x4 grid would be more precise (but with more work generating recognizable patterns). The pattern is then a sequence of which cells the mouse travelled through. (Well actually, our system used a pen, but it's the same thing). The nice thing with this is that the pattern can be detected no matter how big the gesture was made.
 
tim
 
tim
Founder, TMurgent Technologies
www.tmurgent.com
tmangan@tmurgent.com
GeneralRe: A different detectionmemberPJ Arends22-Jun-05 6:16 
I read about the grid method here[^]. My concern with that method was how big of a grid is needed to cover as many possible gestures as possible, and how to code it? I think I came up with a reasonable solution.
 
The other thing I was concerned about was recognizing diagonal motions in the gesture. While this class does not recognize diagonals, it can be extended to do so by using a hexagon instead of a square. The code for edge detection would be a little more complicated than what I have now. But currently I have no need for it in my app, so I did not go through the effort.
 


"You're obviously a superstar." - Christian Graus about me - 12 Feb '03
 
"Obviously ???  You're definitely a superstar!!!" - mYkel - 21 Jun '04
 
"There's not enough blatant self-congratulatory backslapping in the world today..." - HumblePie - 21 Jun '05
 
Within you lies the power for good - Use it!
GeneralRe: A different detectionmembertmangan22-Jun-05 6:47 
We used a 3x3 grid fairly effectively. Typically a user would set up 1 to 2 dozen motions to be detected. The tablet had a template with about 100 commands, so this was a reasonable subset for the things you did most often. This was before the days of using pull down menus and the like - everything was command driven.
 
The "corner" issue was handled by using multiple pattern definitions for the same motion.
 
With a grid of:
7 8 9
4 5 6
1 2 3
 
A check mark motion might be patterns:
4-2-5-9
4-1-2-5-8-9
4-1-2-5-6-9
4-5-2-5-8-9
4-5-2-5-6-9
etc...
 
As we let users register their own motion patterns nobody had to do the math! What I do like in your solution is the feedback box option.
 
Regards,
 
tim
Founder, TMurgent Technologies
www.tmurgent.com
tmangan@tmurgent.com
GeneralRe: A different detectionmemberredfish3421-Jan-06 10:07 
I see a problem with the cell grid approach. What if the gesture includes movements that reverse? Example, what if the user wants to move Right-Left-Right, make two horizontal lines? The cell grid approach fails because it only takes a snapshot of the final gesture state. It is unable able to distinguish between a single line and double-line gesture.
 
PJ Arends solution is pretty elegant. I dismissed it at first because the demo was sporadic. But taking a second look after looking at those neural network solutions and others, i think this 20th century-ish technology is quite adequate for the "working" programmer.
 


GeneralRe: A different detectionmembertmangan21-Jan-06 11:06 
Oh right. It's my memory that is bad. You use a bounding box that contains all measured points. So you would on a 3x3 box you would see a pattern of cells 6-5-4-5-6.
 
I also like the solution referenced by PJ - just remembering another old approach.

 
tim
Founder, TMurgent Technologies
www.tmurgent.com
tmangan@tmurgent.com
GeneralRe: A different detectionmemberKevin Pinkerton24-Jun-05 4:55 
Did that software run on a pdp-11? I worked with some cad software that allowed the operators to customize their own set of commands using a pen (instead of a mouse). It was very impressive and fast and the operators loved it. I cannot remember the software name, but the one pdp-11 with some lowly amount of ram supported something like 5 CAD workstations...
 
Kevin
GeneralRe: A different detectionmembertmangan24-Jun-05 8:13 
These were our own machines (ComputerVision) - basically a DG Nova clone.
 
tim
Founder, TMurgent Technologies
www.tmurgent.com
tmangan@tmurgent.com
GeneralEgo a gogosussHumblePie21-Jun-05 0:12 
Thank goodness we all know that your msg sig is tongue-in-cheek, PJ Smile | :)
 
There's not enough blatant self-congratulatory backslapping in the world today...

GeneralRe: Ego a gogo [modified]sussPJ Arends21-Jun-05 12:04 
Thank you, I was looking for something new to put in my sig and now I found it:p
 


"You're obviously a superstar." - Christian Graus about me - 12 Feb '03
 
"Obviously ???  You're definitely a superstar!!!" - mYkel - 21 Jun '04
 
"There's not enough blatant self-congratulatory backslapping in the world today..." - HumblePie - 21 Jun '05
 
Within you lies the power for good - Use it!
 
modified on Monday, May 12, 2008 3:39 PM

GeneralRe: Ego a gogosussHumblePie21-Jun-05 23:00 
I am an inspiration... some would argue that it marks me out as a true genius. Who am I to disagree?
 
Big Grin | :-D
QuestionMouse Gesture?memberDavidCrow20-Jun-05 6:48 
For the uninitiated, I think you should add a blurb or two about what a mouse gesture is and why you would want it.
 
- DC
 

"Ideas are a dime a dozen. People who put them into action are priceless." - Unknown


GeneralBravomemberStuart Konen16-Jun-05 13:40 
I must say you've done an excellent job with this. I especially like your coding style, it has the 'same' feel as a lot of my recent work; so it's easy to glance through it with haste. Keep up the good work =)
 
One suggestion though, some people may initially be confused about the blank list box. I originally thought it was the window attached to your gesture class, but it didn't take me much time to figure out it wasn't.
GeneralRe: BravomemberPJ Arends17-Jun-05 6:43 
Stuart Konen wrote:
you've done an excellent job with this
 
Thanks StuartSmile | :)
 
Stuart Konen wrote:
One suggestion though
 
Point taken. I have updated the demo to hopefully remove the confusion.
 


"You're obviously a superstar." - Christian Graus about me - 12 Feb '03
 
"Obviously ???  You're definitely a superstar!!!" mYkel - 21 Jun '04
 
Within you lies the power for good - Use it!
Honoured as one of The Most Helpful Members of 2004

GeneralPut a little AI in it, ...memberWREY16-Jun-05 10:05 
... and it no longer remains a "Poor Man's Mouse Gesture."
 
Wink | ;)
 
William
 
Fortes in fide et opere!
GeneralRe: Put a little AI in it, ...memberPJ Arends16-Jun-05 11:19 
I know nothing about AI, so I am open to suggestionsBig Grin | :-D
 


"You're obviously a superstar." - Christian Graus about me - 12 Feb '03
 
"Obviously ???  You're definitely a superstar!!!" mYkel - 21 Jun '04
 
Within you lies the power for good - Use it!
Honoured as one of The Most Helpful Members of 2004

GeneralRe: Put a little AI in it, ...memberWREY16-Jun-05 11:27 
You tease.
 
Cool | :cool:
 
William
 
Fortes in fide et opere!
GeneralModifiedmemberPJ Arends15-Jun-05 9:34 
To any who have already dl'ed the code, I have made a small change. I renamed Create() to Attach(), and I added Detach().
 


"You're obviously a superstar." - Christian Graus about me - 12 Feb '03
 
"Obviously ???  You're definitely a superstar!!!" mYkel - 21 Jun '04
 
Within you lies the power for good - Use it!
Honoured as one of The Most Helpful Members of 2004

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Permalink | Advertise | Privacy | Mobile
Web02 | 2.6.130617.1 | Last Updated 15 Jun 2005
Article Copyright 2005 by PJ Arends
Everything else Copyright © CodeProject, 1999-2013
Terms of Use
Layout: fixed | fluid