Click here to Skip to main content
14,603,826 members

Mouse automation with AutoItX

Rate this:
5.00 (2 votes)
Please Sign up or sign in to vote.
5.00 (2 votes)
9 Feb 2020CPOL
Hello and welcome, In this post, we are going to talk about how we can automate part of the windows desktop environment using AutoItX. What is AutoItX? Before we can talk about AutoItX first we need to talk a little bit about AutoIt.

Hello and welcome,

In this post, we are going to talk about how we can automate part of the windows desktop environment using AutoItX.

What is AutoItX?

Before we can talk about AutoItX first we need to talk a little bit about AutoIt.

AutoIt as we see from the description on their page and their wiki is a scripting tool for desktop automation. I will no go into the details as it’s too much to cover and the wiki seems to be quite comprehensive, just know that it is the basis of what we are going to talk about. One thing I want to mention is that when I was working as a game tester, there was a very smart guy who automated out deployment system using AutoIt, so you can be amazed at what can be achieved with proper automation in the workplace.

Also if you need help with just AutoIt, they have a good forum for problem-solving issues, and of course the StackOverflow community.

Ok, I know what AutoIt is, what’s AutoItX?

Now that we got the AutoIt side of things cleared at a very rudimentary level, AutoItX is a C# wrapper over the AutoIt engine that comes as a nuget package with a .Net assembly and the native AutoIt dll inside. At the time of this writing, the latest version is 3.3.14.5 released on the 24th of January 2019 and has a total download number of 396,649 which from my perspective it means it is somewhat battle-proven to be stable.

Something to point out is that even though it’s a wrapper over the engine, the wrapper does not feature the full functionalities that the native AutoIt has like setting up hotkeys and such, but this is something we will look at in the future. But if you want to use features not available in C# but still want to use C#, an idea would be to write the script in AutoIt, compile it, and then just launch it from C#; Of course, the drawback from this is the maintenance cost of debugging.

How to install AutoItX

Since AutoItX comes as nuget package, adding it to your project is easy enough, just search for it and add it, but there’s a catch because it’s using the native dll, we also need to have either the AutoItX3.dll (for 32-bit applications) or the AutoItX3_x64.dll (for the 64-bit applications) present in our output folder so keep that in mind because it will come up in the demo we’re about to do.

Demo: Automating cookie clicker

For this demo we will be using the following:

  • LinqPad 6 – I prefer (as mentioned in previous posts) to use LinqPad as a scratch pad, to prototype functionalities in it. I also think (though I didn’t confirm) that it would only work smoothly with LinqPad 6 version as this version allows you to include files in the output without a lot of plumbing code to get the native assemblies to copy to the output folder. Another thing to note is that nuget is a paid feature for LinqPad. If you don’t have a paid version, then you need to manually download the nuget package, unpack it and then reference the assemblies and dlls, or use Visual Studio.
  • Cookie Clicker – This will be our gateway into playing around with automation
  • The AutoItX nuget package – from which we will also reference in LinqPad 6 the native dll for it to work, this shouldn’t be a problem for Visual Studio since the package contains a .targets file for build, though I admit, I have yet to use it in Visual Studio, but if any issues occur then the same approach as for LinqPad would work.

Step 1: Referencing the assemblies and libraries

Open LinqPad 6 and search for the nuget package AutoItX for .NET.

After downloading the nuget package and adding the namespace for it, click on the Add / Browse… button from the Query Properties window (should be already there to add the package if not the default key to open this window is F4).

Navigate to where the nuget package was downloaded at, default it’s at “%HOMEPATH%.nuget\packages\autoitx.dotnet\3.3.14.5“, and from the build folder, select the native dll to be used, I will use the AutoItX3_x64.dll.

Step 2: Cookie Clicker

The reason I chose Cookie Clicker is that it’s an easy and fun way of show-casting how you can spam click from code and get something in return (cookies, yey!!!).

Of course, you can use something else to try this out, though I recommend something that can be spam clicked for quire some time and can also give some kind of feedback (not on the empty desktop) for this example. You will see why in a minute.

Step 3: Writing the code

And now for the fun stuff.

AutoItX uses a set of static methods to perform actions, for example, the following code snipped will mouse click (default left/main click) at the current mouse position.

AutoItX.MouseClick();

This method has optional parameters specifying which mouse button to click, the coordinates at which to click on the screen, the number of clicks and the speed, all of these should be in line with the AutoIt documentation if you want to see what each parameter does.

Because we want to keep on clicking in a loop, it’s important to also be able to stop it. This is the reason we will not be giving coordinates to the MouseClick method (because it will prevent us from moving the mouse) and why the clicking loop needs to stop so that even if we do bring up the task manager to kill the process, it doesn’t continue to click on random things at those coordinates.

To accomplish this we will be using a command that returns the current coordinates of the mouse, that command is as follows:

Point startPosition = AutoItX.MouseGetPos();

The coordinates are returned in the format of a System.Drawing.Point containing the current X and Y coordinates of the mouse.

Now let’s write our code:

Point startPosition = AutoItX.MouseGetPos();
while (startPosition == AutoItX.MouseGetPos())
{
	AutoItX.MouseClick();
}

With this, we then focus LinqPad, move the mouse over the giant cookie in the browser (without clicking in the browser) and then press F5 to run the script. This will start generating all those sweet sweet cookies as long as we don’t move the mouse, and as soon as we move the mouse, the script will end as to not have issues with the mouse continuing to click while we move it to close the script.

Fun homework

Now for some fun homework, try to make the script to keep running only while your mouse is still hovering over the giant cookie. Some hints for this would be to find the top and bottom Y value for the cookie, then find the left and right X values for the cookie, find the center then put a condition in the while loop to check that the current coordinates of the mouse are inside the area of the cookie.

Another homework idea would be to find the cookie no matter where the browser is located at (assuming it’s not maximized or on the same screen) by looking for other helpful methods that AutoItX provides.

Conclusion

I know it’s a silly example but I do believe that is his a fun way to play around with mouse automation for starters and a good starting point to research into other automation that can be done using AutoItX and C#.

I hope you enjoyed it and we might come back to this topic with some other fun additions to our scrips, like maybe adding hotkeys to turn on and off our script 😉

Until next time,

Vlad V.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Vlad Neculai Vizitiu
Software Developer
Romania Romania
When asked, I always see myself as a .Net Developer because of my affinity for the Microsoft platform, though I do pride myself by constantly learning new languages, paradigms, methodologies, and topics. I try to learn as much as I can from a wide breadth of topics from automation to mobile platforms, from gaming technologies to application security.

If there is one thing I wish to impart, that that is this "Always respect your craft, your tests and your QA"

Comments and Discussions

 
GeneralMy vote of 5 Pin
LightTempler10-Feb-20 7:21
MemberLightTempler10-Feb-20 7:21 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Technical Blog
Posted 9 Feb 2020

Stats

2.5K views
3 bookmarked