WPF UI Automation

• Download source code - 19.61 KB

Introduction

Overview

Microsoft UI Automation is the new Accessibility framework for Microsoft Windows, available on all Operating Systems that support the Windows Presentation Foundation (WPF). UI Automation provides programmatic access to most user interface (UI) elements on the desktop.

This article demonstrates how to use UI automation to control a simple application. Basically, the controller application will search for a test application which is running, and if the application is found, we will search through the UI elements hierarchy, and we will send dummy data.

Applies to:

• Microsoft® .NET Framework 3.0
• Microsoft® Windows Presentation Foundation - WPF
• Microsoft® UI Automation Framework

UI Automation Framework

The main idea behind this framework is that it provides access to the hierarchy of user interface elements which are visible in Win applications. Imagine the root element of the hierarchy as being the desktop, and all the running applications are the first level nodes (children) of the root, and the same for each visible form which can be detailed to its controls, the controls to its children, and so on. Each node in this tree is called UI element.

The framework helps us by giving access to these UI elements and their properties. The main features supported by this framework are:

• Search support, that lets you find the required user interface component.
• Filtering the tree structure of elements; for example when querying the hierarchy of elements, we can get only the enabled controls.
• Lets you interact with UI elements; for example, you may programmatically click a button(which is contained in a different application), from your application.
• Subscribing to events; this helps us to monitor UI elements and handle external events.

UI Automation API

System.Windows.Automation is the main namespace of the UI Automation framework, and provides support for interacting with UI elements, also called AutomationElements. AutomationElement is the main class of the framework, and corresponds to a user interface component (window, list, button,...). The main features of this namespace are related to searching for AutomationElement objects, interacting with different control patterns or subscribing for UI automation events.

AutomationElement contains a property called RootElement, which contains a reference to the correspondent UI element of the desktop, which in fact returns an AutomationElement.

As for the search support, AutomationElement class contains FindFirst() and FindAll() methods:

public AutomationElement FindFirst(TreeScope scope, Condition condition)

We can use the FindFirst method to search for a specific AutomationElement which matches the criteria passed. In order to call this method, we need to pass the scope of search (instance of TreeScope enum), for example to search through the first level of children, or to search through all descendants. Besides scope parameter, we have to pass the condition that should be met in order to identify the required AutomationElement. We will pass the condition as an instance of a class that extends Condition abstract class. In this way, we can use PropertyCondition (for testing a property to have a specific value), but also available are: AndCondition, NotCondition or OrCondition classes.

public AutomationElementCollection FindAll(TreeScope scope, Condition condition);

Similar to FindFirst, FindAll() let us query the UI Automation tree using the parameters described above, but return a collection of AutomationElements that have passed the search condition.

Based on the above details, we can write a simple line of code, which will search through all the descendants of the root element, for a UI element that has the name equals to 'Notepad':

AutomationElement appElement = AutomationElement.RootElement.FindFirst
(TreeScope.Descendants,new PropertyCondition(AutomationElement.NameProperty, "Notepad"))

The Implementation

In order to use the UI Automation API, we need to add references to these assemblies:

• UIAutomationClient.dll
• UIAutomationTypes.dll

For this simple implementation, I used the base class AutomationElement, which, as I said earlier, keeps the reference to a UI element from desktop applications. The idea is that we need to start from the desktop element, which is the root element, AutomationElement.RootElement, and we will search through all the child objects of RootElement for a test application having the title: "UI Automation Test Window". After getting a valid reference to the AutomationElement of the test application, we can then interact with the different contained controls. In this way, the controller application sets some values to two TextBox controls.

AutomationElement rootElement = AutomationElement.RootElement;
if (rootElement != null)
{
    Automation.Condition condition = new PropertyCondition
        (AutomationElement.NameProperty, "UI Automation Test Window");

    AutomationElement appElement = rootElement.FindFirst(TreeScope.Children, condition);

    if (appElement != null)
    {
        AutomationElement txtElementA = GetTextElement(appElement, "txtA");
        if (txtElementA != null)
        {
            ValuePattern valuePatternA = 
              txtElementA.GetCurrentPattern(ValuePattern.Pattern) as ValuePattern;
            valuePatternA.SetValue("10");
        }

        AutomationElement txtElementB = GetTextElement(appElement, "txtB");
        if (txtElementA != null)
        {
            ValuePattern valuePatternB = 
              txtElementB.GetCurrentPattern(ValuePattern.Pattern) as ValuePattern;
            valuePatternB.SetValue("5");
        }
    }

Here is the GetTextElement function:

private AutomationElement GetTextElement(AutomationElement parentElement, string value)
{
    Automation.Condition condition = 
      new PropertyCondition(AutomationElement.AutomationIdProperty, value);
    AutomationElement txtElement = parentElement.FindFirst
                (TreeScope.Descendants, condition);
    return txtElement;
}

As you can see in the above code, we are using a control pattern, ValuePattern. UIA (UI Automation) uses control patterns to represent common controls. Control patterns define the specific functionality that is available in a control by providing methods, events, and properties. The methods declared in a control pattern allow the UIA clients to manipulate the control, for example, the ValuePattern.SetValue() method. Besides ValuePattern which represents a control that stores a string value, as a different example of control pattern, we can take the Invoke pattern control, InvokePattern, which represents the controls that are invokable, controls like buttons. In order to use a control pattern, we need to query the object to see what interfaces are supported, and only after getting a valid control pattern can we interact with it by using its methods, events, and properties. The following list shows the most common control patterns:

• SelectionPattern - used for controls that support selection, controls like ListBox, ComboBox
• TextPattern - used for edit controls
• ValuePattern - used to get/set a value on a control that does not support multiple values
• InvokePattern - used for invokable controls
• ScrollPattern - used for controls that have scrollbars
• RangeValuePattern - used for controls that have a range of values

The following example shows how to use a InvokePattern; in other words, it will click a button contained in parentElement:

{
    Automation.Condition condition = new PropertyCondition
            (AutomationElement.AutomationIdProperty, "Button1");
    AutomationElement btnElement = parentElement.FindFirst
            (TreeScope.Descendants, condition);
    
    InvokePattern btnPattern = 
    btnElement.GetCurrentPattern(InvokePattern.Pattern) as InvokePattern;
    btnPattern.Invoke();
}

Conclusion

UI Automation is a powerful Accessibility framework which lets you control other applications. It can be used in the development of automated UI testing/debugging apps and so on.

There is more to come on this subject.

History

• 4^th February, 2009: Initial post
• 5^th February, 2009: Article updated
• 19^th February, 2009: Article text updated