UITestBench, a lightweight UI testing library

• Download UITestBench - 204.43 KB

Introduction

This article describes UITestBench, a small but efficient library to implement User-Interface tests that can be run with NUnit or any other unit test framework. The source code provided with this project, a VisualStudio 2005 solution, is split into three parts/projects:

• A small sample application
• the UITestBench classes
• two NUnit test cases for the demo application using the UITestBench

Requirements

The following requirements/assumptions were considered during development of the UITestBench:

• The application under test (AUT) is written in pure .NET
• The AUT does not need to be developed with UI testing in mind
• The AUT need not be modified for implementing/running the tests
• The AUT shall not be dependent on any test classes (so also a release version can be tested)
• The UITestBench library shall be independent of the application under test and the UnitTEst framework

The following UML diagram shows the dependencies between the different parts of the project:


It can be seen that the requirements are met as the diagram shows the AUT is not dependent on any test classes or other packages and the UITestBench is independent of the NUnit framework and the AUT.

Tasks

In order to perform a UI test the following tasks need to be implemented:

• Launching the AUT from within a test case
• Scanning the available UI elements of the application (buttons, menue items, lists, etc...)
• Performing actions on these elements
• Ensure the application is shut down when the test case ends

The first and the last step can either be performed for every test case or once for a number of test cases.

The followng UML sequence diagram shows how a minimal case may look like:

Launching the application under test

Applications using Windows Forms have to be run in an STA thread appartment (for more details see http://blogs.msdn.com/jfoscoding/archive/2005/04/07/406341.aspx).
Usually this is done by applying the STAThreadAttribute to the applications Main method:

[STAThread]
static void Main() { 
    //Start the WinForms application...
    ...
}

However, when invoking the application from within a unit test, one cannot be sure what the actual thread appartment state is. But this is no problem. As we need to create a new thread for running the application anyway (the original test thread is used for running the commands (the test case) against the application in the new thread), we simply have to set up this thread as an STA thread. This is done as follows:

public void StartApplication(string assemblyName, object args)
{
  Assembly assembly = Assembly.Load(assemblyName);

            if (assembly != null)
            {
                //Invoke the application under test in a new STA type thread
                uiThread = new Thread(new ParameterizedThreadStart(this.Execute));
                uiThread.TrySetApartmentState(ApartmentState.STA);
                uiThread.Start(new ApplicationStartInfo(assembly, args));
            }
            else
            {
                throw new Exception("Assembly '" + assemblyName +"' not found!");
            }
}

In the constructor of the thread a new delegate as entry point for the thread is passed. The Execute method that is passed as delegate method invokes the application's main-method by using the EntryPoint property of the Assembly-Type object that is passed as parameter. Actually the object passed as parameter is a helper class ApplicationStartInfo which contains the assembly object to use as well as an argument object (e.g. as string[]) that is passed to the applications main-method.

private void Execute(object param)
{
    Console.WriteLine("UIExecute ThreadId: " + Thread.CurrentThread.ManagedThreadId);

    Application.ThreadException += new ThreadExceptionEventHandler(Application_ThreadException);

    ApplicationStartInfo startInfo = (ApplicationStartInfo)param;

    Assembly ass = startInfo.AssemblyToStart;
    if (ass.EntryPoint.GetParameters().Length == 1)
    {
        ass.EntryPoint.Invoke(null, new object[] { startInfo.Arguments });
    }
    else
    {
        ass.EntryPoint.Invoke(null, new object[] { });
    }
}
 

So launching the application from within the test fixture's SetUp method is as easy as this:

[SetUp]
public void SetUp()
{
    myTestBench = new UITestBench();
    myTestBench.StartApplication("FlexRayReplayScriptGenerator", new string[] { });

    //Some time to start the demo app
    Thread.Sleep(2000);
    ///Here the framework could be extended to wait for a certain dialog title to appear instead of a fixed delay...
}

Scanning the available UI elements

Once the application is started and the first UI action can be invoked by the unit test (see next section), the open forms of the application have to be scanned for available UI controls.
So how to get access to the open forms of the application? This is much easier than expected. One can simply use the static OpenForms property of the Application class:

  foreach (Form openForm in Application.OpenForms)
 {
                //Let the name be the form id
                string formId = formToScan.GetType().Name;
                
                ScanUIElementsOfForm(openForm, formId);
  }

The method ScanUIElementsOfForm now does the following steps:

1. Creates a dictionary that will contain the scanned elements accessible via a unique id,
2. calls ScanUIElementsOfControl to recursively scan the elements of the form (which also is of type Control),
3. and replaces the old elements of the form (if scanned before) with the newly scanned elements

private void ScanUIElementsOfForm(Form formToScan, string formId)
{
    IDictionary newlyScannedElements = new Dictionary();

    //Get all supported UI elements of the form
    ScanUIElementsOfControl(formToScan, newlyScannedElements, formToScan.GetType().Name, formToScan.GetType().Name);

    //Set the owner of the elements to the scanned form
    foreach (string key in newlyScannedElements.Keys)
    {
        newlyScannedElements[key].OwningForm = formToScan;
        Console.WriteLine(key);
    }

    //Remove existing form info 
    if (uiForms.ContainsKey(formId))
    {
        uiForms.Remove(formId);
    }

    //And replace with new form info about the newly scanned elements
    uiForms.Add(formId, new UIFormInfo(formToScan.Text, newlyScannedElements));
}

For each scanned element an UIElementInfo object is created which contains a WeakReference to the scanned element and to the owning form. The latter is required for being able to invoke actions on the element. For each scanned form a UIFormInfo object is created which contains the UIElementInfo objects of the form. The following class diagram shows this structure:

The .NET class WeakReference is used to store the references to the scanned forms and elements as otherwise these would be prevented from being collected by the .NET garbage collector even after the dialogs have been closed. A normal, strong reference, e.g. by assigning an object to a variable, prevents the referenced object from being collected. However, when there are no other strong references to the object, the target becomes eligible for garbage collection even though it still has a weak reference.
So, by using weak references the UI test does not interfere with the natural way the applications garbage collection would behave.

The UIElementInfo class allows access to the WeakReferences only via the corresponding Properties which include a check if the referenced object is still alive. If this is not the case an exception is thrown and the test case will fail.

The method ScanUIElementsOfControl used in the above method recursively adds all available child controls of the passed control to the dictionary. Therefore (and of course for being able to build a test case) each element must have a unique id. This unique id (unique within a form) is constructed using the following rules:

• If the AccessibleName property of a control is a string longer than 0 it is used as key
• otherwise the Text property is used if it is not null and longer than 0

If this key is already used by another control (e.g. if it has the same AccessibleName) the complete path to the control (via all parent controls) is taken as key. This is always unique as an index is assigned to each control. For better understanding, the path name of a control is constructed of this index and the type name of the control.

private void ScanUIElementsOfControl(Control control, IDictionary uiElements, string path, string parent)
{
    int itemIdx = 0;
    foreach (Control childControl in control.Controls)
    {
        string myPath = path + "/" + childControl.GetType().Name + "[" + itemIdx + "]";
        string key = myPath;
        //Use the parent name + accessible name as key if possible
        if (childControl.AccessibleName != null && childControl.AccessibleName.Length > 0)
        {
            key = parent + "/" + childControl.AccessibleName;
        }
        else if (childControl.Text != null && childControl.Text.Length > 0)
        {
            //Else use the parent name and the controls text as key
            key = parent + "/" + childControl.Text;
        }

        //Use the shorter key if not yet used
        if (!uiElements.ContainsKey(key))
        {
            uiElements.Add(key, new UIElementInfo(childControl, key));
        }
        else
        {
            //Else use the unique path to the element
            uiElements.Add(myPath, new UIElementInfo(childControl, myPath));
        }

        ScanUIElementsOfControl(childControl, uiElements, myPath, childControl.GetType().Name);
        itemIdx++;
    }

    //It might be a ToolStrip
    ToolStrip strip = control as ToolStrip;
    if (strip != null)
    {
        ScanToolStripItems(strip.Items, uiElements, path, strip.Text);
    }
}
 

Finally it is checked if the given control is a ToolStrip as the foreach loop does not work for ToolStrips because for these the Controls property always returns an empty collection. As the ScanToolStripItems method works similar it is not elaborated in more detail at his point.

For being able to construct the test cases easily the keys of the scanned elements are written to the console. So when using the NUnit GUI one can easily see the keys in the "Console.Out" tab. For the demo application the elements of the main form are identified using the following keys:

Form1/SplitContainer[0]
Form1/SplitContainer[0]/SplitterPanel[0]
Form1/SplitContainer[0]/SplitterPanel[0]/ListBox[0]
Form1/SplitContainer[0]/SplitterPanel[1]
Form1/SplitContainer[0]/SplitterPanel[1]/ListBox[0]
Form1/toolStrip1
toolStrip1/Merge
Form1/menuStrip1
menuStrip1/File
File/Open text file 1...
File/Open text file 2...
File/
File/Exit

As the search for UI elements needs to be performed each time a new form is opened the algorithm can either be called manually from within the test case or it is called when the access of a UI element fails, that is the UI element id is not found. If the element is still not found after the rescan an exception is thrown and the test case fails.

Performing UI actions

For different types of UI Elements different actions can be performed. The UITestBench offers several conveniance methods for accessing the most frequently occuring actions, e.g. clicking a button or setting a text in a text box. Of course, it is easy to add more conveniance methods if some action is needed frequently in your application. The following methods are provided:

• public void PerformClick(String formKey, String itemKey)
• public void SetText(String formKey, String itemKey, string text)
• public void SetSelectedIndex(String formKey, String itemKey, int index)

For performing an action the id of the form and the unique id of the UI element within the form have to be provided. Depending on the desired action additional parameters (e.g. the text to be set) have to be passed. Internally these methods use a delegate to invoke the action on the related UI element as under some circumstances (dependent on how the application is started) an "Illegal Cross Thread Operation" exception is thrown when a control is called from a different thread than it was created on. Just google for "illegal cross thread c#" to get more info about this problem.

Dealing with external dialogs

One last problem when implementing a UI unit test is the occurrence of external, that is not user implemented or even not .NET based dialogs. An typical example for such a dialog is the "Open File" dialog, which is used by the demo application, too. An easy way to deal with such dialogs, as long as they are not too complex, is to send keystrokes to the dialog by using the SendKeys class. So to open the file dialog, select a file and open it only the following steps are required for the demo application, which uses the standard .NET file dialog:

myTestBench.PerformClick("Form1", "File/Open text file 1...");
SendKeys.SendWait("file1.txt");
SendKeys.SendWait("{ENTER}");

Finishing the test

When a test case is finished NUnit calls the TearDown method. This is now used to call the TerminateApplication method of the test bench which ensures that the started application is closed.

        [TearDown]
        public void TearDown()
        {
            myTestBench.TerminateApplication();
        }




public void TerminateApplication()
        {
            Console.Write("Forcing application to shut down if it has not terminated already....");
            if (uiThread != null && uiThread.IsAlive)
            {
                Console.WriteLine("Done!");
                uiThread.Abort();
            }
            else
            {
                Console.WriteLine("not required!");
            }
        }

Limitations

Currently it is not possible to have two open forms of the same type as the type name is used as unique key. This could be changed by simply appending an index to duplicate types. However this is not required for this demo project or simple applications.

Using the Code

The download contains a single Visual Studio 2005 Solution which is split up into three different projects: The demo application, the test bench framework and the test case implementation. After opening the solution with Visual Studio, the demo application is set as start project, i.e. it can be started by simply pressing F5.
To execute the test cases you have to open the dll "UITestDemoApp_UITest.dll" which is contained in the folder "UITestDemoApp_UITest\bin\Debug" with NUnit GUI.

Conclusion

This project has shown how it is possible to build a simple but pretty useful UI testing framework based on the standard .NET framework. The major points of interest hereby are the access to the open forms using the Application.OpenForms property, the execution of the application under test in a separate STA type thread, the collection of the UI elements using weak references as well as the invocation of the UI elements using delegates to prevent an invalid cross thread operation to occur. Based on this basic framework it is easy to implement additional UI testing functionality to tailor this project to specific needs. This is perfect for testing small to medium .NET applications without having to buy expensive and/or complex external solutions.

History

[03.04.2008] - 1.0 - Initial release.