Click here to Skip to main content
13,452,284 members (52,479 online)
Click here to Skip to main content
Add your own
alternative version


11 bookmarked
Posted 7 Apr 2004

A11Y and I18N Testing of Java based GUIs

, 21 Jun 2004
Rate this:
Please Sign up or sign in to vote.
In this article, I am going to discuss the A11Y and I18N testing of Java based GUIs


There are a lot of different approaches to software testing. One can use special verification tools based on Model Checking or Hoare Verification approach. It is also possible to use software that basically generates Unit tests or performs simple testing on the base of specification scripts, etc.

There is still one approach that is widely used – development of the test suites according to the software specifications.

I would not like to argue about applicability of the last approach because many companies still choose this way to test their software and there are plenty of developers who are writing tests right now!

That is why I decided to write this paper and share my experience in this area. The main theme here will be A11Y and I18N testing of Java based GUIs. I am not going to provide any source code examples, as the topic is quite simple.

Please note that in this article, I presume that the reader is aware about the following:

  • Software testing
  • GUI testing
  • A11Y
  • I18N
  • Unit testing


Anyone who is aware of Java based GUI testing knows about the following tools and libraries that are usually used in this process:

  • XTest
  • JUnit
  • Jemmy

The only problem with A11Y and I18N testing is that it is hardly compatible with Unit testing and the main tool for running tests so called XTest supported only the JUnit based tests. That is why it was a disaster to implement multi-step GUI tests on the base of Unit testing. I am not going to discuss all the troubles that one may meet on this way here but I will mention them below.

Problems of Java GUI Testing

First, let me give the brief description of what A11Y, I18N and Unit testing are:

  • A11Y testing is accessibility testing i.e. verification of the ability to access each and any GUI components without a mouse.
  • I18N testing is the Internationalization testing, i.e. testing of the correctness of application labels and messages internationalization.
  • Unit testing is an approach in which the atomic functionality of the system is tested. It is widely used in Extreme Programming and apart from that, in many other situations. For example, when you want to test a class by testing each of its methods.

Now it may become obvious that A11Y and I18N GUI testing are not compatible with Unit testing. The main reason for that is that any GUI testing is usually divided into the following steps:

  1. Perform application start up. Set required application options, open windows required for testing
  2. Perform test cases
    1. StartUp test case - common for all test cases
    2. StartUp test case – open some special windows or set test case specific options
    3. Perform test sequence
    4. TearDown test case – close specific windows and unset test case specific options
    5. TearDown test case – common for all test cases
  3. Shut down application

On the other hand, Unit testing (to be more precise JUnit and thus Jemmy) presumed that there can be only one common set up for the set of test cases and then all of them are performed one by one with the common shut down.

To make things plainer, note that on each of GUI testing steps, the failure can occur. This may interfere with performing the rest of the steps and thus dramatically influence the testing results.

This was a short and brief introduction to the problems that the GUI test developers may face. In the next section, I am going to describe the approach that I and my colleague, Alexey V. Ivashchenko, had developed for productive A11Y and I18N Java based GUI testing.

Solution to the Problems Stated Above

The solution is quite simple. You need to extend JUnit (Jemmy) so that it will be possible to create common StartUp and TearDown methods. The developer should also be able to add customized StartUp and TearDown methods.

You should also keep in mind that if you use Jemmy (it was unstable and it still may be unstable) you must perform all the SturtUp and TearDown operations exclusively by the API functions, i.e. do not use mouse or keyboard event emulations. This will allow you to avoid test crashes during the StartUp and TearDown operations.

Also an important note should be made for the A11Y testing. Any test should be as atomic as possible, for example, the forward focus cycle (Tab) should be tested in one test case and backwards focus cycle in (Shift-Tab) another. Also pressing of each button and accessibility of each component should be checked in separate atomic test cases. This will increase the tests run-time but will seriously decrease the support needed by the test suite and will also increase its stability. By support, I mean the necessity to modify test cases in case any A11Y bug is found.

These three simple rules will help you to increase stability and maintainability of the test suite.

In the following chapter, I will describe how to extend Jemmy to rapidly implement A11Y and I18N tests.

An Approach to I18N and A11Y Testing of Java Based GUI

The approach described here is incredibly simple but still very effective. I call it “Checkers” approach.

The base idea here is that there is a finite set of atomic checks (AtomicChecker later) that are performed in A11Y and I18N testing. For example, in the A11Y testing usually you have to check for focus on this or that component or for opening of a window after pressing some combination of keys. For I18N testing, it is even more obvious. A main check here is the check for value of the text label that is placed on this or that component (this includes combo boxes and windows’ titles).

So all you have to do is to create a simple AtomicCheck interface with almost one check method that will allow performing the atomic check of the component (here I mean Java Component, i.e. button for example) and then all you have to do is create a set of classes that will implement it. For example, let us consider we have a button on the form.

For A11Y testing, one might need to check that after pushing (Tab), this button will obtain the focus. So you implement the AtomicCheck interface mentioned above by the class ButtonA11YAtomicCheck in the following way. This class should have a constructor that receives the keys combination to be pressed and the ID of the button in the container (our form). The check method derived from the AtomicCheck interface first emulates this keys combination pushing, then searches for the button with the given ID in the obtained Java Container (our form), and then checks that the button has the focus.

For I18N testing, everything will be quite similar except that the constructor should receive the expected label value instead of the combination of keys and in the check method, the corresponding button’s label should be compared with the one provided in the constructor.

Now, if you use your imagination, you will understand that this idea together with the polymorphism of Java can provide you with a relatively small but flexibly extendable set of classes (library later) that will allow you to perform standard checks for A11Y and I18N tests.

To perform test case, you have only to create an instance of AtomicChecker implementer to perform an atomic test. You can also create a list of different instances that will be activated one by one and will perform a complex testing sequence in the scope of one test case (focus cycle check for example).

Of course this is only the general idea and a “power reader” understands that there are a lot of details and nuances that stayed aside of this article, but all of them can be simply and naturally solved in the scope of the proposed paradigm.

The main advantage of the described approach is that if you want to create a set of test cases, you should care only about the StartUp and TearDown methods' implementation. The testing sequence itself will only consist of instantiation of AtomicChecker component(s) with the proper arguments and starting them in the loop that of course should be also unified and carried out to the level of the Checkers library as some kind of performance manager.

Also note that it will be very easy to implement the Checkers library on the base of the API provided by Jemmy.


In this article, I have briefly covered the different approaches to software testing and mainly concentrated on the A11Y and I18N testing of Java GUIs. I displayed several problems that any developer may face during the tests creation and the ways to avoid these problems.

The main aim of this article is to introduce the new approach of Checkers that will allow anyone to speed up A11Y and I18N test cases development and increase the stability and maintainability of the test suite.



I would like to thank Alexey V. Ivashchenko who took part in the development of the Checkers approach.


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

Dr. Ivan S Zapreev
Netherlands Netherlands
PhD in Numerical and Statistical Model Checking of Probabilistic systems. Bachelor and Masters degrees (with honors) in Theoretical Mathematics. Thirteen (13) international research publications, among which nine (10) are DBLP recognized. Seventeen (17) years of work experience in Research, Design and Development of Verification Algorithms for Probabilistic and Control Systems, Scientific Software, Embedded Systems and Web applications. Excellent English (TOEFL-257-PC) and Good Dutch (NT2-2) skills. Permanent Dutch residence.

• Theoretical and applied research in Control and Formal Verification (numerical and statistical);
• Acquiring information, sharing knowledge, lecturing, mentoring, motivating and evaluating people;
• Working on large-scale distributed, multi-threaded, event-driven, cross-disciplinary software projects;

Research experiences:
• Numerical and Statistical Model Checking of Markov Chains;
• Type-2 Computable Topological semantics for CTL* on Dynamic Systems;
• Statistical Machine Translation;
• Deterministic and Symbolic-regression based compression of Symbolic controllers;
• Multi-dimensional trajectory planning with position and derivative constraints.

Teaching and supervision:
• 5 years of teaching at Novosibirsk State University, University of Twente, Fontys Hogescholen
• Supervising 3 master students at TU Twente and RWTH Aachen
• Supervising 11+ trainee-ship/internship students at Fontys Hogescholen
• Managing a group of 7 volunteers in the Russian school foundation
• Leading leading 3 project groups at Fontys Hogescholen

Software experiences (years):
• C++ (8), UML (6), Java (5), C (5), Matlab (4), C# (2), Python (1), Mathematica (1)
• GIT (4), SVN (5), Clearcase (4), Clear Quest (4), SCCS (2)
• CMake (3), Make (2), Ant (2)
• CSS (3), HTML (3), JavaScript (2)

You may also be interested in...


Comments and Discussions

GeneralAutomation Testing with Java GUI Controls Pin
Member 45967572-Dec-10 23:02
memberMember 45967572-Dec-10 23:02 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web03 | 2.8.180318.3 | Last Updated 22 Jun 2004
Article Copyright 2004 by Dr. Ivan S Zapreev
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid