Click here to Skip to main content
15,868,141 members
Articles / Desktop Programming / MFC
Article

Unit Test's Reporter

Rate me:
Please Sign up or sign in to vote.
4.86/5 (28 votes)
19 Feb 2006CPOL11 min read 64.5K   2.8K   58   8
The article describes an application built to visualise testing processes for the unit test framework of CppUnitLite.

Image 1

Introduction

During the last two years, I had been writing software in C#. Just during that time, I started using NUnit and CSUnit as tools for testing my programs. Later on, I had to return to traditional C/C++. I didn't want to give up the methods I was used to for so long, like the use of TDD (Test Driven Development) has proved to be quite satisfying in my everyday professional experience. After a few trials, I discovered a product named CppUnitLite the simplicity and compactness of which attracted me a lot.

In my opinion, all the open-source software products of this class can be divided into two major groups. The first one includes the so-called frameworks that do not have additional visualization tools. Here, one is able to build a test or a group of them by using special classes and macros and then test the module encapsulated in a console application. Messages generated by the framework are then directed to a standard output device stdout. If Microsoft Visual Studio IDE is available, the messages can also be viewed in the so-called output window. However, if the application is run separately from the working environment, the messages go to a console. A list, perhaps incomplete, of such frameworks is given below:

The second group includes frameworks equipped with a special program, test runner, designed for running the tests and displaying the test results. In this case, the tests are grouped into dynamic libraries. The test runner loads a library built, runs the tests, and shows the test results in a tree-like way. CppUnit belongs to this group. A similar product has recently been added to your site: TILO.

Both the approaches, outlined above, have their advantages and disadvantages. I don't want to discuss them at length. Note that the frameworks belonging to the first group are much easier to use even though they do have a serious disadvantage – lack of visualization tools.

I decided to fill this gap by solving the shortage in the following way: The console application functions in the desired way, but the test results are sent to both a standard output and a special application that catches the messages and further displays them as a tree. The standard framework acquires special functions that form and subsequently send these packets of messages. From the point of view of a user, the process of software development stays intact. A software engineer just gets an additional tool allowing him/her to evaluate the test results at a glance. I hope that the project attached to this article will make the life of programmers a bit easier!

A brief description of the program

The application described here is named Unit Test's Reporter. It is represented in Fig. 1.

The left panel shows a tree of tests. There are three types of nodes:

  • Application
  • Test
  • Macro

The right panel displays the information related to the node chosen. In case, the node chosen is either an application or a test, a statistical information is displayed.

Image 2

Fig. 1: This is how the application looks like. The information panel contains the data of a testing session.

Image 3

Fig. 2. Test results are displayed in the information panel.

If an error occurs, the information about its location and causes can be retrieved as detailed in Fig. 3:

Image 4

Fig. 3. Detailed information about an error.

There are four additional options in the program:

  • Clear the report.
  • Keep the tree of tests as an XML file.
  • Recover the tests tree from an XML file.
  • Print a report on the testing session.

These options are available on the menu File and also supported by the buttons of a toolbar.

Changes introduced in CppUnitLite

A canonical text of a packet located here was used. The changes and additions made to it are listed below:

  1. The macros added are: ASSERT_TRUE, ASSERT_FALSE, TEST_FUNCTION and USE. The macro ASSERT_TRUE is a synonym of the macro CHECK. Macros TEST_FUNCTION and USE allow you to determine the inner functions of the test. All the macros of CppUnitLite can be used in those functions. An example is given below:
    TEST_FUNCTION(SimpleFunc)
    {
       int a = 8;
       CHECK_EQUAL (3,a);
       long b = 777677;
       LONGS_EQUAL (777677,b);
       DOUBLES_EQUAL (1.88587,1.9,0.01);
    }
    
    TEST(Test,Function)
    {
       USE(SimpleFunc);
    }
  2. The order of the tests was changed: from the first to the last, and not vice versa.
  3. A flag continue_if_error_ was introduced in the Test (ITest) to allow running the test in case of an error. Similar changes were done in other macros.
  4. The variables, keeping the statistical information, were added to the class Test (ITest).

Interaction with the Unit Test's Reporter

The interaction with the program Unit Test's Reporter is done by means of the following functions:

  • TVTStartConnection: This function is called before running the tests, in the constructor of the object TestRegistry.
  • TVTFinalConnection: This function is called after completing the tests, in the destructor of the object TestRegistry.
  • TVTSendTestInfo: This function is called when adding a test to a list of tests in the object TestRegistry.
  • TVTSendTestSuccess: This function is called if a test was successfully completed in the macro TEST.
  • TVTSendTestError: This function is called if a test has failed in the macro TEST.
  • TVTSendTestFailure: This function is used in the function AddFailure of the object TestResult.

Transfer of messages

The information from the framework CppUnitLite is transferred to the program Unit Test's Reporter by means of the message WM_COPYDATA prepared in XML format. The use of XML allows flexible way of data transfer of messages of any complexity and structure. The messages sent are divided into two groups. The first group allows you to add objects to a tree of tests. The second group allows you to make changes in the objects.

The message below corresponds to the session start:

XML
<easyTest version="0.9" copyright="EasyTest">
  <Action command="insertObject"></Action>
</easyTest>
<easyTest version="0.9" copyright="EasyTest"> 
  <Action command="insertObject"> 
    <Attributes object="session" session="TestApp"/> 
  </Action> 
</easyTest>

Adding a test is accompanied by sending the message:

XML
<easyTest version="0.9" copyright="EasyTest">
  <Action command="insertObject">
    <Attributes object="test" session="TestApp" 
                test="TestAssertFalse" group="Sample"/>
  </Action>
</easyTest>

If an error is detected, the following message is sent:

XML
<easyTest version="0.9" copyright="EasyTest">
  <Action command="insertObject">
    <Attributes object="macro" session="TestApp" 
                test="TestAssertTrue" group="Sample" 
                macro="ASSERT_TRUE" line="16"/>
  </Action>
</easyTest>

Once an error is detected, the information about its character and location should be sent:

XML
<easyTest version="0.9" copyright="EasyTest">
  <Action command="updateObject">
    <Attributes object="macro" session="TestApp" 
                   test="TestAssertTrue" group="Sample" 
                   macro="ASSERT_TRUE" line="16" 
                   result="Error"/>
      <Entity>
        <Line number="1">
          <Token text="Expected condition is "/>
          <Token text="true"/>
          <Token text=","/>
        </Line>
        <Line number="2">
          <Token text="detected condition"/>
        </Line>
        <Line number="3">
          <Token text="«(string(typeid(expectedInt).name()) == ´integer´)»"/>
        </Line>
        <Line number="4">
          <Token text="is "/>
          <Token text="false"/>
          <Token text="."/>
        </Line>
        <Line number="5">
          <Token text="Error is fixed in line "/>
          <Token text="16"/>
        </Line>
        <Line number="6">
          <Token text="in file "/>
          <Token text="d:\currentwork\utestreporter\sample\sample.cpp"/>
          <Token text="."/>
        </Line>
      </Entity>
  </Action>
</easyTest>

Once the test is completed, a message is sent containing the information about the test result and some statistical data:

XML
<easyTest version="0.9" copyright="EasyTest">
  <Action command="updateObject">
  <Attributes object="test" session="TestApp" 
                 test="TestLongsEqual" group="Sample" 
                 result="Success"/>
    <Entity>
      <Line number="1">
        <Token text="Result: "/>
        <Token text="Test is executed successfully."/>
      </Line>
      <Line number="2">
        <Token text="Conclusion: "/>
        <Token text="1"/>
        <Token text=" check was made."/>
      </Line>
      <Line number="3">
        <Token text="Statistics:"/>
      </Line>
      <Line number="4">
        <Token text="Macro LONGS_EQUAL was used "/>
        <Token text="1"/>
        <Token text=" times."/>
      </Line>
    </Entity>
  </Action>
</easyTest>

Similar information is transferred after completion of the test application:

XML
<easyTest version="0.9" copyright="EasyTest">
  <Action command="updateObject">
  <Attributes object="session" session="TestApp"/>
    <Entity>
      <Line number="1">
        <Token text="Conclusion: "/>
        <Token text="40"/>
        <Token text=" checks were made. There were "/>
        <Token text="34"/>
        <Token text=" failures."/>
      </Line>
      <Line number="2">
        <Token text="Statistics:"/>
      </Line>
      <Line number="3">
        <Token text="6"/>
        <Token text=" tests are executed."/>
      </Line>
      <Line number="4">
        <Token text="3"/>
        <Token text=" tests are completed successfully. "/>
      </Line>
      <Line number="5">
        <Token text="3"/>
        <Token text=" tests failed."/>
      </Line>
    </Entity>
  </Action>
</easyTest>

Templates of messages represent some XML files. These can be found in the folder XMLTemplates of the project attached to this article.

A brief description of the program Unit Test's Reporter

The functioning principle of the program Unit Test's Reporter is transparent and simple. It accepts messages in XML format sent by the application being tested and further translates them into commands for building and refreshing a tree of tests.

A tree of tests and GUI components

A tree of tests is the object composite with the interface IComponent (see Composite/Composite.h). The test is described in the class TestObject (see Objects/TestObject.h).

It was one of my goals to separate a tree of tests represented by the object IComponent from its graphical representation in Windows. This separation allows me to solve the following two problems:

  1. The first one is to build a program and completely test it by means of unit tests. Putting it differently, one may project a GUI application without using the GUI components specific to a concrete operating system. This is easy to do as the behaviour of CTreeCtrl can be modeled by an object similar to composite.
  2. The second one is to ease the procedure of porting the application into an environment different from Windows, if such a need does exist.

Let me stress that such a solution is very useful. I have no doubts that any GUI application can be built and tested by means of unit tests. The only point is the price to be paid for it.

There is another point to highlight. In principle, the application Unit Test's Reporter can be rebuilt in such a way that it will support any framework without visualization tools. In that case, one would not be able to avoid the virtualization of a tree of tests.

Keeping a tree of tests

A test is kept as an XML file with the extension SXML. This file consists of two sections. The section Entity keeps a tree of tests. The nodes of a tree contain the links to the objects application, test, and macro. These objects are kept in the section Data.

Stages of building the program

Description of a tree of tests

Creation of the program starts with a description of the interfaces. Since a tree of tests is the heart of the project, an interface describing the tree was created: IComponent (see Composite/Composite.h). Access to specific elements of a tree as well as ways of building and modifying a tree are all realized using the objects: Leaf and Composite. Correctness of the algorithms was verified with the help of mock-objects: MockLeaf and MockComposite, see Mocks/MockComposite.h.

An element situated in the node of a tree of tests is described by the class TestObject. The attributes of this class make it possible to describe the objects application, test and macro, as well as the various states of these objects. Creation and modification of the GUI object reflecting a tree must go in parallel with the creation and modification of a tree of tests. For Windows, the object is CTreeCtrl.

This object is created indirectly by means of the element environment_ with the interface IEnvironment<T>*. It is precisely this object that adds a link to the corresponding object of a parallel tree to the node of a tree of tests: void * linkedObject_. A parallel tree gets access to the object TestObject in order to read and represent its attributes in a graphical form.

An object with the interface IEnvironmentObjectCreator<T>, belonging to IEnvironmentObject<T>, is responsible for the creation of a graphic image (GUI) of the tree. In fact, this is an adapter of GUI objects. Due to this division of roles, it is absolutely unimportant what a GUI object will reflect as a tree of tests: a tree, a list, a hybrid of a tree and a list, or whatever else.

This concept also allows us to verify the functioning of a future application by means of the mock-objects inheriting the interfaces IEnvironmentObject<T> and IEnvironmentObjectCreator<T> (see the files Mocks/MockEnvironmentObject.h, Mocks/MockEnvironmentObjectExt.h, Mocks/MockEnvironmentObjectCreator.h, Mocks/MockEnvironmentObjectCreatorExt.h).

Any Windows application makes use of the objects WinEnvironment, WinEnvironmentObject and WinEnvironmentObjectCreator together with the interfaces given above (see the files Common/WinEnvironment.h, Common/WinEnvironmentObject.h and Common/WinEnvironmentObjectCreator.h).

Serialization of a tree of tests

Serialization of a tree of tests is performed with the help of two objects. The interface of the first object is IExternalObjectStorage<T>. It is intended for keeping and loading the object TestObject. The interface IExternalStorage<T> is responsible for the serialization of the tree structure. The classes XMLObjectStorage and XMLStorage which realize the above interfaces serialize the data into XML files. If needed, the object can be designed to serialize data into text files, databases etc. Since the interfaces stay intact, this will not affect the functioning of the program.

Printing a tree of tests

The process of printing a tree makes use of the virtual method Action which is present in the interface IComponent<T>. This method allows one to do a recursive exploration of the tree. In doing so, a hierarchical object Text is built; this object is further sent for printout. The method Action acts indirectly, via the object with the bridge interface IActionHelper. This object can be simulated in order to perform any other action related to the tree exploration.

Description of the solution attached to the article

The solution accompanying the article consists of three projects:

  1. EasyTest: This project contains the interfaces of the objects which are a part of Unit Test's Reporter; in addition, it also contains the mock-objects allowing one to check the functioning of this program by means of the unit tests. I wish to draw the reader's attention to the file test.cpp containing the unit tests. This file was created in parallel with building the interfaces.
  2. EasyTestViewer: This project contains the text of the program, Unit Test's Reporter.
  3. Sample: This project contains the file test.cpp with examples of tests.

Please make sure to see the file EasyTest.reg. It contains information about the configuration of the program and the location of the templates. When installing the product through UnitTestsReporterSetup.exe, this information is automatically placed into the registry. Alternatively, you may want to manually download the file EasyTest.reg and correct it with the editor regedit.

The program Unit Test's Reporter serves programmers working with Visual Studio 98, Visual Studio .NET 2003 and Visual Studio .NET 2005. Special attention should be paid to the projects placed in the files Sample_VC6.zip and Sample_2k3.zip attached to the paper.

Finally, make sure to define Post-Build Event in MS Visual Studio as detailed in Fig. 4-6. This will let the tests run automatically upon completion of the process of building the program.

Image 5

Fig. 4 Configuring the Post-Build Event in MS Visual Studio .NET 2005.

Image 6

Fig. 5 Configuring the Post-Build Event in MS Visual Studio 98.

Image 7

Fig. 6 Configuring the Post-Build Event in MS Visual Studio .NET 2003.

History

  • September, 2005: Version 1.0
    • Environment: Visual Studio .NET 2003.
  • February, 2006: Version 1.1
    • Environment: Visual Studio .NET 2005. Macros added: TEST_FUNCTION and USE. Macro restored: CHECK. Two projects created in Visual Studio 98 and Visual Studio .NET 2003, and making use of the Unit Test's Reporter are added.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Software Developer (Senior)
Israel Israel
MSc in System Engineering from Tallinn Technical University, Estonia. Currently, I work in a hitech enterprise in Israel.

Comments and Discussions

 
QuestionHow to use this tool?? Pin
GirishRamaiah3-Jul-15 1:15
GirishRamaiah3-Jul-15 1:15 
NewsSmall change Pin
Michael-K[anzieper]22-Mar-14 8:42
Michael-K[anzieper]22-Mar-14 8:42 
GeneralThanks for your efforts! Pin
thomas_nuaa14-Dec-08 22:42
thomas_nuaa14-Dec-08 22:42 
GeneralProblems compiling "EasyTestViewer" with VS2005 Express Pin
Michael0000000110-Oct-06 6:36
Michael0000000110-Oct-06 6:36 
GeneralRe: Problems compiling "EasyTestViewer" with VS2005 Express [modified] Pin
matt warren23-Nov-06 23:57
matt warren23-Nov-06 23:57 
QuestionOnly CppUnitLite? Pin
mannywine13-Jun-06 20:28
mannywine13-Jun-06 20:28 
AnswerRe: Only CppUnitLite? [modified] Pin
Michael-K[anzieper]15-Jun-06 18:14
Michael-K[anzieper]15-Jun-06 18:14 
GeneralGreat job! Pin
kellyonlyone19-Feb-06 23:30
sponsorkellyonlyone19-Feb-06 23:30 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.