Click here to Skip to main content
15,867,568 members
Articles / Programming Languages / C++
Article

ShortCUT - A Short C++ Unit Testing Framework

Rate me:
Please Sign up or sign in to vote.
4.60/5 (9 votes)
15 Feb 2007CPOL7 min read 65.2K   948   44   10
A very simple, customizable unit-testing framework for C++ developers
Unit test console output

Introduction

This unit testing framework consists of about 125 lines of code in a single header. It aims to be the simplest way to get started writing unit tests for C++ developers. Being simple, it is also easy to customize. Many unit testing frameworks require linking to a separate library or require jumping through several hoops just to get started. This can make it more difficult to begin writing tests.

One of the situations developers often face when they set out to write unit tests is that they're already working on a project. If the project isn't broken out into independent libraries, it can be difficult to write stand-alone unit tests. In addition, many of the core functions of a program simply can't be broken out into a separate test executable. Ideally, a developer should be able to write a set of unit tests, include it with a single function call within the program itself, and easily #define the test out later on.

This may not seem like a strong approach to software engineering. However, it is generally acknowledged that testing early and often is better than the alternative. By minimizing the effort required to get started writing tests, the goal of testing early can be more easily accomplished. As the test suite grows and the project moves forward, the tests can be factored into a separate library or executable as time permits.

Background

I started my quest by looking for existing solutions. There is a lot of interest and activity around the xUnit frameworks within the unit testing community. It originated with Smalltalk's SUnit framework, which inspired the developers of JUnit. This, in turn, inspired the creators of NUnit, CppUnit, and several other similar frameworks.

One of the things that most of these frameworks have in common is that they are built using languages that support some reflection capability. This makes it easier to assign attributes to a test function or a setup function and have it automatically included in a test run. C++ developers are not so fortunate. C++ developers must do things more manually (or resort to templates or macros). This isn't a big deal, though. It's what we expect.

Many articles have been written on unit testing in the xUnit community and the various libraries available. However, there is surprisingly little written about options for C++ developers. One such article at Games from Within offers a survey of some of the frameworks available.

For an introduction to unit testing in general, have a look at one of the comprehensive articles here at The Code Project.

Design

After evaluating several frameworks, I decided that none of them met my basic criteria of being extremely simple to use and modify. I decided to see how hard it would be to write a framework that would fit into a single header and would consist of as few lines of code as possible. Here is a list of my basic design criteria:

  • Fits into a single header file (no source modules or libraries)
  • No longer than a couple of hundred lines of code
  • Easy to modify and extend
  • Re-routable message output
  • Optional macros
  • No templates
  • No dynamic memory allocation
  • Usable in an embedded environment
  • Usable on down-level C++ compilers (no fancy C++ features)

The design constraint that it should not dynamically allocate any memory would allow tests to be created on the stack. This would make it easy to write a simple main program entry point and just declare and run the tests all at once without worrying about cleanup or memory leaks.

A consequence of these constraints was that a class would be required for each test. The alternative of using a function pointer wouldn't allow for chaining tests within a suite without allocating memory. Also, it didn't seem in the spirit of C++ to use function pointers.

Using the Code

To get a test up and running, three things are necessary:

  • A test case must be written
  • A test suite may be written
  • The test suite and test case must be added to a runner class and then called

We'll follow this sequence in the illustration. The sample included for download is different.

Writing a Test Case

In this example, we derive our test case from the TestCase base class. TestCase, like the other components of the framework, is a struct. This helps us avoid a lot of public access specifiers.

Test code is added to the single test method. The TestSuite class contains any data that is shared across test cases and it is passed to every test call. The name method is used to provide meaningful output in the event of a test case failure.

C++
struct TestAccountWithdrawal : TestCase
{
    const char* name() { return "Account withdrawal test"; }

    void test(TestSuite* suite) 
    {
        TestAccountSuite* data = (TestAccountSuite*)suite;

        data->account->Deposit(10);

        bool succeeded = data->account->Withdraw(11);

        T_ASSERT(succeeded == false);
        T_ASSERT(data->account->Balance() == 10);
    }
};

Adding a Test Suite

The test suite contains a group of related tests. It serves the purpose of both the test suite and test fixture in some other unit testing frameworks.

The two key methods (both optional) are setup and teardown. Each call to a test case is framed with this call pair.

C++
struct TestAccountSuite : TestSuite
{
    const char* name() { return "Account suite"; }

    void setup()
    {
        account = new Account();
    }

    void teardown() 
    {
        delete account;
    }

    Account* account;
};

Putting It All Together

Once a test suite and at least one test case have been written, they may be added to a runner and executed.

C++
#include <stdio.h>
#include "shortcut.h"
#include "tests/account.h"

int main(int argc, char* argv[])
{
    TestRunner runner;
    TestAccountSuite accountSuite;
    TestAccountWithdrawal accountWithdrawalTest;

    accountSuite.AddTest(&accountWithdrawalTest);
    runner.AddSuite(&accountSuite);
    runner.RunTests();
    
    return 0;
}

One of the benefits of this lightweight system is that all test code may be kept in headers. This avoids some duplication between a separate class declaration and implementation. Because the unit test framework is implemented in a single header, only a single driver module (containing main, for example) is required.

This system also makes it easy to add tests to an existing application. For example, the tests could be called at program startup within a #ifdef DEBUG section. In release mode, no trace of the tests would exist in the application binary. This may not be the case when linking to other unit testing libraries.

Obviously, this is not a long-term solution. It is a good way to get started, though. Developers can start writing tests immediately and the tests can be factored into a separate executable when time permits.

Internals

This section may be skipped. It explains a little bit about how the (very few) moving pieces work.

Base Classes

All test cases derive from a very basic class, called TestCase.

C++
struct TestCase
{
    TestCase() : next(0) {}
    
    virtual void test(TestSuite* suite) {}
    virtual const char* name() { return "?"; }
    
    TestCase* next;
};

The class has a name accessor method, which is used for logging errors. The test suite uses the next pointer to chain the test cases into a linked list. The test itself is implemented in the virtual test method.

The test suite has the same structure, except that it contains a list of tests and has different methods to override: setup and teardown.

C++
struct TestSuite
{
    TestSuite() : next(0), tests(0) {}

    virtual void setup() {}
    virtual void teardown() {}
    virtual const char* name() { return "?"; }

    void AddTest(TestCase* tc)
    {
        tc->next = tests;
        tests = tc;
    }

    TestSuite* next;
    TestCase* tests;
};

As stated earlier, the test suite class plays the same role as the test suite and test fixture classes in other frameworks. In such frameworks, suites often play the role of a test grouping construct while the fixture provides a setup/teardown mechanism. Since ShortCUT is such a simple framework, there was no need to create this additional level of complexity. If a development team needs this feature, it may be easily added as a customization.

The Runner

The test runner is the heart of the system. It, too, is very straightforward. The main routine, RunTests calls RunSuite for each suite.

C++
struct TestRunner
{
    ...
    void RunSuite(TestSuite* suite, int& testCount, int& passCount)
    {
        TestCase* test = suite->tests;
        while (test)
        {
            try
            {
                suite->setup();
                test->test(suite);
                passCount++;
            }
            catch (TestException& te)
            {
                log->write("FAILED '%s': %s\n", test->name(), te.text());
            }
            catch (...)
            {
                log->write("FAILED '%s': unknown exception\n", test->name());
            }

            try
            {
                suite->teardown();
            }
            catch (...)
            {
                log->write("FAILED: teardown error in suite '%s'\n", suite->name());
            }

            test = test->next;
            testCount++;
        }
    }
    ...
}

The key points to note here are that, first, the log class can be implemented and set outside of the framework. This makes it easy to display results to another output target, such as a window.

The second point, which can be an annoyance, is that the test suites and test cases are chained together in singly-linked lists. This means that they are traversed and executed in LIFO order. This is the reverse from the order in which they were added.

It would be a simple matter to customize the framework to fix annoyances like this. I chose not to, since the goal was to make the framework as simple as possible.

Customization and Conclusion

The main goal of the framework was to have the absolute simplest system possible, within the design requirements and constraints. Every line of code was scrutinized for its value. In some cases, such as with the TestLog class, a few lines were added because they helped to meet a design requirement. Even though the framework would have been simpler, it would have lost basic flexibility.

The header is about 200 lines of code. A quarter of the code is actually unnecessary. It was included as an example of how to implement custom assert functionality through exceptions and how to implement a couple of helper macros to avoid repetitive code.

The framework is useable in its basic form. It is hoped that it will form the basis of systems that are tailored to the needs of the developers who use them (instead of the other way around). It should provide enough utility to get going quickly, and its basic structure should make it easy to modify, customize, and extend going forward.

History

  • 15th February, 2007: Original article

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Web Developer
United States United States
I work as a developer in the Seattle area. I am currently residing in France.

Comments and Discussions

 
Generalthe absolute simplest system possible Pin
ravenspoint16-Feb-07 9:23
ravenspoint16-Feb-07 9:23 
GeneralRe: the absolute simplest system possible Pin
Todd-Lucas16-Feb-07 9:59
Todd-Lucas16-Feb-07 9:59 
Your system makes clever use of macros and is indeed a *very* lightweight system. This is interesting material for comparison. Thanks for posting it.

My goal was to provide the simplest vanilla C++ system which met some additional design requirements. This did cause the code to 'bloat' to over 100 lines. Things like adding a TestSuite between the TestRunner and the TestCase did add complexity. I put in that requirement, though, because most unit test frameworks today have setup/teardown mechanisms. Indeed, it is one of the most useful things about SUnit and its derivatives.

My framework also requires a little extra setup in terms of defining a class for each test case. I could have added macros (in fact, I do provide a couple in the header and demo as an example). I avoided them, in general, because, a) many C++ developers dislike macros and, b) people can easily add macros themselves to suit their needs. I also wanted to make it very clear what was going on at the C++ level so that people could customize things more easily.

Thanks a lot for the feedback!

- Todd

GeneralRe: the absolute simplest system possible Pin
ravenspoint16-Feb-07 10:30
ravenspoint16-Feb-07 10:30 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.