Click here to Skip to main content
Click here to Skip to main content

MbUnit : Generative Unit Test Framework

, 15 Apr 2004
Rate this:
Please Sign up or sign in to vote.
A new highly flexible unit test framework with new fixtures

MbUnit was previously named GUnit. It has been renamed to avoid name clashing with another project.

Introduction

Unit testing is a great tool for ensuring an application quality and frameworks like NUnit [1] or csUnit [2] have made it very simple to implement. However, as the number of tests begins to grow, the need for more functionalities begin to show up. The above frameworks are based on the Simple Test Pattern which is basically the sequence of SetUp, Test, TearDown actions. Although highly generic, this solution lets a lot of work to be done by the test writer. Sadely, there is no easy way to derive and include a new "fixture" type in those frameworks.

Recently, Marc Clifton has formalized more than twenty unit test patterns in [3]. In a perfect world, each of these patterns would have a "specialized" fixture: testing a collection should not require rewritting the test class for each collection instance, this is what I thought after reading Marc's article, the next thought was "let's do it".

While enjoying my stay in hospital for surgery, I spent the time writing a new generative test unit framework called MbUnit. The objective of the framework is give to the end-users the "high order" test fixtures and to the develpers the tools to build new custom fixtures without modifying the Core. To illustrate that, MbUnit implements the Simple Test Pattern and provides new fixture types, like the usefull TypeFixture which applies tests to a particular type instance. It must emphasize that MbUnit is still a prototype, was written under the influence of morphine and other pain-killer substance, and is published on the CodeProject to get constructive feedback from CPians and specially from members of the AUT project.

In the following, it is assumed that the reader has basic knownledge of unit tests (NUnit, csUnit, JUnit)..

A simple example

In this example, we show the motivation for MbUnit and the key features about it. While developping the example, we show what kind of data-structure is needed and how the core was be built.

Simple Test Pattern

Let us consider the Simple Test Pattern which is implemented by most test unit framework available. This is the classic way of writing unit test as described in the figure below. A TestFixture attribute tags the test class, one SetUp method, tests are done in the Test tagged method and clean up is performed in TearDown tagged method. This is illustrated in the left of the figure.

The figure already illustrates one of the key idea of MbUnit: Fixture class should be definable at runtime by creating an execution pipe, using basic building blocks (SetUpAttribute, TestAttribute, etc...) provided by the framework.

Test Pattern

In the above example, we call the TestFixture class a TestFixturePattern, SetUp and TearDown are NonTestPattern tags for mehods that are not considered as tests. Test is a TestPattern. A TestFixturePattern must be able to describe itself at runtime and return it's execution path.

Defining the TestFixture attribute in MbUnit

As mentionned in the introduction, MbUnit provides basic building blocks that a developer can use to build complex fixtures. For example, the Simple Test pattern is implemented as follows in MbUnit (source first, comment below)

public class TestFixtureAttribute : TestFixturePatternAttribute 
{
     public override IRun GetRun()
     {
          SequenceRun runs = new SequenceRun();
            
          // setup
          OptionalMethodRun setup = new 
                              OptionalMethodRun(typeof(SetUpAttribute),false);
          runs.Runs.Add( setup );
            
          // tests
          MethodRun test = new 
                            MethodRun(typeof(TestPatternAttribute),true,true);
          runs.Runs.Add(test);
            
          // tear down
          OptionalMethodRun tearDown = new 
                           OptionalMethodRun(typeof(TearDownAttribute),false);
          runs.Runs.Add(tearDown);
            
          return runs;                        
     }
}

where

  • TestFixturePatternAttribute is the abstract base class for all new fixture attribute in MbUnit,
  • the GetRun method is called by the MbUnit core to know what is the execution path of the fixture. The fixture can use built-in basic attributes to build it's execution path.
  • An IRun instance can represent the call to a method, or to a sequence of methods, etc...
  • SequenceRun is a sequence of IRun's,
  • MethodRun is a IRun instance that wraps a call to a method tagged by a predefined attribute.
  • OptionalMethodRun is inherited from MethodRun and describes optional methods.

Tagging a class and scanning the test class

When you provide a test assembly to the framework, it will scan for test class tagged with TestFixtureAttribute derived attributes. In this case, we wrote one tagged with TestFixture.

Once the fixture is extracted, the framework calls GetRun() method and starts to explore the execution path.

The figure below show the different execution routes that are possible in the ClassicTest class. The two test method generate to branch in the execution tree.

Fixture

Method decorators

It is also possible to add "test decorator" like ExpectedException, which checks that an exeception is thrown:

...
   [Test]
   [ExpectedException(typeof(ArgumentNullException))]
   public void TestThrow()
   {
       throw new ArgumentNullException();
   }

Other supported decorators are:

  • ExceptedException, checks that an exception is thrown,
  • Repeat, test repeated sequentially,
  • ThreadedRepeat, test repeated on parralel threads,
  • Duration, lower and upper execution duration test,
  • Ignore, ignores the test

Execution Splitting

In order to have fully separated test, we create and store all the execution path and attach it to some GUI, usually a TreeView. Each execution path can be attached to a TreeNode for example, which makes it very easy to launch separately, or to launch all tests in separate threads, etc...

Fixture

The execution pipes are called RunPipe and are composed of a sequence of IRunInvoker instances.

Framework, buillt-in fixtures

MbUnit already comes with a few usefull fixtures, such as the Simple Test Fixture. Fixtures classes must derive from the TestFixturePatternAttribute class. Execution path diagrams are automatically generated by the framework using the QuickGraph [4] and Graphviz [5].

TestFixture

This fixture implements the Simple Test Pattern:

[TestFixture]
public class MyTest
{
    [SetUp]
    public void SetUp()
    {
        // set up the fixture
    }
    
    [Test]
    public void Test1()
    { 
       // do some test
    }
    
    [Test]
    public void Test2()
    {
      // another test
    }    
    
    [TearDown]
    public void TearDown()
    {
        // clean up
    }
}

TypeFixture

This fixture applies the test to a particular type. It is highly convinient when you want to write a fixture for an interface and test the interface implementations. The method tagged with Provider attribute must return an initialized object that is assignable with the tested type. This object is then feeded to the test methods:

[TypeFixture(typeof(IList),"IList test")]
public class TypeFixtureAttributeTest
{
    [Provider(typeof(ArrayList))]
    public ArrayList ProvideArrayList()
    {
        return new ArrayList();
    }

    [Provider(typeof(ArrayList))]
    public ArrayList ProvideFilledArrayList()
    {
        ArrayList list = new ArrayList();
        list.Add("hello");
        list.Add("world");
            
        return list;
    }
                
    [Test]
    public void AddElement(IList list)
    {
        int count = list.Count;
        list.Add(null);
        Assert.AreEqual(list.Count,count+1);
    }
}

EnumerationFixture

This fixture implements the testing of an IEnumerable / IEnumerator pair. The DataProvider tagged methods must return an enumerable collection that will be used to populate the test IEnumerable. This lets you test different scenarios like the empty collection case, null value, duplicate entries, etc... The collection is then feeded to the CopyToProvider methods that must create an instance of the tested IEnumerable instance and populate with the feeded data. The rest of the testing is done by the framework:

[EnumerationFixture]
public class DictionaryEnumerationFixtureAttributeAttributeTest
{
    private Random rnd = new Random();
    private int count = 100;        
    [DataProvider(typeof(Hashtable))]
    public Hashtable HashtableData()
    {
       Hashtable list = new Hashtable();
        for(int i=0;i<count;++i)
           list.Add(rnd.Next(),rnd.Next());
       return list;
    }
    
    [Provider(typeof(Hashtable))]
    public Hashtable HashtableProvider(IDictionary source)
    {
        Hashtable list = new Hashtable();
        foreach(DictionaryEntry de in source)
        {
            list.Add(de.Key,de.Value);
        }
        return list;
    }
        
    [Provider(typeof(SortedList))]
    public SortedList SortedListProvider(IDictionary source)
    {
        SortedList list = new SortedList();
        foreach(DictionaryEntry de in source)
        {
            list.Add(de.Key,de.Value);
        }
        return list;
    }
}

Examples and demo

Here are some details about the projects inclosed in the demo solution:

  • MbUnit.Core contains the core of the MbUnit framework. No fixtures are defined here.
  • MbUnit.Framework contains predefined fixtures discussed above
  • MbUnit.Tests contains some test classes using the fixtures from MbUnit.Framework,
  • MbUnit.Cons is a console demo application
  • MbUnit.GUI is a GUI application that you can use to load test assemblies. It will show the available tests and run the tests. At the current moment, this application is very basic.

Conclusions and further work

MbUnit is yet another way of looking at Unit testing. I beleive it has idea that should interrested the AUT team: It provides enough flexibility to let developer define their own fixture which makes it very powerful with regards to classic framework. While there is much work to do, (a lot of unit patterns to implement), MbUnit already provides new type of fixtures that ease up the test writer life.

History

  • 8/04/2004
    • Changed name to MbUnit,
    • Moved to Tigris
  • 1.1
    • Added colors to the GUI to know if tests failed, run successful,
    • Added progress bar, with test count
    • Better layout to see exception source.

References

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author

Jonathan de Halleux
Engineer
United States United States
Jonathan de Halleux is Civil Engineer in Applied Mathematics. He finished his PhD in 2004 in the rainy country of Belgium. After 2 years in the Common Language Runtime (i.e. .net), he is now working at Microsoft Research on Pex (http://research.microsoft.com/pex).

Comments and Discussions

 
GeneralProblem with Invoke() method. Pinmemberamit_lole13-Mar-07 21:20 
Generaltypefixture with invalid number of parameters Pinmemberforis.gabor@stud.u-szeged.hu15-Nov-04 6:58 
GeneralGUnit changed named to MbUnit and moved of site PinmemberJonathan de Halleux24-Mar-04 7:16 
GeneralGUnit at Tigris PinmemberJonathan de Halleux15-Mar-04 21:32 
GeneralProblems with 1.2 PinmemberStephen Johns2-Mar-04 10:57 
GeneralRe: Problems with 1.2 PinmemberStephen Johns2-Mar-04 11:05 
GeneralRe: Problems with 1.2 PinmemberJonathan de Halleux2-Mar-04 11:25 
Stephen Johns wrote:
Is there anyway to save a configuration? My "File" menu does not contain any MenuItems.
 
Hi, the GUI is still very beta. And yes, it is annoying to not being able to store the configuration.
 
I'm adding that to my todo list Smile | :)
 
Jonathan de Halleux.

www.dotnetwiki.org

GeneralRe: Problems with 1.2 PinmemberJonathan de Halleux2-Mar-04 11:26 
Generalv1.2 is out PinmemberJonathan de Halleux24-Feb-04 15:39 
GeneralComment your votes PinmemberJonathan de Halleux17-Feb-04 22:23 
GeneralVC++ 6.0 PinmemberJose Toledo16-Feb-04 22:18 
GeneralRe: VC++ 6.0 PinmemberJonathan de Halleux17-Feb-04 0:57 
GeneralNice PinmemberGiles15-Feb-04 1:44 
GeneralRe: Nice PinmemberJonathan de Halleux15-Feb-04 3:37 
GeneralIn general PinmemberDavid Gallagher11-Feb-04 6:44 
GeneralRe: In general PinmemberJonathan de Halleux11-Feb-04 7:54 
GeneralRe: In general PinmemberGarth J Lancaster11-Feb-04 11:29 
GeneralRe: In general PinmemberJonathan de Halleux12-Feb-04 1:34 
GeneralSomewhat difficult to get started PinmemberPål K Tønder10-Feb-04 22:57 
GeneralRe: Somewhat difficult to get started PinmemberJonathan de Halleux11-Feb-04 0:40 
GeneralRe: Somewhat difficult to get started Pinmemberkwokoek17-Feb-04 13:42 
GeneralRe: Somewhat difficult to get started PinmemberJonathan de Halleux17-Feb-04 22:19 
GeneralRe: Somewhat difficult to get started PinmemberJonathan de Halleux24-Feb-04 6:23 
GeneralG-Unit Pinmembermystro_AKA_kokie10-Feb-04 11:43 
GeneralRe: G-Unit PinmemberJonathan de Halleux10-Feb-04 12:19 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web03 | 2.8.141223.1 | Last Updated 16 Apr 2004
Article Copyright 2004 by Jonathan de Halleux
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid