Click here to Skip to main content
Click here to Skip to main content

Tagged as

Go to top

Running NUnit tests in parallel

, 8 Feb 2013
Rate this:
Please Sign up or sign in to vote.
NUnit doesn't support running the tests in a fixture in parallel. A trick for getting around this limitation is given.

Problem

For a project I'm working on, I have several NUnit test fixtures with integration tests which are not very resource intensive as they call web services and have to wait for results. As NUnit tests are run in sequence, this means that the whole suite of tests takes a long time to run on the build server. An obvious solution to this problem would be to somehow parallelize the tests but unfortunately NUnit has no support for this. Also I want to be able to still run the tests individually from Visual Studio for debugging purposes.  

Possible solutions

After browsing around a bit, I have come up with the following options:  
  1. Wait for NUnit 3, which should support running tests in parallel.
  2. Use an alternative unit testing framework. The one integrated in Visual Studio 2010 should have support for parallel tests, though I haven't tested it.
  3. Download source code for NUnit and customize.
  4. Come up with some ad-hoc solution.
Options 1 and 2 are not relevant for me as I'm forced to use the version of NUnit available now. I downloaded the NUnit source code and actually managed to make it run unit tests in parallel without too much effort, but the solution was kind of hacky and I would have to deploy custom NUnit DLLs to the build server, which will probably be a hassle and may not be an option at all.  

Ad-hoc Solution

The solution I came up with is a base class which uses reflection to start all tests at once. This base class is inherited by test fixtures which call the Parallelize method in their TestFixtureSetUp method to start running all tests. The test fixture now has two methods for each test: The TestX method decorated with the Test attribute just calls the RunTest method while the actual testing is performed in the XAction method which is called in a separate thread. The RunTest method joins with the thread matching the calling test method and rethrows any exceptions thrown in the test thread. This allows errors to be correlated with the right test.  

public class ParallelizedTestFixture
{
    private readonly IDictionary<string, TestRunner> threads = new Dictionary<string, TestRunner>();
    private readonly IDictionary<string, Action> actions = new Dictionary<string, Action>();
 
    protected bool RunFromConsole { get; set; }
 
    protected void Parallelize()
    {
        string executable = Environment.GetCommandLineArgs()[0];
        RunFromConsole = executable.ToLower().Contains("nunit-console");
 
        foreach (MethodInfo methodInfo in GetType().GetMethods(BindingFlags.Public | BindingFlags.Instance))
        {
            if (methodInfo.GetCustomAttributes(typeof(TestAttribute), false).Length == 1 &&
                methodInfo.GetCustomAttributes(typeof(IgnoreAttribute), false).Length == 0)
            {
                string key = GetKey(methodInfo);
                MethodInfo actionMethod = GetType().GetMethod(key + "Action",
                                                                BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance);
                if (actionMethod != null)
                {
                    actions.Add(key, () => actionMethod.Invoke(this, new object[] { }));
                }
            }
        }
 
        if (RunFromConsole)
        {
            foreach (KeyValuePair<string, Action> actionPair in actions)
            {
                TestRunner testRunner = new TestRunner(actionPair.Value);
                ThreadStart ts = testRunner.DoWork;
                Thread thread = new Thread(ts);
                testRunner.Thread = thread;
                thread.Start();
                threads.Add(actionPair.Key, testRunner);
            }
        }
    }
 
    protected void RunTest()
    {
        string key = GetKey(new StackFrame(1, false).GetMethod());
        if (RunFromConsole)
        {
            TestRunner testRunner = threads[key];
            testRunner.Thread.Join();
            Assert.IsNull(testRunner.Error, string.Format(CultureInfo.InvariantCulture, "Unexpected exception {0}", testRunner.Error));
        }
        else
        {
            actions[key]();
        }
    }
 
    private static string GetKey(MethodBase method)
    {
        return method.Name.Substring("Test".Length);
    }
 
    private class TestRunner
    {
        private readonly Action action;
 
        public TestRunner(Action action)
        {
            this.action = action;
        }
 
        public Thread Thread { get; set; }
        public Exception Error { get; private set; }
 
        public void DoWork()
        {
            try
            {
                action();
            }
            catch (Exception e)
            {
                Error = e;
            }
        }
    }
}
 
[TestFixture]
public class SomeTextFixture : ParallelizedTestFixture
{
    [TestFixtureSetUp]
    public void FixtureSetup()
    {
         Parallelize();
    }
   
    public void Scenario1Action()
    {
         // Do actual testing.
    }
 
    [Test]
    public void TestScenario1()
    {
         RunTest();
    }
 
    public void Scenario2Action()
    {
    }
 
    [Test]
    public void TestScenario2()
    {
         RunTest();
    }        
}
Note the RunFromConsole property which indicates if the tests are run using nunit-console. This means that the tests are run in parallel when run on the build server and run individually when I start them from Visual Studio.   This solution has a few problems:  
  • Each test requires two methods.
  • There should be no SetUp and or TearDown in fixtures inhering from ParallelizedTestFixture as they will be called at the wrong time.

Conclusion

The solution described is not particularly beautiful but it has cut the execution time of NUnit on the build server by 66% so I will stick with it for now. Suggestions for improvements or alternatives are very welcome.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Andreas Andersen
Software Developer (Senior)
Denmark Denmark
.NET developer. I wanted to be first an astronaut, then a jet pilot, but when I got a Commodore 64 for Christmas I never looked back. Also I would never have qualified for the first two things and everybody knows computer programmers get all the girls.

Comments and Discussions

 
GeneralThanks! Didn't know about that. PinsubeditorWalt Fair, Jr.19-Dec-10 16:13 
GeneralInteresting. Didn't know about NUnit 3 supporting parallel t... PinsubeditorIndivara8-Dec-10 21:14 
GeneralReason for my vote of 5 An interesting technique to speed up... PinmemberCIDev8-Dec-10 6:54 
QuestionWhat about test cases? PinmemberDmitri Nesteruk10-Apr-11 10:25 
AnswerRe: What about test cases? PinmemberCharles Ritchea11-Aug-14 11:45 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140916.1 | Last Updated 8 Feb 2013
Article Copyright 2010 by Andreas Andersen
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid