Running NUnit tests in parallel

Problem

For a project I'm working on, I have several NUnit test fixtures with integration tests which are not very resource intensive as they call web services and have to wait for results. As NUnit tests are run in sequence, this means that the whole suite of tests takes a long time to run on the build server. An obvious solution to this problem would be to somehow parallelize the tests but unfortunately NUnit has no support for this. Also I want to be able to still run the tests individually from Visual Studio for debugging purposes.  

Possible solutions

After browsing around a bit, I have come up with the following options:  

1. Wait for NUnit 3, which should support running tests in parallel.
2. Use an alternative unit testing framework. The one integrated in Visual Studio 2010 should have support for parallel tests, though I haven't tested it.
3. Download source code for NUnit and customize.
4. Come up with some ad-hoc solution.

Options 1 and 2 are not relevant for me as I'm forced to use the version of NUnit available now. I downloaded the NUnit source code and actually managed to make it run unit tests in parallel without too much effort, but the solution was kind of hacky and I would have to deploy custom NUnit DLLs to the build server, which will probably be a hassle and may not be an option at all.  

Ad-hoc Solution

The solution I came up with is a base class which uses reflection to start all tests at once. This base class is inherited by test fixtures which call the Parallelize method in their TestFixtureSetUp method to start running all tests. The test fixture now has two methods for each test: The TestX method decorated with the Test attribute just calls the RunTest method while the actual testing is performed in the XAction method which is called in a separate thread. The RunTest method joins with the thread matching the calling test method and rethrows any exceptions thrown in the test thread. This allows errors to be correlated with the right test.  

public class ParallelizedTestFixture
{
    private readonly IDictionary<string, TestRunner> threads = new Dictionary<string, TestRunner>();
    private readonly IDictionary<string, Action> actions = new Dictionary<string, Action>();
 
    protected bool RunFromConsole { get; set; }
 
    protected void Parallelize()
    {
        string executable = Environment.GetCommandLineArgs()[0];
        RunFromConsole = executable.ToLower().Contains("nunit-console");
 
        foreach (MethodInfo methodInfo in GetType().GetMethods(BindingFlags.Public | BindingFlags.Instance))
        {
            if (methodInfo.GetCustomAttributes(typeof(TestAttribute), false).Length == 1 &&
                methodInfo.GetCustomAttributes(typeof(IgnoreAttribute), false).Length == 0)
            {
                string key = GetKey(methodInfo);
                MethodInfo actionMethod = GetType().GetMethod(key + "Action",
                                                                BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance);
                if (actionMethod != null)
                {
                    actions.Add(key, () => actionMethod.Invoke(this, new object[] { }));
                }
            }
        }
 
        if (RunFromConsole)
        {
            foreach (KeyValuePair<string, Action> actionPair in actions)
            {
                TestRunner testRunner = new TestRunner(actionPair.Value);
                ThreadStart ts = testRunner.DoWork;
                Thread thread = new Thread(ts);
                testRunner.Thread = thread;
                thread.Start();
                threads.Add(actionPair.Key, testRunner);
            }
        }
    }
 
    protected void RunTest()
    {
        string key = GetKey(new StackFrame(1, false).GetMethod());
        if (RunFromConsole)
        {
            TestRunner testRunner = threads[key];
            testRunner.Thread.Join();
            Assert.IsNull(testRunner.Error, string.Format(CultureInfo.InvariantCulture, "Unexpected exception {0}", testRunner.Error));
        }
        else
        {
            actions[key]();
        }
    }
 
    private static string GetKey(MethodBase method)
    {
        return method.Name.Substring("Test".Length);
    }
 
    private class TestRunner
    {
        private readonly Action action;
 
        public TestRunner(Action action)
        {
            this.action = action;
        }
 
        public Thread Thread { get; set; }
        public Exception Error { get; private set; }
 
        public void DoWork()
        {
            try
            {
                action();
            }
            catch (Exception e)
            {
                Error = e;
            }
        }
    }
}
 
[TestFixture]
public class SomeTextFixture : ParallelizedTestFixture
{
    [TestFixtureSetUp]
    public void FixtureSetup()
    {
         Parallelize();
    }
   
    public void Scenario1Action()
    {
         // Do actual testing.
    }
 
    [Test]
    public void TestScenario1()
    {
         RunTest();
    }
 
    public void Scenario2Action()
    {
    }
 
    [Test]
    public void TestScenario2()
    {
         RunTest();
    }        
}

Note the RunFromConsole property which indicates if the tests are run using nunit-console. This means that the tests are run in parallel when run on the build server and run individually when I start them from Visual Studio.   This solution has a few problems:  

• Each test requires two methods.
• There should be no SetUp and or TearDown in fixtures inhering from ParallelizedTestFixture as they will be called at the wrong time.

Conclusion

The solution described is not particularly beautiful but it has cut the execution time of NUnit on the build server by 66% so I will stick with it for now. Suggestions for improvements or alternatives are very welcome.