How To Set Up A Parameterized Unit Test

In my previous post, we saw how to set up unit testing using Visual Studio. The code we want to test is a (simple) function to calculate the greatest common divisor, and we wrote our first test.

The test we wrote passes 5 and 10 in the function, and expects 5 back as the GCD. We then adapted the function to just return 5 and this test passes now.

The Problem

So we wrote a test to calculate GCD(10, 5). Now we must try with some other values to at least test some other cases:

• Let’s try this with some other values
• What if there is no GCD (besides 1)? Will the function calculate this correctly?
• What if we swap the parameters? Will the result be the same?
• What if a == 0? or b == 0? or a == 0 and b == 0?

With our current knowledge, we now have 2 choices:

1. We add all these cases to the current test. This will work, but if one of the cases fails the rest of the cases will not be executed anymore. So we’ll be correcting our code very slowly and we may miss the patterns of why the function is wrong.
2. We write a separate test for each case. This is a lot of work. Let’s say you want to test 10 cases, then you’ll need to write 10 tests. This is an open invitation for copy/paste programming!

In both cases, we have a lot of repetitive work (or copy/paste) to do. And if we want to change the tests, we have to change it in all the places.

Parameterized Unit Tests

So we need a way to write the test once, and then pass different sets of parameters in the test. In the Microsoft testing framework, this is not so easy. Testing frameworks like XUnit allow to put the different test cases in the code of the tests, but this is not implemented (yet?) in the Microsoft one.

So How Do We Do It?

The default way is to create an external file with the test data in it. You then use this file as a data source for your tests. The data source can be an Excel file, a CSV file, an Access table, a SQL Server table, … A lot of data sources can be used, but unfortunately they all live outside of the source file for your test. This means that for your tests, there is now an external dependency. If the files are not available during the tests, your tests will fail, and if somebody changes the files in a bad way (maybe because they don’t know what the files are for), you’ll be in trouble as well.

On top of that, for some file formats (like Excel, etc.) you’ll need additional software to be installed on your test machine, which may be a problem as well. See also the footnote in this post.

Creating the Data Source

I like to organize my work, so in the test project I created a new folder called “Data” (not a surprise). In this folder, create a CSV file called “TestData.csv” with for each interesting combination a row of 3 values: a, b and their GCD. Some caveats:

• Make sure that the file is comma-separated, don’t use a semicolon as a separator. This may depend on the regional settings on your machine. This may also be a cause of problems when you execute your tests on another machine.
• Make sure that the file is saved as ANSI. You can use a tool like Notepad++ to verify this. If this is not correct, the name of your first column (“a” in my case) may contain some weird characters.

Also set “Copy to output directory” to “Copy Always” and make sure that the file is also in your source code controls system.

To make it easier to follow the rest of the discussion, here are the contents of my CSV file:

a,b,gcd
5,10,5
10,5,5
12,8,4
8,12,4
12,7,1
7,12,1
1,0,1
0,1,1

Notice that currently only the first 2 cases will work because in the current implementation, the function just returns 5.

Next, we need to add the TestContext property to our test class.

public TestContext TestContext { get; set; }

This will be used by MS Test to refer to the data source. This is actually the data source that we refer to in our test method. We will use the DataRow property of TestContext, which is hidden in the System.Data assembly. So you’ll need to add this assembly into your test project references.

Now, we can extend the test method to use the data in the Excel file:

[DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", 
@".\Data\TestData.csv", "TestData#csv", DataAccessMethod.Sequential)]
[TestMethod()]
public void CalcGCD2ParameterizedTest()
{
// arrange
int a = Convert.ToInt32(TestContext.DataRow["a"]); ;
int b = Convert.ToInt32(TestContext.DataRow["b"]); ;
int expected = Convert.ToInt32(TestContext.DataRow["GCD"]); ;
Calc c = new Calc();

    // act
int actual = c.CalcGCD2(a, b);

    // assert
Assert.AreEqual(expected, actual, $"a={a}, b={b}");
}

The DataSource attribute contains the provider name, in this case the file name, and for a CSV file we need to also specify a table name which is the filename again, but with the period replaced by a hash sign. So “TestData.csv” becomes “TestData#csv”.

This test will be executed for each row in the CSV file. TestContext.DataRow contains the data for the current row, which can be obtained by DataRow[“fieldname”]. This will always be a string, so we need to convert to an integer before we can actually use it.

In the assertion, I added a string mentioning the values for a and b. MSTest will just tell you which row failed, not the values that were used in the test. When you add a string to the assertion you know for which values a test has failed.

Adding new test cases is now as easy as adding an additional line in the CSV file.

If you prefer to use a configuration file to define your data source, look at this link: https://msdn.microsoft.com/en-us/library/ms243192.aspx.

Fixing the Code

So now that we know that our code isn’t working properly, let’s fix it. We’ll use Euclid’s algorithm to calculate the GCD:

public int CalcGCD3(int x, int y)
{
if (y == 0)
return x;
return CalcGCD3(y, x % y);
}

With this code, all the tests succeed!

Conclusion

Using data-driven unit tests, we are able to describe a lot of test cases in a simple CSV file. Unfortunately, we have to create a separate file for each data-driven test. Test frameworks like XUnit have an easier way of accomplishing this, by defining the test data as attributes for the tests.

It is possible to use Unit Test Type Extensibility to provide a way to refer to your test cases inline, if you’re interested, here is the link: http://blogs.msdn.com/b/vstsqualitytools/archive/2009/09/04/extending-the-visual-studio-unit-test-type-part-2.aspx.

In Microsoft’s defense: there is a new feature called “IntelliTest” that will provide something like data-driven unit tests, but even better. You guessed it, my next post will be about IntelliTest.

Footnote

I tried to run the tests using Excel 2016 as a data source, but this doesn’t seem to work on my machine. If you succeed to make this work, then put something in the comments. I’ll update the article and will mention your name so that you’ll get instant eternal fame!

References

• https://msdn.microsoft.com/en-US/library/ms182527(v=vs.140) => How to create a data-driven unit test
• https://msdn.microsoft.com/en-US/library/ms243192 => Using a config file to define a data source
• http://callumhibbert.blogspot.be/2009/07/data-driven-tests-with-mstest.html