VSTest is a powerful unit-testing framework that's tightly integrated into Visual Studio Team System. However, there's still room for improvement and a great way to improve upon it is to add to the framework using some of the new C# language features.
Test-driven Development (TDD) has been around for awhile now. There are numerous frameworks available in .NET that enable this development methodology, such as NUnit, MbUnit and VSTest, introduced with Visual Studio Team System 2005.
Recently, TDD has been undergoing a metamorphosis into Behavior-driven Development (BDD). This newer development methodology is a subtle refinement on TDD. In BDD, the emphasis is on specifications, not testing. This is mostly a change in the point of view, not a radical change in the methodology. Instead of tests, you have specifications. Instead of test methods, you have behavior specifications.
The traditional TDD libraries can still be used when following BDD, although the trend is to change the libraries so that they follow the newer terminology. This helps the developer, especially the initiates, to focus on specifying behaviors instead of focusing on testing. There are a few newer .NET libraries taking this approach, such as NSpec, NBehave and NSpecifiy.
In an article I've been working on, I was using VSTest and attempting to do TDD. As someone new to this methodology, having always in the past created the code first and then created unit tests, I was finding the whole concept to be difficult. While researching the topic to determine why, I started to realize that I was writing my tests with a focus on the implementation details of the code that didn't yet exists, rather than on the behavior I desired. This is precisely what BDD, I believe, is all about. Thinking in the proper way started to make it easier (I won't yet claim that it's easy) to follow the "test first" way of developing software.
I considered switching to one of the other BDD-specific testing frameworks at this point, but I really like the integration of VSTest. The only real differences between the BDD testing frameworks and the TDD frameworks is the vocabulary. Realizing this, I noticed that in VSTest there's very little vocabulary forced on you. There's
Assert, for the most part. All of these use vocabulary that BDD would prefer we not use. However, for me, the two attribute names are not that big of an issue, as long as you name your test methods in a manner more in-line with specifications. It would be nice to change the attribute names, especially for BDD initiates, but it's not really necessary. The
Assert class(es), on the other hand, did cause me some heart burn.
First, from a practical point of view, the
Assert class(es) suffer from the simple fact that they are not complete. Although there are a lot of assertions you can make using VSTest, the other frameworks all provide more. Adding assertions to fill out the gaps would actually be awkward. A simple example of this would be when you realize that
StringAssert does not have a
DoesNotContain method. In order to add this assertion, you'd have to create your own assertion class and there in lies the problem: what would you call it? Even in its own namespace, you couldn't use the name
StringAssert with out causing usage to be difficult because of the reuse of the name. Any other name would be likely to be confusing or overly verbose.
Second, from a BDD perspective, the names of the assertions are already poor. BDD wants your specifications to be written in a "fluent" manner, as close to plain English as possible. Not only does this make the specifications easier to read, but tools can be used to strip out the specifications from the code and produce documentation that is readable by non-developers.
These are the factors that led to my developing these extensions.
Using the Code
The extensions are provided as a framework that can be easily used alongside the VSTest framework. Simply add a reference to the
CodeProject.Visual Studio.QualityTools.UnitTestFramework assembly.
The main class used from this framework is the
Specify static class. Two static methods in this class (with logical overloads for specific types) provide the starting point for any specifications:
ThatAction. These methods return a
SpecificationValue, which provides a convenient type for writing extension methods that describe the actual specification.
Extension methods are a new concept in C# 3.5 (and other .NET languages). They are simply static methods that have their first parameter qualified with the
this keyword. The compiler will allow you to call these extension methods using a syntax identical to calling a member of the object, even though these are not member functions. For example, there's a member of
public static void ShouldEqual<T>(this SpecificationValue<T> self,
With this extension method, we can write a specification that some property on our class should have a specific value under some given condition by writing very fluent code.
public void SomeProperty_should_construct_with_the_value_xyzzy()
SomeObject target = new SomeObject();
The framework already includes most of the typical specifications found in the other unit testing libraries. However, if you find that you need a new specification, it's easy to create an extension method -- taking
SpecificationValue as the first parameter -- that throws the normal
AssertFailedException if the specification fails.
SpecificationValue is a generic type and there are a few rules you should follow when writing your extension methods because of this.
- If the specification should work on any type, make the extension method generic with a generic parameter that's used to specify the type of
- If the specification should take a specific type or a type derived from that type (or that implements the interface type), make the extension method generic with a generic parameter that's used to specify the type of
SpecificationValue. Then use generic constraints to constrain the type.
- If the specification should take a specific type that is either sealed or built-in, do not make the extension method generic. Instead, specify the exact
If you look at the code for the framework, you'll find numerous ways in which these rules were applied. In addition to the
Specify class, there are a few specialty classes included in the framework that I've found useful when creating data models. The
PropertyChangedWatcher makes it easy to specify that your properties on objects implementing
INotifyPropertyChanged should raise
PropertyChanged events when modified.
public void SomeProperty_should_raise_PropertyChanged_when_modified()
SomeObject target = new SomeObject();
PropertyChangedWatcher watcher = new PropertyChangedWatcher(target);
target.SomeProperty = "xyzzy";
CollectionChangedWatcher provides similar functionality for collections that implement
TestExtensions provides an extension method,
SerializeClone, that helps in testing serializable objects.
public void SomeObject_should_be_serializable()
SomeObject target = new SomeObject();
string expectedValue = "xyzzy";
target.SomeProperty = expectedValue;
SomeObject clone = target.SerializeClone();
Points of Interest
This framework provides a nice vocabulary for writing specifications when doing BDD, while continuing to use VSTest for its tight integration with the IDE. It does not, however, provide a complete BDD framework, since we're still relying on the VSTest framework for tests. BDD purists may find this distasteful. It would be nice if VSTest provided us more extensibility points so that we could remove the "test" vocabulary entirely, but currently I do not believe that there is any way to accomplish that.
In addition to providing a better specification vocabulary, the framework makes it easier to extend the specifications by using extensions methods, new in .NET 3.5. This is the key to the entire framework.
12-8-2007: Initial release.