Unit Testing StyleCop Rules

CodeProject

One of the single most influential cost factors for software projects is code readability and understandability - and the most important factor for readability in turn lies in the adherence to Coding Style Guidelines. To enforce such guidelines, Microsoft has provided a free tool which checks a given set of source code files against a given set of style and consistency rules: MS StyleCop. The tool comes with quite a lot of predefined rules - however, they mirror the coding practice at Microsoft and not all of them may be appropriate or sufficient for others. Therefore, StyleCop allows for authoring custom rules to extend or modify the original set of rules. How this can be done, is well covered elsewhere on the web (e.g. here, here, or here).

But contrary to the rules authoring issue, very little - almost nothing - can be found that deals with testing issues. All I found was this post, which gave me a first insight into the issue and inspired me to write a custom Gallio/MbUnit test fixture that lets you write tests against StyleCop rules as simple as this:

[Test, TestsOn("SA1200: UsingDirectivesMustBePlacedWithinNamespace")]
public void TestUsingDirectivesMustBePlacedWithinNamespace()
{
   AddSourceFile("../../TestFiles/UsingDirectiveOnTop.Testfile.cs");
 
   RunAnalysis();
 
   AssertViolated("UsingDirectivesMustBePlacedWithinNamespace");
}

Alternatively, you may check if a certain rule was violated for the expected number of times...

AssertViolated(3, "UsingDirectivesMustBePlacedWithinNamespace");

...or at the expected line numbers...

AssertViolated("UsingDirectivesMustBePlacedWithinNamespace", 1, 2, 3);

...or you may check against a caller-provided predicate function....

AssertViolated(@violation => @violation.Rule.CheckId == "SA1200");

...and if none of the above options fits your needs, you may directly access the generated violations and/or the generated text output:

Assert.AreEqual(3, Violations.Count);
Assert.AreEqual(2, Output.Count);

If such a test fails, then you might find something similar to this in the test report:

To make this work, the containing test fixture derives from a base class called StyleCopFixtureBase. This class takes care of setting up the StyleCop engine, (silently) running it against the specified files, capturing the resulting violations together with the generated console output, and cleaning up after each test run. It also provides the necessary methods and properties for adding files and additional addin path, and also some specialized assertion methods. To test a rule, you only have to specify the file(s) that you want to test, run the analysis, and then check the results. This is not only shamefully simple and easy to use (i.e. to write) - such unit tests are also a pleasure to read, in that they tell the reader clearly and unambiguously what they are doing...

In the usual use case scenario (the assembly with the rule(s) under test can be found in the calling assemblies' bin directory, together with the StyleCop assemblies), you only have to provide a StyleCop.Settings file [*] with your test project, which must be copied to the bin folder (i.e. the Copy to Output Directory property must be set to Copy always or Copy if newer), and you're done. If necessary, however, the StyleCopFixtureBase class also allows for eventually required path adjustments. I won't go further into details here - you can get the StyleCopFixtureBase class from the download of the sample code and then refer to the class' documentation, which is pretty detailed.

* Tip: You can copy the settings file from the StyleCop install directory and then edit it as required...

The Sample Solution

The sample solution (VS 2008) can be downloaded from here. It contains the StyleCopFixtureBase class along with detailed XML documentation and some basic usage examples. To compile and run, you need to have the Gallio framework installed (which you can get from here) and also Microsoft StyleCop (downloadable here). You may also have to adjust the project's assembly references accordingly, if you have StyleCop installed in a non-default path or you're using a different version of the tool (the sample solution uses 4.3.1.3).