Unit Testing Watcher

• Download source code - 17.4 KB

Introduction

This is a simple solution for monitoring and reporting unit testing results for apps written in C# 2.0 and stored in a SourceSafe database. I started writing the code and the article after reading some papers about command-line compilation in C# and command-line unit testing. I decided to write the code as part of an exercise with command-line tools, and also because I love unit testing, and I thought it would be great to experiment with a tool that could give me reports of my unit testing results. I'll be happy to listen to the readers' feedback - positive or negative - about the code and the article, and I'm willing to make improvements on the code while the readers respond to it.

Background

At first, the solution looks like a continuous integration tool, but in fact, it's much simpler, and doesn't cover the key aspects of a continuous integration process, such as automatic deployment and production environment testing. There are many tools that do what this TestWatcher does and much more. You can look for robust and continuous integration tools such as CruiseControl.NET and MSBuild. This may seem like I'm reinventing the wheel, and that's true in a sense. But, as I said before, this is just an experiment. In addition, you don't need to create zillions of configurations. The code is very small, and you can configure and extend it as you wish. I think you can easily modify it to support automatic deployment and test coverage, for example.

Test Watcher

In short, when you run Test Watcher, you are saying: "please check whether my projects on SourceSafe are passing their tests, and keep the in-charge people informed".

Test Watcher performs four main tasks, described in some detail:

1. Connects to SourceSafe and gets the latest versions of the projects

Test Watcher connects to SourceSafe using the SS API. It passes the login and password to SS, and gets the latest version of the assigned projects (both the base project and the test project). The code is then downloaded to a configurable folder.

2. Compiles the projects

Once we get the projects, we have to compile each of them. But first, we have to check which reference assemblies the project needs to be built with. For example, every test project needs a reference to the NUnit framework assembly. So, the project's configuration must inform Test Watcher to copy the NUnit framework assembly to the local build folder, so that it can be successfully compiled. We compile the C# classes through the C# command-line compiler, csc.exe (you might replace csc.exe by vbc.exe if you are trying to compile vb.net code). Each compilation step must produce an assembly DLL.

3. Runs unit tests on the test projects through NUnit, and save the results

The compilation of the the target assembly (which holds the code being tested) must be followed by the compilation of the test assembly (which checks whether the target assembly passes or fails). So, the unit testing step must skip the projects which are the target projects, and run only on the test projects.

Each unit testing step produces an XML file, which is then transformed into a humanly-readable summary. This summary shows how many tests were performed, how many passed, how many failed, and shows the failure details.

4. Sends email to project recipients, informing them of the test results

Finally, Test Watcher sends an email, reporting the summary tests, to the assigned project recipients. This is the final step and the main goal of the application.

Application design

The solution itself comprises of two projects: TestWatcherConsole and TestWatcherCore. The first is a Win32 console application, and basically controls the application flow and holds the configuration files, while the latter contains helpers that do the hard work. There are two more projects in the solution: Calculator and CalculatorTest. These projects don't take part in the application, but are kept together inside the solution just to test the application.

The code

TestWatcherConsole

TestWatcherConsole is (guess what..) a console project. It has just one class, Program, which contains the main application flow. It controls the Get Latest Version/Compile/Run NUnit/Send Mail process, according to the project's configuration. In each step, the TestWatcherConsole calls an appropriate method inside the TestWatcherCore assembly.

TestWatcherCore

This project contains the helpers and the models for the application.

The helpers are SSHelper, Compile, NUnitHelper, EmailHelper, and XMLHelper. They do the specialized work in the application.

The model classes are WatchSolution and WatchProject. WatchSolution represents a group (or a cycle) of WatchProjects, which in turn are a representation of C# projects that must be compiled, tested, and then their results are sent by email.

The calculator example

We will be using two projects, Calculator and CalculatorTest, to test the application. Calculator performs the four basic arithmetic operations, and the CalculatorTest project just checks whether the operations are performed correctly.

First of all, we must have these projects on SourceSafe, so that they can be downloaded to a local folder to be compiled:

Now that we have the projects on SourceSafe, we should tell Test Watcher how to deal with our projects. We do this through the WatchSolutions.xml file:

<?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot;?>
<ArrayOfWatchSolution 
    xmlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot; 
    xmlns:xsd=&quot;http://www.w3.org/2001/XMLSchema&quot;>
  <WatchSolution>
    <SSPath>$/CodeProject/TestWatcher</SSPath>
    <Includes>
      <string>
        C:\Arquivos de programas\NUnit 2.4.1\bin\nunit.framework.dll
      </string>
    </Includes>
    <WatchProjects>
      <WatchProject>
        <Name>Calculator</Name>
        <AssemblyFile>Calculator.dll</AssemblyFile>
        <Includes/>
        <MailRecipients/>
        <ReportOnlyInCaseOfFailure>false</ReportOnlyInCaseOfFailure>
        <IsNunitTarget>false</IsNunitTarget>
      </WatchProject>
      <WatchProject>
        <Name>CalculatorTest</Name>
        <AssemblyFile>CalculatorTest.dll</AssemblyFile>
        <Includes>
          <string>
            C:\Arquivos de programas\NUnit 2.4.1\bin\nunit.framework.dll
          </string>
          <string>Calculator.dll</string>
        </Includes>
        <MailRecipients>
          <string>mclricardo@gmail.com</string>
        </MailRecipients>
        <ReportOnlyInCaseOfFailure>false</ReportOnlyInCaseOfFailure>
        <IsNunitTarget>true</IsNunitTarget>
      </WatchProject>
    </WatchProjects>
  </WatchSolution>
</ArrayOfWatchSolution>

Let's explain each part of the configuration file:

For the WatchSolution class:

• SSpath: The SourceSafe solution path, which holds all the contained projects. It must always start with a dollar ($) sign. In our case, it's $/CodeProject/TestWatcher.
• WatchProjects: The array of WatchProjects (see below) contained by the solution and which should be built. Those projects are Calculator and CalculatorTest. They are arranged in an order so that CalculatorTest must be compiled only after Calculator has been compiled.
• Includes: An array of paths of references needed so that NUnit can run on the assemblies. In our case, there is just one assembly: nunit.framework.dll.

For the WatchProject class:

• Name: The name of the project in SourceSafe. It will be used in combination with the solution's SSPath.
• AssemblyFile: The name of the output DLL file generated by the compiler.
• IsNunitTarget: Boolean. Indicates whether the DLL of the project should be tested by NUnit. In our case, it's false for the Calculator project and true for CalculatorTest.
• Includes: An array of paths of assembly references needed so that the project can be compiled.
• MailRecipients: A list of email addresses who will receive the test results.
• ReportOnlyInCaseOfFailure: Indicates whether the recipients will always receive the summary email, or will receive email only in the case of a failure.

Application configuration

You should still modify your app.config file before running Test Watcher.

<?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot; ?>
<configuration>
  <appSettings>
    <add key=&quot;ssDBIniPath&quot; value=&quot;C:\vss\srcsafe.ini&quot;/>
    <add key=&quot;ssUser&quot; value=&quot;Admin&quot;/>
    <add key=&quot;ssPassword&quot; value=&quot;&quot;/>
    <add key=&quot;localPath&quot; value=&quot;C:\temp\TestWatcher\&quot;/>
    <add key=&quot;cscPath&quot; 
         value=&quot;C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\csc.exe&quot;/>
    <add key=&quot;nunitEXEPath&quot; 
         value=&quot;C:\Arquivos de programas\NUnit 2.4.7\bin\nunit-console.exe&quot;/>
    <add key=&quot;smtpHost&quot; value=&quot;smtp.gmail.com&quot;/>
    <add key=&quot;mailSender&quot; value=&quot;mclricardo@gmail.com&quot;/>
  </appSettings>
</configuration>

• ssDBIniPath: The path for the SourceSafe database .ini file.
• ssUser: The SourceSafe login that will be used by Test Watcher.
• ssPassword: The SourceSafe password for the above user.
• localPath: The temporary folder that will get the latest versions of the projects.
• cscPath: The path for the css.exe file (C# command-line compiler).
• nunitEXEPath: The path for the nunit-console.exe file (NUnit command-line executable).
• smtpHost: The SMTP host used to send the summary e-mails.
• mailSender: The e-mail address used to send the test results.

Now that you configured your app, and if everything went OK, you should receive an email like this:

Enjoy it!

History

• 2008-07-20: Initial posting.