Click here to Skip to main content
Click here to Skip to main content

Continuous Integration using CruiseControl.NET, NANT, CVS, and NUnit

, 25 Sep 2006 CPOL
Rate this:
Please Sign up or sign in to vote.
A guide to configuring a Continuous Integration server.

Introduction

Recently, I set up a build system for a software development team working in .NET. I was interested in the Continuous Integration approach, where builds occur automatically as changes are detected in source control. My other goals were to keep my team's build process simple and understandable, and to group tasks in a way that makes sense and encourages reuse.

Most of my information was gained by reading of other developers' experiences documented on the web, but I could not find a single, comprehensive "newbie guide" on how to set up an entire system. My intention is to provide such an article, explain the logic behind my decisions, and perhaps ease the process for others implementing similar systems.

This article assumes that you have the requisite tools already set up. I used Visual Studio 2005, CruiseControl.NET 1.0.1, NAnt 0.85-rc3, and NUnit 2.2.7. Tortoise CVS 1.8.26 is used as a CVS client, and IIS is needed for CruiseControl.NET's Web Dashboard.

Background

The developer's toolset is usually composed of at least the following tools:

  • Source control repository to store code in a common location and allow multiple developers to share a code base (e.g., CVS, Subversion, VSS).
  • Unit testing framework to run unit tests which are written as part of the code development process and can flag unintentional changes to code (e.g., NUnit, JUnit).
  • Automated build tool to allow building from the command line and to perform related tasks (e.g., Ant, NAnt, MSBuild).
  • Continuous Integration tool which monitors for code changes and triggers builds (e.g., CruiseControl, Draco).

Build tools have increased in complexity over the years, and many overlap in functionality. As a result, implementation decisions can be endless. For example, should unit tests be run directly from CruiseControl, NAnt, or MSBuild? If email notification is used, how should this be wired up? The answers lie in the strengths and weaknesses of each tool, but implementing or investigating each one can be time-consuming.

Flow Control

To aid in understanding how this system works and what each component does, I've provided a typical scenario:

  1. Developer commits code changes/additions to CVS.
  2. CruiseControl.NET detects changes.
  3. CruiseControl.NET updates code on build machine.
  4. CruiseControl.NET invokes NAnt script.
  5. NAnt script builds application using devenv.com.
  6. NAnt script runs unit tests. Final CruiseControl.NET tasks support display of unit testing information.
  7. CruiseControl.NET manages output files for display in its Web Dashboard.
  8. CruiseControl.NET sends emails as appropriate.

Continuous Integration Tool: CruiseControl.NET

Of the two most common Open Source Continuous Integration tools, I chose CruiseControl.NET. I did not spend a lot of time comparing them, and my decision was based on the fact that some developers consider it more flexible. Its most useful features are its ability to poll source control for changes, and a powerful Web Dashboard (requires IIS) that provides a history of all builds and their details.

Initial Setup

The heavy lifting will involve setting up a project block in the ccnet.config file. Aside from that, you will probably want to turn on logging in the ccnet.exe.config file.

Automated Build Tool: NAnt

I found that running unit tests and builds in NAnt (invoked by CruiseControl) is more manageable than running them directly from CruiseControl. Packaging build and unit tests into a single NAnt script also allows for easier script debugging, and gives developers a tool which can build and test from the command line if desired. Because the default.build file is checked into CVS with the rest of the project, changing it will cause CruiseControl.NET to rebuild.

Both NAnt and CruiseControl.NET can run any executable and have tags dedicated to running unit tests. But, NAnt provides more support for ancillary tasks such as file management, and its flow control is more flexible.

Initial Setup

Set up a default.build NAnt file in the base directory of the project, which builds the project and runs NUnit tests. For the tests, I recommend Thea Burger's pattern that allows unit tests from multiple assemblies all to run to completion despite a possible failure in one assembly.

<target name="test" description="runs the unit tests" >
    <!-- test first Assembly. -->
    <exec program="nunit-console.exe" failonerror="false" 
                resultproperty="testresult.Main.exe">
        <arg value="RelativePath/Main.exe" />
        <arg value="/xml=UnitTest-Main.xml" />
    </exec>

    <!-- test second Assembly. -->
    <exec program="nunit-console.exe" failonerror="false" 
                  resultproperty="testresult.SecondaryAssembly.dll">
        <arg value="RelativePath/SecondaryAssembly.dll" />
        <arg value="/xml=UnitTest-SecondaryAssembly.xml" />
    </exec>

    <!-- Check the results properties and fail if necessary -->
    <fail message="Failures reported in unit tests." 
          unless="${int::parse(testresult.Main.exe)==0}" />
    <fail message="Failures reported in unit tests." 
          unless="${int::parse(testresult.SecondaryAssembly.dll)==0}" />
</target>

Compiler: MSBuild.exe or Devenv.com

These are the two options for building a .NET solution or project from NAnt or at the command line. I experimented with both, but used devenv.com because the current version of MSBuild cannot build setup files! MSBuild offers additional capabilities which overlap with the Ant/NAnt family, and some developers prefer it to devenv because it does not necessitate installation of Visual Studio on the build machine. However, the setup shortcoming was a deal breaker, and there was no need to use both MSBuild and NAnt in the process.

It's worth noting that some tools belong in the solution, in post-build events. An example of this is obfuscation, which occurs only as part of the Release build and needs to be run before constructing the setup files.

Other Tools: CVS and NUnit

For my project, the choices of CVS and NUnit were predetermined. CruiseControl provides similar tools for monitoring a variety of other code repositories as well. NUnit is prevalent, and has been used by my team on a number of projects.

More CruiseControl.NET Setup Pointers

The Merge Tag

Depending on how you wire up CruiseControl.NET, it might be necessary to "Merge" output from an invoked application, in order to get test results to appear in the Web Dashboard and in email messages. This is the case when invoking NUnit from NAnt as outlined above. When NAnt invokes NUnit, XML output is explicitly requested.

<exec program="nunit-console.exe" failonerror="false" 
              resultproperty="testresult.Main.exe">
    <arg value="RelativePath/Main.exe" />
    <arg value="/xml=UnitTest-Main.xml" /> <!-- keep xml output -->
</exec>

These files are then "merged" in ccnet.config.

<merge>
    <files>        
        <!-- these file names need to conform to those found in default.build -->
        <file>C:\cruisecontrol-dev\test\RegexDemo\UnitTest*.xml</file>
    </files>
</merge>

CruiseControl.NET recognizes the XML from NUnit (and a number of other apps as well) and knows what to do with it.

The Web Dashboard

Once all processes are running and visible, you can tweak the CruiseControl.NET Web Dashboard by changing dashboard.config or the XML, XSL, and CSS that shape its content. The email messages sent out by CruiseControl.NET are also controlled in the same way, by a separate set of files.

Summary

In the final analysis, the build process outlined here is configured in three places: CruiseControl.NET's ccnet.config, the NAnt default.build, and in the solution itself. CruiseControl is a great system, and we've found unexpected uses for it even in the first few weeks. I continue to be impressed with its flexibility.

Sample Code

The sample code contains an example of ccnet.config which builds a simple application with one unit test and no unusual dependencies. The sample app can be checked into CVS and used for testing. The default.build file is included as well.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Michael Mendelson

United States United States
Feel free to contact me directly about my articles.

Comments and Discussions

 
Questionbarcode suite for .NET Pinmembersteve7g5-Jan-12 21:58 
GeneralCCNet XmlConfig Plugin Pinmembervoid leenux();16-Apr-11 8:09 
GeneralFacing problem on cruisecontrol.net with cvs Pinmembershivakumar0199-Mar-11 21:17 
GeneralWorking Directory In Remote System! Pinmembers_9022-Oct-08 23:03 
Generalabout ci Pinmemberflyingchen1-Nov-07 5:20 
GeneralI would like to store the sucessful build in a different folder with version or label. Pinmemberkuncha18-Oct-07 2:13 
GeneralRe: I would like to store the sucessful build in a different folder with version or label. PinmemberAlex Rovner28-Jul-08 5:35 
GeneralGood, but needs more to be truely helpful. [modified] Pinmemberpoodull764-Oct-06 5:09 
Generaldefault.build Pinmemberpoodull764-Oct-06 5:10 
<?xml version="1.0"?>
<project name="Solution" default="rebuild">
 
     <!--
     *** The config target sets up any property variables needed for this, or
     sub targets.   Put any global properties here.   Specific variables, like
     Environment variables should go in the target they are need in only.
     Subtargets needed these variables can link to them with the depends=
     parameter.***
     -->
     <target name="config">
 
          <!-- The FrameworkDir is the directory where the MSBuild and .Net framework exists -->
          <property name="FrameworkDir" value="C:\windows\Microsoft.net\framework\v2.0.50727" unless="${property::exists('FrameworkDir')}"/>
          <echo message="FrameworkDir = '${FrameworkDir}'"/>
 
          <!-- The IDEDir is the directory where the VS IDE devenv.exe exists -->
          <property name="IDEDir" value="C:\Program Files\Microsoft Visual Studio 8\Common7\IDE" unless="${property::exists('IDEDir')}"/>
          <echo message="IDEDir = '${IDEDir}'"/>
 
          <!-- The nDocDir is the directory where nDocConsole.exe exists -->
          <property name="nDoc.bin" value="C:\software\nDoc\bin" unless="${property::exists('nDoc.bin')}"/>
          <echo message="nDoc.bin = '${nDoc.bin}'"/>
 
          <!-- The Configuration property tells the compiler what version to build -->
          <property name="Configuration" value="Debug" unless="${property::exists('Configuration')}"/>
          <echo message="Configuration = '${Configuration}'"/>
 
          <!-- The ArtifactDir holds the build logs and output files.   not binaries. -->
          <property name="ArtifactDir" value="${directory::get-current-directory()}\Artifact" unless="${property::exists('ArtifactDir')}"/>
          <echo message="ArtifactDir = '${ArtifactDir}'"/>
          <mkdir dir="${ArtifactDir}" failonerror="false"/>
          <delete >
               <fileset basedir="${ArtifactDir}">
                    <include name="**/*"/>
               </fileset>
          </delete>
 
          <!-- The CCNet property is used to assign some non-commandline settings when
          CCNet is used to drive this NAnt file. -->
          <if test="${property::exists('CCNet')}">
               <echo message="*** CCNET Override Property found! ***"/>
               <property name="MSBuildLogger"
               value="ThoughtWorks.CruiseControl.MsBuild.XmlLogger,C:\Program Files\CruiseControl.NET\server\ThoughtWorks.CruiseControl.MSBuild.dll"
                    unless="${property::exists('MSBuildLogger')}"/>
          </if>
          <ifnot test="${property::exists('CCNet')}">
               <property name="MSBuildLogger"
               value="ConsoleLogger,${FrameworkDir}\Microsoft.Build.Engine.dll"
                    unless="${property::exists('MSBuildLogger')}"/>
          </ifnot>
          <echo message="MSBuildLogger = '${MSBuildLogger}'"/>
 
          <!-- The nDoc.Output is the directory where nDoc output will be placed -->
          <property name="nDoc.Output" value="${directory::get-current-directory()}\Doc" unless="${property::exists('nDoc.Output')}"/>
          <echo message="nDoc.Output = '${nDoc.Output}'"/>
 
          <!-- The nDoc.Build is the directory where nDoc temporary build files will be placed -->
          <property name="nDoc.Build" value="${directory::get-current-directory()}\Doc\Build" unless="${property::exists('nDoc.Build')}"/>
          <echo message="nDoc.Build = '${nDoc.Build}'"/>
          <mkdir dir="${nDoc.Build}" failonerror="false"/>
          <delete >
               <fileset basedir="${nDoc.Build}">
                    <include name="**/*"/>
               </fileset>
          </delete>
 
          <!-- The install.Output is the directory where Install target output will be placed -->
          <property name="install.Output" value="${directory::get-current-directory()}\Deploy" unless="${property::exists('install.Output')}"/>
          <echo message="install.Output = '${install.Output}'"/>
          <delete >
               <fileset basedir="${install.Output}">
                    <include name="**/*"/>
               </fileset>
          </delete>
     </target>
 
     <!--
     *** The 'get' target gets the latest version from sourcesafe ***
     -->
     <target name="get">
          <!--Sets the local environment variable SSDir-->
          <setenv name="ssdir" value="\\pigeon\VSS\APXX\"/>
 
          <exec program="C:\Program Files\Microsoft Visual SourceSafe\SS.EXE" useruntimeengine="true">
               <!--<arg value="get '$/Spiral 4' -R -YCCNET,CCNET -I-N -W- -GF- -GTM -GWR" />-->
               <arg value="get" />
               <arg value="$/Spiral 4" />
               <arg value="-R" />
               <arg value="-YCCNET,CCNET" />
               <arg value="-I-N" />
               <arg value="-W" />
               <arg value="-GF-" />
               <arg value="-GTM" />
               <arg value="-GWR" />
          </exec>
     </target>
 
     <target name="clean" depends="config">
          <exec program="${FrameworkDir}\MSBuild.exe" useruntimeengine="true"
               output="${ArtifactDir}\msbuild-${target::get-current-target()}-results.xml">
               <arg value="Implementation.sln"/>
               <arg value="/target:Clean" />
               <arg value="/noconsolelogger" if="${property::exists('CCNet')}" />
               <arg value="/nologo" if="${property::exists('CCNet')}" />
               <arg value="/p:Configuration=${Configuration}" />
               <arg value="/verbosity:normal" />
               <arg value="/logger:${MSBuildLogger}" unless="${MSBuildLogger==''}"/>
          </exec>
     </target>
 
     <target name="build" depends="config">
          <exec program="${FrameworkDir}\MSBuild.exe" useruntimeengine="true"
               output="${ArtifactDir}\msbuild-${target::get-current-target()}-results.xml">
               <arg value="Implementation.sln"/>
               <arg value="/target:Build" />
               <arg value="/noconsolelogger" if="${property::exists('CCNet')}" />
               <arg value="/nologo" if="${property::exists('CCNet')}" />
               <arg value="/p:Configuration=${Configuration}" />
               <arg value="/verbosity:normal" />
               <arg value="/logger:${MSBuildLogger}" unless="${MSBuildLogger==''}"/>
          </exec>
     </target>
 
     <target name="test" depends="config">
 
          <exec program="${FrameworkDir}\MSBuild.exe" useruntimeengine="true"
               output="${ArtifactDir}\msbuild-${target::get-current-target()}-results.xml">
               <arg value="Implementation.sln"/>
               <arg value="/target:Rebuild" />
               <arg value="/noconsolelogger" if="${property::exists('CCNet')}" />
               <arg value="/nologo" if="${property::exists('CCNet')}" />
               <arg value="/p:Configuration=Test" />
               <arg value="/verbosity:normal" />
               <arg value="/logger:${MSBuildLogger}" unless="${MSBuildLogger==''}"/>
          </exec>
         
          <nant target="${target::get-current-target()}">
               <buildfiles>
                    <include name="**/*.build" />
                    <exclude name="default.build" />
               </buildfiles>
          </nant>
     </target>
 
     <!-- There is no good way to build MSI files from NAnt yet.   WIX is too
     complex.   A hack is to open the IDE and have it build the 'Installers'
     configuration-->
     <target name="install" depends="config">
          <!-- Installer Configuration builds all installers.   Have to manually
          add this configuration setting because command-line arguements cannot be
          overwritten.   ie, setting D:Configuration=Release would cause this to build
          the Release config, despite any re-assignments in this script. -->
          <echo message="Configuration = 'Installers'"/>
 
          <exec program="${IDEDir}\devenv.exe" >
               <arg value="Implementation.sln"/>
               <arg value="/build" />
               <arg value="Installers" />
               <arg value="/out" />
               <arg value="${ArtifactDir}\installer.build.txt" />
          </exec>
 
          <copy todir="${install.Output}" flatten="true">
               <fileset>
                    <include name="**/Release/*.msi" />
               </fileset>
          </copy>
     </target>
 
     <target name="doc" depends="config">
          <copy todir="${nDoc.Build}" flatten="true">
               <fileset>
                    <include name="**/*.xml" />
                    <include name="**/*.dll" />
                    <include name="**/*.exe" />
               </fileset>
          </copy>
          <!-- Calling the .nDoc xml file does not work with nDoc 1.3.1851.0
          <exec program="${nDoc.bin}\nDocConsole.exe" useruntimeengine="true" >
               <arg value="-project=APEX.ndoc" />
               <arg value="-verbose" />
          </exec>-->
 
          <exec program="${nDoc.bin}\nDocConsole.exe" useruntimeengine="true" >
               <!-- Projects -->
               <arg value="${nDoc.Build}\Agent.HM.HMClient.Module.dll,${nDoc.Build}\Agent.HM.HMClient.Module.xml" />
               <arg value="${nDoc.Build}\Agent.HM.HMClient.Shell.exe,${nDoc.Build}\Agent.HM.HMClient.Shell.xml" />
               <arg value="${nDoc.Build}\Agent.HM.HMConsole.exe,${nDoc.Build}\Agent.HM.HMConsole.xml" />
               <arg value="${nDoc.Build}\Agent.HM.HMEngine.dll,${nDoc.Build}\Agent.HM.HMEngine.xml" />
               <arg value="${nDoc.Build}\Agent.HM.HMLibrary.dll,${nDoc.Build}\Agent.HM.HMLibrary.xml" />
               <arg value="${nDoc.Build}\Agent.HM.HMService.exe,${nDoc.Build}\Agent.HM.HMService.xml" />
               <arg value="${nDoc.Build}\Analysis.dll,${nDoc.Build}\Analysis.xml" />
               <arg value="${nDoc.Build}\Agent.HM.HMClient.Common.dll,${nDoc.Build}\Agent.HM.HMClient.Common.xml" />
               <arg value="${nDoc.Build}\Vecmath.dll,${nDoc.Build}\Vecmath.xml" />
               <arg value="${nDoc.Build}\Utility.dll,${nDoc.Build}\Utility.xml" />
               <arg value="${nDoc.Build}\TDService.exe,${nDoc.Build}\TDService.xml" />
               <arg value="${nDoc.Build}\TDServerLibrary.dll,${nDoc.Build}\TDServerLibrary.xml" />
               <arg value="${nDoc.Build}\TDServer.exe,${nDoc.Build}\TDServer.xml" />
               <arg value="${nDoc.Build}\TDMessages.dll,${nDoc.Build}\TDMessages.xml" />
               <arg value="${nDoc.Build}\TDLibrary.dll,${nDoc.Build}\TDLibrary.xml" />
               <arg value="${nDoc.Build}\TDConsole.exe,${nDoc.Build}\TDConsole.xml" />
               <arg value="${nDoc.Build}\SocketClientLibrary.dll,${nDoc.Build}\SocketClientLibrary.xml" />
               <arg value="${nDoc.Build}\ScanEngineLib.dll,${nDoc.Build}\ScanEngineLib.xml" />
               <arg value="${nDoc.Build}\Scan Engine GUI.exe,${nDoc.Build}\Scan Engine GUI.xml" />
               <arg value="${nDoc.Build}\RMService.exe,${nDoc.Build}\RMService.xml" if="${file::exists('${nDoc.Build}\RMService.xml')}" />
               <arg value="${nDoc.Build}\RMServerLibrary.dll,${nDoc.Build}\RMServerLibrary.xml" />
               <arg value="${nDoc.Build}\RMServer.exe,${nDoc.Build}\RMServer.xml" />
               <arg value="${nDoc.Build}\RMMessages.dll,${nDoc.Build}\RMMessages.xml" />
               <arg value="${nDoc.Build}\RMLibrary.dll,${nDoc.Build}\RMLibrary.xml" />
               <arg value="${nDoc.Build}\RMConsole.exe,${nDoc.Build}\RMConsole.xml" />
               <arg value="${nDoc.Build}\RMClientLibrary.dll,${nDoc.Build}\RMClientLibrary.xml" />
               <arg value="${nDoc.Build}\Resources.dll,${nDoc.Build}\Resources.xml" />
               <arg value="${nDoc.Build}\MssClient.dll,${nDoc.Build}\MssClient.xml" />
               <arg value="${nDoc.Build}\EnergyHistoryInterface.exe,${nDoc.Build}\EnergyHistoryInterface.xml" />
               <arg value="${nDoc.Build}\DevicesLibrary.dll,${nDoc.Build}\DevicesLibrary.xml" />
               <arg value="${nDoc.Build}\DataLibrary.dll,${nDoc.Build}\DataLibrary.xml" />
               <arg value="${nDoc.Build}\Controls.dll,${nDoc.Build}\Controls.xml" />
               <arg value="${nDoc.Build}\CalibrationLibrary.dll,${nDoc.Build}\CalibrationLibrary.xml" />
               <arg value="${nDoc.Build}\Calibration.exe,${nDoc.Build}\Calibration.xml" />
               <arg value="${nDoc.Build}\Blackbird.dll,${nDoc.Build}\Blackbird.xml" />
               <arg value="${nDoc.Build}\BetaModules.dll,${nDoc.Build}\BetaModules.xml" />
               <arg value="${nDoc.Build}\AnalysisModules.dll,${nDoc.Build}\AnalysisModules.xml" />
 
               <!-- Documentor -->
               <arg value="-documenter=MSDN" />
               <arg value="-OutputDirectory=${nDoc.Output}\MSDN" />
               <arg value="-Title='APEX Documented Class Library'" />
               <arg value="-BinaryTOC=True" />
               <arg value="-CopyrightText=Copyright (c) Agent Technologies 2003-2006&#xD;&#xA;&#xD;&#xA;All Rights Reserved.   Reproduction, adaptation, or translation without prior written permission is prohibited, except as allowed under the   copyright laws.&#xD;&#xA;&#xD;&#xA;Agent Technologies, Inc.&#xD;&#xA;395 Page Mill Road, Palo Alto, CA 94303-0870, U.S.A.&#xD;&#xA;" />
               <arg value="-CleanIntermediates=True" />
 
               <!-- Linear_Html creates one large html file.
               <arg value="-documenter=Linear_Html" />-->
               <!-- JavaDoc not working in nDoc 1.3.1851.0
               <arg value="-documenter=JavaDoc" />-->
          </exec>
 
          <delete>
               <fileset>
                    <include name="${nDoc.Build}\*.*"/>
               </fileset>
          </delete>
     </target>
 
     <target name="mangle" depends="config">
          <delete >
               <fileset basedir="Dotfuscator/${Configuration}/Dotfuscated">
                    <include name="**/*"/>
               </fileset>
          </delete>
          <exec program="${FrameworkDir}\MSBuild.exe" useruntimeengine="true"
               output="${ArtifactDir}\msbuild-${target::get-current-target()}-results.xml">
               <arg value="Implementation.sln"/>
               <arg value="/target:Rebuild" />
               <arg value="/noconsolelogger" if="${property::exists('CCNet')}" />
               <arg value="/nologo" if="${property::exists('CCNet')}" />
               <arg value="/p:Configuration=Dotfuscator" />
               <arg value="/verbosity:normal" />
               <arg value="/logger:${MSBuildLogger}" unless="${MSBuildLogger==''}"/>
          </exec>
          <call target="replaceDlls"/>
 
     </target>
 
     <!-- Replaces dlls from the Dotfuscated folder to any instance of them elsewhere.
          This is the only way I could get our install packages working without re-
          writing the whole thing.
     -->
     <target name="replaceDlls">
          <foreach item="File" property="source">
               <in>
                    <items>
                         <include name="Dotfuscator/${Configuration}/Dotfuscated/**" />
                    </items>
               </in>
               <do>
                    <echo message="looking for: ${path::get-file-name(source)}." />
 
                    <foreach item="File" property="destination">
                         <in>
                              <items>
                                   <include name="**/${Configuration}/${path::get-file-name(source)}" />
                                   <exclude name="Dotfuscator/**" />
                              </items>
                         </in>
                         <do>
                              <echo message="found copy in ${destination}." />
                              <copy file="${source}" tofile="${destination}" overwrite="true"/>
                         </do>
                    </foreach>
               </do>
          </foreach>
     </target>
 

     <target name="rebuild" depends="clean,build"/>
 
     <target name="all" depends="clean,build,test,doc,install"/>
 
     <target name="*">
          <echo message="Target '${project::get-name()}.${target::get-current-target()}' not implemented."/>
     </target>
</project>
Generalccnet.config Pinmemberpoodull764-Oct-06 5:18 
GeneralRe: Good, but needs more to be truely helpful. PinmemberMichael Mendelson5-Oct-06 20:45 
GeneralRe: Good, but needs more to be truely helpful. Pinmembershivakumar01910-Mar-11 20:00 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.141220.1 | Last Updated 25 Sep 2006
Article Copyright 2006 by Michael Mendelson
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid