Agile Sandbox (Part II)

Introduction

This article covers a selection of best practices related to the internals of the project “sandbox”. The sandbox is understood as a folder with a complete structure containing the source files of a specific product.

This is Part II of “Agile Sandbox” article. Make sure you've read Part I.

Code Base: Detailed Review

As previously mentioned in this article, CodeBase is a sandbox in a sandbox designed for working from Visual Studio. The main rule that we know now regarding Visual Studio and this folder – never try swimming against the current. Let me try to explain the rule using a couple of examples:

• At first, we tried to edit csproj files manually in order to reroute output and intermediate folders into the corresponding Output folder of the sandbox. This way, we tried to get rid of the default bin and obj folders within each C# project. We found modifying settings for each new project exhausting; sometimes typos caused the output to appear in totally wrong locations; however, the default obj folder continued to rise from ashes when Visual Studio was working – so is its behavior and it cannot be changed.
• Fine. Next time we added the obj folder to the SVN ignore-list to avoid monotonous and fallible work of changing csproj and created Visual Studio project templates with all the information included. Seemed like the problem was solved, but the answer was ”no” again. First, we had to distribute these templates among developers and explain that they needed to use them only. This task cannot be easily automated. Secondly – and in full accordance with Murphy’s law – if no custom template was created for a specific project type (e.g. Smart Device Application), this template would eventually be required for the project. So we had to take the source templates, clone the existing one, customize it, build a new template and send it to all of our developers.
• All right. If you have a lot of time, you can create all the necessary templates one by one. But even in this case the problem remains. First of all, regardless of what they say, you can’t know and plan everything ahead. There will come a moment when you will need to change the structure of the Output folder and all hell will break loose: several dozen existing C# projects will have to be adjusted to these changes, csproj will have to be modified manually, you'll have to adapt all the templates, send them to programmers and spend another day fixing typos.
• I have already described the problems related to reconfiguring the output folder, but this is not the only aspect. We also tried making all new projects automatically use the signature key in the root of the CodeBase folder; tried moving the general part of AssemblyInfo.cs to a separate ProductInfo.cs file and made new projects read data from it. Nonetheless, we kept getting all kinds of problems all the time. In the example above, the problems were still the same as with reconfiguring the output folder plus more: attempts to refer to a parent folder (for instance, ..\ ProductInfo.cs) from the csproj template resulted in numerous messages from the Visual Studio security system when the solution was opened. I doubt that such behavior will be much appreciated by your potential customers receiving the source code.
• Finally, any tricks you use and any side-effects that may take place need to be explained to all new users of the CodeBase folder, which is tiresome.

The thesis is the following: inside CodeBase, agree with Visual Studio in everything. Let it create the AssemblyInfo.cs it deems correct, let it put the binaries into the standard bin and obj folders, don’t intervene. Just make sure to add all the side products of Visual Studio to the SVN-ignore-list: *.user, *.suo, bin, obj (by the way, VisualSVN does it all automatically). The contents of the CodeBase folder are intended for developers only and are not real builds. Real ones with correct assembly metadata, keys and other attributes can be created by NAnt. That’s what we are going to take a look at below.

Code Base: Naming Projects, Folders and Files

Practice shows that it’s better to name projects in the following manner: YourCompanyName.ProductName.Qualifier. As namespace and assembly name are initially inserted into the project name, they will automatically comply with Microsoft guidelines if this rule is observed.

This approach has the following advantages:

• It will be easy to define filters for batch processing of assemblies, such as “all assemblies of my company”, “all assemblies of my product” and “all third-party assemblies”. For instance, YourCompanyName.*.dll or YourCompanyName.ProductName.*.dll.
• It is easier to visually identify your projects in Solution Explorer or the file browser.
• Of course, the case of conflicting library names is not possible if there are references to these libraries.

Another useful rule: the structure of namespaces must match the folder structure and the names of source files must match the types they define. Observing this rule will eliminate any ambiguities in juxtaposing type names and files, namespaces and folders. Such juxtaposition may be required for build automation with NAnt. In fact, in order to stick to this rule, you just don’t need to hamper Visual Studio: it automatically generates namespaces according to the following formula: RootNamespace.Level1Subdirectory.LevelNSubdirectory. In our case, RootNamespace is YourCompanyName.ProductName.Qualifier, therefore the resulting namespace will have a proper form.

If a source file defines only one type and the name of the file is the same, Visual Studio also associates them with one another and will, for instance, rename (refactor) the type if the file is renamed. The only exception pertains to enums and interfaces. They are all defined at once in Enumerations.cs and Interfaces.cs respectively. Their definitions are always compact and, so allocating a separate file to each of them is unpractical – visually, it only complicates the structure and makes it less readable.

A couple of words about unit tests. Or, to be specific, about the projects containing them. The names of such projects correspond to the names of “tested projects” and have a “Tests” qualifier. For example, a test project for YourCompanyName.ProductName.Core would be called YourCompanyName.ProductName.Core.Tests. This scheme allows you to easily filter out all assemblies with tests using the *.Tests.dll mask knowing that the tested and test projects are stored together in Solution Explorer. Ironically, we concluded that we had to break the above namespace/folder matching rule inside test projects. Not that you need to create anarchy in them, no. Let me use an example. Imagine a project called Company.Project.Core. It will be tested by Company.Project.Core.Tests with the same root namespace. Everything’s fine when types from the root namespace are tested. The Company.Project.Core.SomeTypeToTest type will be available from the Company.Project.Core.Tests namespace according to .NET rules. Now try imagining a type called Company.Project.Core.Network.Helpers.SomeTypeToTest. It won’t be available from Company.Project.Core.Tests or Company.Project.Core.Tests.Network.Helpers, which will be suggested separately. You will have to add a number of “using” directives to the Test Fixture file:

• using Company.Project.Core.Network.Helpers
• using Company.Project.Core.Network

Their number will increase along with the depth of the namespace. For the example above, the “right” namespace for Test Fixture is Company.Project.Core.Network.Helpers.Tests. This helps avoid creating multiple “using” directives, be more flexible in terms of refactoring, although forcing you to correct the default namespace while creating any file of level 2 and deeper in a test project.

Code Base: How Many Solutions Do I Need

There are plenty of holy wars being waged about it on the Internet. Some claim that only one file is needed for all projects; others advocate for a “one solution – one deployment entity” approach or an intermediate variant. We use one solution per group of projects, which form something useful and self-sufficient for a specific type of consumers. If it’s an online game, then one sln file contains all the server-related stuff, the other one - the end-user client and the third one holds the level editor. Of course, different solutions can use the same projects. This approach causes no problems, except for one: during automatic refactoring that changes the interface of types in common projects, these changes will become known only to those consumers who are defined in projects belonging to the currently opened solution. Other solutions will show something like “error CS0246: The type or namespace name ‘Foo’ could not be found” during the build process.

In practice, this has never been a major problem. We quickly made changes manually. Although in difficult cases, you may want to create a temporary .sln file for all the projects.

Code Base: Other Thoughts

You could also notice 2 subfolders in CodeBase: Libraries and MSBuildExtensions. MSBuildExtensions contains all the .targets and .dll files of MSBuild, which are not available in the default .NET Framework package. For instance, if you need to build an ASP.NET application in Visual Studio 2003 style under version 2005, you will need the Microsoft.WebApplication.targets extension; that’s the right place for it. In reality, the contents of this folder are not used for work within CodeBase. They are required for building with NAnt to guarantee the possibility of building on any system. We’ll talk about it later, however.

The Libraries category contains all third-party .NET assemblies used in the product. Please note that it contains absolutely all libraries, even those ones that have a nice and comfortable installer to the GAC. This is related to providing independence from the machine used for building. That is why any reference in your project is either an assembly from the standard .NET framework or a reference to an assembly in the Libraries folder.

I also want to note that even the nunit.framework.dll assembly is stored here. Building test projects and their testing are totally different things. That is why nunit.framework.dll is separated from the main distributive located in the mentioned BuildTools folder.

An assembly can end up being in Libraries only if it is available from the manufacturer in a precompiled form or if it was built without any changes being made to the source codes. In other cases, its source files should be placed into CodeBase along with your own ones and be a part of the build process. Why would you need that? It’s simple: if you do not save the history of changes in the library source files, then, when an update is released, you will face serious problems trying to combine “your” and “their” modifications.

In fact, modifying third-party libraries is a separate topic. The more drastic your changes are and the longer the time since your first modification, the greater the possibility that you will be unable to combine your own changes and those of the manufacturer. That is why, each time when you are going to “slightly fix” the source codes of a third-party product, think: is it a quick hack for satisfying your personal needs or a useful change to complement and improve the manufacturer’s functionality. If it’s a hack, try to forget about it - sooner or later this decision will yield some pretty bad results. If it’s something useful, make sure you do the following: implement your changes in the product’s style, make a patch and send to the manufacturer. The next version will feature your changes by default, and the source codes can be replaced by a precompiled library. So the goal is to get rid of the source codes of “alien” libraries.

To be continued: Part III.