Make NDoc compile the code examples contained in your documentation using NLiterate

• Download demo project - 38.8 Kb

Introduction

NLiterate is a tool that allows to write C# documentation in the Literate Programming Style (LPS). It merges, compiles and runs the code snippets contained in the comments, keeping the documentation and the source code synchronized. NLiterate comes as a custom documenter for NDoc (see [3]), so you easily use it through the NDoc GUI.

Disclaimer: The intention of NLiterate is not to replace the C# code, far from that. It mainly is a tool to automate the compilation of code examples contained in the documentation.

About Literate Programming

The literate programming style developed by the mighty D.Knuth consists of writing source code and documentation together in the same file. A tool then automatically converts the file into both a pure source code file and into a documentation file with pretty-printed source code. The literate programming style makes it easier to ensure that the code examples in the book/documentation really compile and run and that they stay consistent with the text.

The source code for each example is broken up into parts. Parts can include references to other parts. An example often starts with a part that provides an outline for the entire computation, which is then followed by other parts that fill in the details.

For that task, D.Knuth has developed the WEB language (later on CWEB, FWEB) which was targeted for C (source code) and TeX (word processor).

"The structure of a software program may be thought of as a web that is made up of many interconnected pieces. To document such a program, we want to explain each individual part of the web and how it relates to its neighbors. The typographic tools provided by TEX give us an opportunity to explain the local structure of each part by making that structure visible, and the programming tools provided by Pascal make it possible for us to specify the algorithms formally and unambiguously. By combining the two, we can develop a style of programming that maximizes our ability to perceive the structure of a complex piece of software, and at the same time the documented programs can be mechanically translated into a working software system that matches the documentation." Extracted from The WEB System of Structured Documentation User Manual (see [1]).

A large piece of this introduction is extracted from the Boost Graph Library User Guide (see [2]). For further information on Literate Programming, refer to the Literate Programming Official Page (see [1]).

Examples of code in the documentation

C# documentation lets you integrate snippets of code in the documentation using the code tag. Ex:

/// <remarks>

/// This methods does something:

/// <code>

/// MyClass c = new MyClass();

/// c.SomeMethod();

/// </code>

/// </remarks>

On one hand, examples take an important place in a good documentation. A lot of users prefer to learn from examples rather from a "literal" description of the method (think about MSDN doc). On the other, examples in the documentation are difficult to maintain: they are not recompiled, not run and not tested. When the number of examples becomes large, it will take more and more time to test each of the examples.

An out-of-sync example will make users lose time and will do more harm than nothing. Therefore, the maintenance problem of examples is a critical one.

NLiterate to the rescue

NLiterate applies LPS to source code contained in C# comments. Therefore, it lets the documenter write interconnected snippets using the LPS style and it provides an automated tool to compile and execute those snippets.

QuickStart Example

In this example, we are going to build a simple "Hello World" code example. We suppose that this sample is used as an example for an imaginary XString class.

Writing the documentation

Let's start from the following empty documentation:

/// <example>

/// </example>

public XString
{...}

First, we write the Hello World sample:

/// <example>

/// <para>

/// <c>HelloWorld</c> is a classic piece of code.

/// </para>

/// <code id="helloworld" title="HelloWorld" compilable="true" language="cs">

/// using System;

/// public class HelloWorld

/// {

///     public static void Main(string[] argv)

///     {

///         Console.WriteLine("Hello Word");

///     }

/// }

/// </code>

There are several remarks on the code above:

• Note the attributes of the code section: id identifies the section, title is the title of the snippet, compilable tells NLiterate that this snippet should be compiled, that it is a root snippet, and language gives the programming language (cs for C#, vb for Visual Basic).

Sometimes, examples can become quite long and it would be interesting to "split up" the example in different parts that you could explain separately. For example, we would like to separate the body of the HelloWorld.Main method from the main code snippet:

/// <example>

/// <para>

/// <c>HelloWorld</c> is a classic piece of code.

/// </para>

/// <code id="helloworld" title="HelloWorld" compilable="true" language="cs">

/// using System;

/// public class HelloWorld

/// {

///     public static void Main(string[] argv)

///     {

///         {[mainbody Implementation of Main]}

///     }

/// }

/// </code>

/// <para>

/// The implementation of Main can be added later:

/// <code id="mainbody" title="HelloWorld.Main">

/// Console.WriteLine("Hello World");

/// </code>

/// </para>

Note that in the first snippet, we have replaced the body of the Main method by a reference to another snippet:

{[ref-id title...]}

This snippet will be merged in place of the reference in the previous snippet. You can continue like this to describe and build a tree of code snippet in your documentation.

Referencing assemblies

By default, NLiterate adds System.dll and the declaring assembly to the list of referenced assemblies by the compiler. If you need other assemblies, you can use the reference tag as follows:

/// <remarks>

/// <reference assembly="MyAssembly.dll"/>

/// </remarks>

...

The full code of this class is contained in the NLiterate project. Recompile the project with documentation enabled.

Run NLiterate in NDoc

First, you need to prepare NDoc to run NLiterate. This is done by the following steps:

1. Download the source code and recompile it.
2. Copy the NDoc.Documenter.Literate.dll assembly to the NDoc bin folder.

When you open NDoc, click on the Documenter list box and choose Literate. You can then simply launch project compilation in the NDoc GUI to launch the NLiterate compilation process.

Implementation

The NLiterate implementation is quite straightforward, thanks to the .NET Framework tools. In fact, a lot of the ingredients needed where already present in the framework:

• Extraction of the documentation is built-in in C#,
• Pretty printing is already done by tools like NDoc( see [3]),
• The documentation is compiled to (highly structured) XML files. Using XPath, snippets can be easily referenced and accessed.
• Regular expressions can be used to efficiently locate and replace reference in code snippets,
• Compiler for C# and VB are available in the Framework,
• NDoc provides an extensible documenter mechanism that can be used to wrap NLiterate into a custom documenter.

Having all those tools under the hood, the NLiterate execution logic is as follows:

1. Load a documentation file (this is done by NDoc),
2. Locate snippets that are "entry point" snippets, those are outer-most code snippets.
3. Find the reference to other snippets using regular expressions, extract those snippets from the XML, and recursively construct the tree of snippets.
4. Merge the snippets and compile using the adequate compile (using System.CodeDom.Compiler) namespace.

Load XML

Locate Snippets in the XML

Code sections that have the attribute compilable="true" are located by NLiterate. They are the root of the tree composed by the code and the reference it contains. For example, the code section with @id="main" is the root of a tree that has one child snippetid:

/// <code id="main" compilable="true">

/// {[snippetid Some comment]}

/// </code>

...
/// <code id="snippetid" >

/// ...

/// </code>

Such code sections are easily located using the following XPath expression: "descendant::code[@compilable='true']".

Build the tree

The syntax to create a reference is as follows:

{[snippetid Comments describing the snippets]}

where snippetid is the id of the reference snippet, while the result of the text is used to describe the snippet. All is enclosed in {[...]}. The references are matched using regular expressions and the Regex class. To build the regular expression, I recommend Expresso (see [4]):

private static Regex snippetRegex = new Regex(
    @"{\[(?<SNIPPETID>\w+\b).*?\]}",
    RegexOptions.IgnoreCase
    | RegexOptions.Multiline
    | RegexOptions.ExplicitCapture
    | RegexOptions.Compiled
    );

With this tool, location of the reference is a piece of cake:

string code; // the code snippet

foreach(Match m in snippetRegex.Matches(code))
{
    string refid = m.Groups["SnippetID"].Value;
    ...
}

We then apply again XPath queries to find the reference code section in the XML document. Note that if you are not happy with this notation, it can easily be modified by changing the regular expression.

Merge snippets

At this point, we have built a tree where each node contains the "raw" code and a list of reference snippets. Again, with regular expressions, merging the parts is straight-forward:

public String GetMergedCode()
{
    return snippetRegex.Replace(this.code,new MatchEvaluator(this.CodeMerger));
}
private string CodeMerger(Match m)
{ 
    // get id

    string cid = m.Groups["SnippetID"].Value;
    // find code snippet with the right id

    CodeSnippetNode child = findNodeByID(cid);
    // return child merged code

    return child.GetMergedCode()
}

Compile

Compilation is done using the System.CodeDom.Compiler classes. Rather than detailing this part, I recommend users to read the DotNetScript article (see [5]) on CodeProject.

History

• 27/04/2004: Added NDoc documenter implementation and some details in the introduction.

References

1. Literate programming home page
2. Boost Graph Library
3. NDoc, .NET documentation tool
4. Expresso, A tool chest for building and testing regular expressions for Microsoft Windows .NET.
5. Dot Net Script