XsdTidy beautifies the Xsd.exe output *with full DocBook .NET Wrapper*

• Download source files - 64.8 Kb
• Download demo project - 17.9 Kb

If you like this tool, support it by voting, if you don't like it, make your vote verbose...

Pre-Introduction

The XsdTidy tool has been entirely rebuild from scratch using CodeDom which is much easier to handle than Emit. The new version is called Refly and available at the Refly article.

Introduction

XsdTidy is a refactoring tool to overcome some silly limitations of the exceptional Xsd.exe (see [1]) tool provided with the .NET framework. More specifically, XsdTidy addresses the following problems:

• Name normalization: if your XSD schema is using lower case names or more generally non ".NET" normalized names, you will end up with types that will make the FxCop (see [2]) spit out hundreds of infractions.
• Fixed Array Sizes: xsd.exe handles multiple elements by creating an array. There is no problem when you are loading the data, but unfortunately this is not convenient if you want to populate a document since arrays do not support Add or Remove. XsdTidy uses ArrayList for more flexibility.
• Default Constructor: Xsd.exe does not care about providing a default constructor that initializes the fields with the proper values. This work can become very silly when the object structure is getting big.

XsdTidy achieves refactoring by recreating new classes for each type exported by the Xsd.exe tool using the System.Reflection.Emit namespace. It also takes care of "transferring" the Xml.Serialization attributes to the factored classes. Hence, the factored classes are more .NET-ish and still outputs the same XML. Moreover, there is no dependency between the refactored code and the original code.

As a nice application of the tool, a full .NET wrapper of the DocBook schema (see [3]) is provided with the project. This .NET wrapper lets write or generate DocBook XML easily with the help of Intellisense.

Fixing problems

Name conversion

The .NET standards define specific naming convention for all types of data: arguments should be camel case, function names capitalized, etc... This is really helpful to keep the framework consistent. Tools like FxCop help us stay on the "normalized" side.

This problem is tackled the dumb way: given a dictionary of "common" words, the class NameConformer tries to split a name in separate words, after that it renders it to the needed convention.

There is much room for improvement on the list of words and the algorithm to split the name, any contribution welcome.

FixedArraySize

Arrays are replaced by System.Collection.ArrayList which are much more flexible. Moreover, array fields are created by default using their default constructor. This is to economize you the hassle of creating a collection before using it.

Properties

Fields are hidden in properties, which is more convenient to use. Moreover, collection fields do not have set property according to FxCop rule.

public class testclass
{
    [XmlElement("values",typeof(int)]
    public int[] values;
}

becomes:

public class TestClass
{
    private ArrayList values;

    [XmlElement("values",typeof(int)]
    public ArrayList Values
    {
        get
        {
            return this.values;
        }
    }
}

System.Reflection.Emit

The System.Reflection.Emit namespace is truly and amazingly powerful, it enables you to create new types at runtime and execute them or store them to assemblies for further use. Unfortunately, there are not much tutorials and examples on this advanced topic. In this chapter, I will try to explain my limited understanding of this tool.

What is Emit?

The Emit namespace gives you the tools to write IL (Interpreted Language) instructions and compile them to types. Hence, you can basically do anything with Emit. A typical emit code will look like this:

// emit
ILGenerator il = ...;
il.Emit(OpCodes.Ldarg_0); 
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Stfld, fb);
il.Emit(OpCodes.Ret);

// C# equivalent
this.fb = value;

If you are a newcomer, it can look cryptic, but we'll try to explain the above a bit.

Where to start ?

The problem with Emit is that debugging is complicated: if you generate wrong IL code, the framework will not execute it, throwing an error without giving any clue.. Moreover, you usually don't have the time to learn the dozens of the code that are part of the OpCodes class. Therefore, it would be nice to always have some "model" IL and then try to implement it with Emit.

Hopefully, creating this model is easy! It is possible, using decompilers such as Reflector (see [4]), to read the IL code of any .NET assembly. The idea is simple: open a dummy project where you create the model class that needs to be factored, compile and use a decompiler to read the IL of your model and there you go...you have IL code!

Emit, the basics

I will cover some very basic facts about using Emit. As mentioned above, the most efficient way to learn is to work with a dummy project and Reflector on the side. We will see here how to make a basic C# statement in some instance method where value is the first argument and field is a member of the class.

if (value==null)
    throw new ArgumentNullException("value");
this.field=value;

Getting a ILGenerator

Usually, you start by creating an AssemblyBuilder, then a ModuleBuilder, then a TypeBuilder and finally you can add methods to the TypeBuilder using TypeBuilder.DefineMethod which returns a MethodBuilder. This instance is then used to retrieve an ILGenerator object which we use to output IL code:

MethodBuilder mb = ...;
ILGenerator il = mb.GetGenerator();

OpCodes

The OpCodes class contains all the IL operations. It has to be used in conjunction with ILGenerator.Emit as we will see in the following.

Arguments

Each time you call a method (static or non-static), the method arguments are accessible through OpCodes.Ldarg_0, OpCodes_1, ... In an instance method, OpCodes.Ldarg_0 is the "this" address.

Labels

Labels are used to make jumps in the IL code. You need to set up Labels if you want to build instructions such as if...else.... A Label is defined as follows:

Label isTrue = il.DefineLabel();

Once the Label is defined, it can be used in an instruction that makes a jump. When you reach the instruction that the Label should mark, call MarkLabel:

il.MarkLabel(isTrue);

Comparing values to null

Comparing a value to null is done using the OpCodes.Brtrue_S. This instruction makes a jump to a Label if the value is not null.

Label isTrue = il.DefineLabel();
il.Emit(OpCodes.Ldarg_1); // pushing value on the stack
il.Emit(OpCodes.Brtrue_S,isTrue); // if non null, jump to label
// IL code to throw an exception here
...
// marking label
il.MarkLabel(isTrue);
...

Creating objects

To create object, you must first retrieve the ContructorInfo of the type, push the constructor arguments on the stack and call the constructor using OpCodes.NewObj. If we use the default constructor of ArgumentNullException, we have:

ConstructorInfo ci = 
    typeof(ArgumentNullException).GetConstructor(Type.EmptyTypes);

Label isTrue = il.DefineLabel();
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Brtrue_S,isTrue);
il.Emit(OpCodes.NewObj,ci); // creating new exception
il.Emit(OpCodes.Throw); // throwing the exception
il.MarkLabel(isTrue);
...

You can clearly see the "jump across the exception" with the label isTrue.

Assigning fields

The last step is to assign the field with the value (stored in the first argument). To do so, we need to push the "this" address on the stack (OpCodes.Ldarg_0), push the first argument (OpCodes.Ldarg_1) and use OpCodes.Stdfld:

// Type t is the class type
FieldInfo fi = t.GetField("field");
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Stdfld,fi);

Finish the work

To close a method, use OpCodes.Ret:

il.Emit(OpCodes.Ret);

Refactoring

Main steps

The refactoring is handled by the XsdWrappedGenerator class. The main factoring steps are:

1. Create an AssemblyBuilder and define a new ModuleBuilder.
2. For each user-provided type that needs to be refactored, define a new TypeBuilder in the ModuleBuilder.
3. For public fields in the source type, generate a field in the refactored type.
4. Define default constructors for each factored class.
5. Add properties for each field in the factored types and copy the XML serialization attributes to the properties.

During the process of factoring, special care is taken about nullable/non nullable types and collection handling:

• collections are preallocated for easier use,
• non-nullable fields are allocated, nullable fields are left to null.
• non-nullable fields are always checked against zero, while nullable fields are not checked.

Once the factoring is finished, the types are created and saved to an assembly.

Using XsdWrappedGenerator

The XsdWrappedGenerator encapsulates all the "wrapping" functionalities: create a new instance, add the types you need to be refactored and save the result to a file:

XsdWrapperGenerator gen = new XsdWrapperGenerator(
    "CodeWrapper", // output namespace and assembly name
    new Version(1.0), // outputed assembly version
    );

// adding types
gen.AddClass( typeof(myclass) );
...

// refactor
gen.WrapClasses();
// save to file, this invalidates gen.
gen.Save();

The name passed to the constructor is used as default namespace and output assembly name.

Using the Command Line application

XsdWrapperGenerator comes with a minimal console application that loads an assembly, searches to types, refactors them and output the results. Calling convention is as follows:

XsdTidy.Cons.exe AssemblyName WrappedClassNamespace OutputNamespace Version

where

• AssemblyName is the name of the assembly to scan (without .dll)
• WrappedClassNamespace is the namespace from which the types are extracted
• OutputNamespace is the factored namespace
• Version is the version number: major.minor.build.revision

NDocBook

DocBook is an XML standard to describe a document. It is a very powerful tool since the same XML source can be rendered in almost all possible output formats: HTML, CHM, PDF, etc... This richness comes to a price: DocBook is complicated for the beginner and it tends to be XML-ish.

This was the starting of the article for me: I needed to generate DocBook XML to automatically generate code in GUnit (see [5]) but I wanted to take advantage of VS intellisense.

The first step was to generate the .NET classes mapping the DocBook schema using the Xsd.exe tool. The generated code had some problems that would make it unusable: non-nullable fields where not initialized automatically and this would lead to a lot of manual work.

Hence, the second was to write XsdTidy and apply it to DocBook. So here's an example of use:

// creating a book object
Book b = new Book();
// title is nullable, so we must allocate it
b.Title = new Title();
// text is a collection, preallocated
b.Title.Text.Add("My first book");
// nullable
b.Subtitle = new Subtitle();
b.Subtitle.Text.Add("A subtitle");

Toc toc = new Toc();
b.Items.Add(toc);
toc.Title = new Title();
toc.Title.Text.Add("What a Toc!");

Part part = new Part();
b.Items.Add(part);
part.Title = new Title();
part.Title.Text.Add("My first page");
            
// generate xml using XmlSerialization tools
using (StreamWriter sw = new StreamWriter("mybook.xml"))
{
    XmlTextWriter writer = new XmlTextWriter(sw);
    writer.Formatting = Formatting.Indented;
    XmlSerializer ser = new XmlSerializer(typeof(Book));
    ser.Serialize(writer,b);
}

The output of this snippet looks like this:

<?xml version="1.0" encoding="utf-8"?>
<book xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <title>My first book</title>
  <subtitle>A subtitle</subtitle>
  <toc>
    <title>What a Toc!</title>
  </toc>
  <part>
    <title>My first page</title>
  </part>
</book>

Now, with Intellisense on our side, I am much more comfortable with DocBook...

Conclusion

System.Reflection.Emit is a powerful tool that deservers more attention than it currently has. It can be used to generate optimized parsers (like Regex is doing), runtime typed DataSets, etc...

History

• 20/2/2004, initial try-out.

References

1. XSD Schema Definition Tool
2. FxCop
3. DocBook
4. Reflector
5. GUnit