Skip to main content
Email Password   helpLost your password?

Sample Image - xsdtidy.png

If you like this tool, support it by voting, if you don't like it, make your vote verbose...

Pre-Introduction

The XsdTidy tool has been entirely rebuild from scratch using CodeDom which is much easier to handle than Emit. The new version is called Refly and available at the Refly article.

Introduction

XsdTidy is a refactoring tool to overcome some silly limitations of the exceptional Xsd.exe (see [1]) tool provided with the .NET framework. More specifically, XsdTidy addresses the following problems:

XsdTidy achieves refactoring by recreating new classes for each type exported by the Xsd.exe tool using the System.Reflection.Emit namespace. It also takes care of "transferring" the Xml.Serialization attributes to the factored classes. Hence, the factored classes are more .NET-ish and still outputs the same XML. Moreover, there is no dependency between the refactored code and the original code.

As a nice application of the tool, a full .NET wrapper of the DocBook schema (see [3]) is provided with the project. This .NET wrapper lets write or generate DocBook XML easily with the help of Intellisense.

Fixing problems

Name conversion

The .NET standards define specific naming convention for all types of data: arguments should be camel case, function names capitalized, etc... This is really helpful to keep the framework consistent. Tools like FxCop help us stay on the "normalized" side.

This problem is tackled the dumb way: given a dictionary of "common" words, the class NameConformer tries to split a name in separate words, after that it renders it to the needed convention.

There is much room for improvement on the list of words and the algorithm to split the name, any contribution welcome.

FixedArraySize

Arrays are replaced by System.Collection.ArrayList which are much more flexible. Moreover, array fields are created by default using their default constructor. This is to economize you the hassle of creating a collection before using it.

Properties

Fields are hidden in properties, which is more convenient to use. Moreover, collection fields do not have set property according to FxCop rule.

public class testclass
{
    [XmlElement("values",typeof(int)]
    public int[] values;
}

becomes:

public class TestClass
{
    private ArrayList values;

    [XmlElement("values",typeof(int)]
    public ArrayList Values
    {
        get
        {
            return this.values;
        }
    }
}

System.Reflection.Emit

The System.Reflection.Emit namespace is truly and amazingly powerful, it enables you to create new types at runtime and execute them or store them to assemblies for further use. Unfortunately, there are not much tutorials and examples on this advanced topic. In this chapter, I will try to explain my limited understanding of this tool.

What is Emit?

The Emit namespace gives you the tools to write IL (Interpreted Language) instructions and compile them to types. Hence, you can basically do anything with Emit. A typical emit code will look like this:

// emit

ILGenerator il = ...;
il.Emit(OpCodes.Ldarg_0); 
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Stfld, fb);
il.Emit(OpCodes.Ret);

// C# equivalent

this.fb = value;

If you are a newcomer, it can look cryptic, but we'll try to explain the above a bit.

Where to start ?

The problem with Emit is that debugging is complicated: if you generate wrong IL code, the framework will not execute it, throwing an error without giving any clue.. Moreover, you usually don't have the time to learn the dozens of the code that are part of the OpCodes class. Therefore, it would be nice to always have some "model" IL and then try to implement it with Emit.

Hopefully, creating this model is easy! It is possible, using decompilers such as Reflector (see [4]), to read the IL code of any .NET assembly. The idea is simple: open a dummy project where you create the model class that needs to be factored, compile and use a decompiler to read the IL of your model and there you go...you have IL code!

Reflector

Emit, the basics

I will cover some very basic facts about using Emit. As mentioned above, the most efficient way to learn is to work with a dummy project and Reflector on the side. We will see here how to make a basic C# statement in some instance method where value is the first argument and field is a member of the class.

if (value==null)
    throw new ArgumentNullException("value");
this.field=value;

Getting a ILGenerator

Usually, you start by creating an AssemblyBuilder, then a ModuleBuilder, then a TypeBuilder and finally you can add methods to the TypeBuilder using TypeBuilder.DefineMethod which returns a MethodBuilder. This instance is then used to retrieve an ILGenerator object which we use to output IL code:

MethodBuilder mb = ...;
ILGenerator il = mb.GetGenerator();

OpCodes

The OpCodes class contains all the IL operations. It has to be used in conjunction with ILGenerator.Emit as we will see in the following.

Arguments

Each time you call a method (static or non-static), the method arguments are accessible through OpCodes.Ldarg_0, OpCodes_1, ... In an instance method, OpCodes.Ldarg_0 is the "this" address.

Labels

Labels are used to make jumps in the IL code. You need to set up Labels if you want to build instructions such as if...else.... A Label is defined as follows:

Label isTrue = il.DefineLabel();

Once the Label is defined, it can be used in an instruction that makes a jump. When you reach the instruction that the Label should mark, call MarkLabel:

il.MarkLabel(isTrue);

Comparing values to null

Comparing a value to null is done using the OpCodes.Brtrue_S. This instruction makes a jump to a Label if the value is not null.

Label isTrue = il.DefineLabel();
il.Emit(OpCodes.Ldarg_1); // pushing value on the stack

il.Emit(OpCodes.Brtrue_S,isTrue); // if non null, jump to label

// IL code to throw an exception here

...
// marking label

il.MarkLabel(isTrue);
...

Creating objects

To create object, you must first retrieve the ContructorInfo of the type, push the constructor arguments on the stack and call the constructor using OpCodes.NewObj. If we use the default constructor of ArgumentNullException, we have:

ConstructorInfo ci = 
    typeof(ArgumentNullException).GetConstructor(Type.EmptyTypes);

Label isTrue = il.DefineLabel();
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Brtrue_S,isTrue);
il.Emit(OpCodes.NewObj,ci); // creating new exception

il.Emit(OpCodes.Throw); // throwing the exception

il.MarkLabel(isTrue);
...

You can clearly see the "jump across the exception" with the label isTrue.

Assigning fields

The last step is to assign the field with the value (stored in the first argument). To do so, we need to push the "this" address on the stack (OpCodes.Ldarg_0), push the first argument (OpCodes.Ldarg_1) and use OpCodes.Stdfld:

// Type t is the class type

FieldInfo fi = t.GetField("field");
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Stdfld,fi);

Finish the work

To close a method, use OpCodes.Ret:

il.Emit(OpCodes.Ret);

Refactoring

Main steps

The refactoring is handled by the XsdWrappedGenerator class. The main factoring steps are:

  1. Create an AssemblyBuilder and define a new ModuleBuilder.
  2. For each user-provided type that needs to be refactored, define a new TypeBuilder in the ModuleBuilder.
  3. For public fields in the source type, generate a field in the refactored type.
  4. Define default constructors for each factored class.
  5. Add properties for each field in the factored types and copy the XML serialization attributes to the properties.

During the process of factoring, special care is taken about nullable/non nullable types and collection handling:

Once the factoring is finished, the types are created and saved to an assembly.

Using XsdWrappedGenerator

The XsdWrappedGenerator encapsulates all the "wrapping" functionalities: create a new instance, add the types you need to be refactored and save the result to a file:

XsdWrapperGenerator gen = new XsdWrapperGenerator(
    "CodeWrapper", // output namespace and assembly name

    new Version(1.0), // outputed assembly version

    );

// adding types

gen.AddClass( typeof(myclass) );
...

// refactor

gen.WrapClasses();
// save to file, this invalidates gen.

gen.Save();

The name passed to the constructor is used as default namespace and output assembly name.

Using the Command Line application

XsdWrapperGenerator comes with a minimal console application that loads an assembly, searches to types, refactors them and output the results. Calling convention is as follows:

XsdTidy.Cons.exe AssemblyName WrappedClassNamespace OutputNamespace Version

where

NDocBook

DocBook is an XML standard to describe a document. It is a very powerful tool since the same XML source can be rendered in almost all possible output formats: HTML, CHM, PDF, etc... This richness comes to a price: DocBook is complicated for the beginner and it tends to be XML-ish.

This was the starting of the article for me: I needed to generate DocBook XML to automatically generate code in GUnit (see [5]) but I wanted to take advantage of VS intellisense.

The first step was to generate the .NET classes mapping the DocBook schema using the Xsd.exe tool. The generated code had some problems that would make it unusable: non-nullable fields where not initialized automatically and this would lead to a lot of manual work.

Hence, the second was to write XsdTidy and apply it to DocBook. So here's an example of use:

// creating a book object

Book b = new Book();
// title is nullable, so we must allocate it

b.Title = new Title();
// text is a collection, preallocated

b.Title.Text.Add("My first book");
// nullable

b.Subtitle = new Subtitle();
b.Subtitle.Text.Add("A subtitle");

Toc toc = new Toc();
b.Items.Add(toc);
toc.Title = new Title();
toc.Title.Text.Add("What a Toc!");

Part part = new Part();
b.Items.Add(part);
part.Title = new Title();
part.Title.Text.Add("My first page");
            
// generate xml using XmlSerialization tools

using (StreamWriter sw = new StreamWriter("mybook.xml"))
{
    XmlTextWriter writer = new XmlTextWriter(sw);
    writer.Formatting = Formatting.Indented;
    XmlSerializer ser = new XmlSerializer(typeof(Book));
    ser.Serialize(writer,b);
}

The output of this snippet looks like this:

<?xml version="1.0" encoding="utf-8"?>
<book xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <title>My first book</title>
  <subtitle>A subtitle</subtitle>
  <toc>
    <title>What a Toc!</title>
  </toc>
  <part>
    <title>My first page</title>
  </part>
</book>

Now, with Intellisense on our side, I am much more comfortable with DocBook...

Conclusion

System.Reflection.Emit is a powerful tool that deservers more attention than it currently has. It can be used to generate optimized parsers (like Regex is doing), runtime typed DataSets, etc...

History

References

  1. XSD Schema Definition Tool
  2. FxCop
  3. DocBook
  4. Reflector
  5. GUnit
You must Sign In to use this message board.
 
 
Per page   
 FirstPrevNext
GeneralHelp, Update or Alternative? Pin
James Coleman
5:54 26 Feb '07  
GeneralRe: Help, Update or Alternative? Pin
trx1
6:23 10 Jul '08  
GeneralHow can I download this tool? Pin
boazru
6:27 27 Nov '05  
Generalcool tool..few questions/issues... Pin
Rama K
4:43 22 Jun '04  
GeneralRe: cool tool..few questions/issues... Pin
Jonathan de Halleux
11:43 23 Jun '04  
GeneralRe: cool tool..few questions/issues... Pin
rama_k
12:56 23 Jun '04  
GeneralRe: cool tool..few questions/issues... Pin
rama k
12:57 23 Jun '04  
GeneralRe: cool tool..few questions/issues... Pin
Jonathan de Halleux
13:05 23 Jun '04  
GeneralAlternative Implementation Pin
Eron Wright
19:49 20 Apr '04  
GeneralRe: Alternative Implementation Pin
Jonathan de Halleux
22:20 20 Apr '04  
GeneralFix for duplicate elements Pin
obe1line
5:40 29 Mar '04  
GeneralRe: Fix for duplicate elements Pin
Jonathan de Halleux
4:41 19 Apr '04  
GeneralWould love to try... Pin
Verdant123
19:04 10 Mar '04  
GeneralRe: Would love to try... Pin
Jonathan de Halleux
9:30 16 Mar '04  
GeneralRe: Would love to try... Pin
Magick93
7:19 20 Oct '07  
GeneralOne nice article per week! Pin
xxxyyyzzz
7:12 20 Feb '04  
GeneralRe: One nice article per week! Pin
Jonathan de Halleux
11:54 20 Feb '04  
GeneralHow does intellisense help DocBook authoring? Pin
Norman Walsh
2:31 20 Feb '04  
GeneralRe: How does intellisense help DocBook authoring? Pin
Jonathan de Halleux
6:52 20 Feb '04  
GeneralCool Pin
rondalescott
13:18 19 Feb '04  
GeneralRe: Cool Pin
Jonathan de Halleux
13:24 19 Feb '04  
GeneralRe: Cool Pin
Lowell Heddings
17:39 19 Feb '04  
GeneralRe: Cool Pin
Jonathan de Halleux
22:21 19 Feb '04  
GeneralRe: Cool Pin
Lowell Heddings
22:24 19 Feb '04  
GeneralRe: Cool Pin
Jonathan de Halleux
23:13 19 Feb '04  


Last Updated 1 Mar 2004 | Advertise | Privacy | Terms of Use | Copyright © CodeProject, 1999-2009