Dynamic Code Generation Using CodeDOM

• Download source code - 92.1 KB

Why generate code?

As the name suggests, dynamic code generation is the technique to generate code at run-time. We write code which would generate code when run. As simplistic the definition seems, it is indeed one of the most powerful techniques in the repertoire of software developers. It is widely used in ORM - Object Relational Mapping - in fact, if you try Googling for samples on dynamic code generation, quite a few of the articles you would find would have ORM as sample. Besides dynamically mapping objects to scalar database entities, code generation can also be used to provide a plug-in framework in our applications which enables users to enhance features in the basic application. Code generation is an important instrument for programming methodologies like Generative Programming and Aspect Oriented Programming.

We can also use this to let the computer do the boring task of writing repetitive code. Almost all real word applications talk to some database, for example. The code required to connect to a database, read data, manipulate data, etc., is more or less the same even when the underlying database and database engines are different. How many of us have not had "deja vu" moments when we write classes like "Employee" with a "Save" method which reads all the properties of the class and passes it to some database class to persist... And how many us have not wished that this dull task is somehow done auto-magically. Patterns, patterns everywhere - that is one of the basic ideas behind code generation. Most of the tasks we perform on a daily basis have certain patterns and can be categorized. Code generation techniques often build upon this idea and separate the "difference" between "similar" things, extract the "common stuff", and provide a way to write similar stuff based on common stuff.

To borrow from Pragmatic Programmer's wisdom, basically, any time you have code that can be derived from metadata (a database schema, XSD, grammar description), you should write a tool that generates the code based on the metadata. Your overall maintenance burden will be lower, because you can just change the metadata as needed and regenerate the code. It may appear that writing such a tool would be difficult or would require programming skills which are not so common. But this is not correct. As with everything else, code generation has never been easy than on the .NET platform.

About our sample

So much so for preamble, now it is time to take a sneak peek at two of the code generation techniques in the .NET world. The ultimate goal of code generation is to get it running, isn't it? CodeDOM lets you create source code. That has to be then dynamically compiled into an assembly so that it can be run in the CLR. Reflection.Emit shortcuts this process by letting you write code which directly "emits" an assembly (and potentially an incorrect one - more on this later).

My goal in writing this article is to present the basics of these two techniques with a sample which uses both to generate the same assembly. I guess it would provide a nice contrast of these two techniques. We would do so in two parts. This is the first part which would use CodeDOM to generate code. The second part using Relection.Emit would follow soon.

Our sample assembly would have two classes. One would be derived from other. We would apply some attributes all through (to assembly, to class, to method). The derived class would have a method override of the base class.

Though there are numerous articles available on this topic, the sample I describe here covers what I could not find in any sample and had to figure out myself. (Not rocket science, but I would feel good if I save someone few hours figuring these out.)

On a final note - our goal is to generate a valid .NET assembly using both CodeDOM and Reflection.Emit. We are targeting .NET Framework 2.0. APIs could be slightly different for 1.1, but I have not checked this.

Part I of II - CodeDOM

CodeDOM (or Code Document Object Model) is a mechanism provided by the .NET Framework which lets us generate source code in multiple languages using a single model. We create code graphs and use the methods provided for CodeDOM to generate code in a language of our choice. Then we can use dynamic code compilation classes (also provided by CodeDOM) to generate assemblies which can then be loaded and used dynamically. The .NET Framework includes code generators and code compilers for C#, JScript, and Visual Basic. To read more about CodeDOM, check MSDN here.

It may be redundant to say this, but before we attempt code generation, we should be absolutely clear what exactly we want to generate. Writing our desired code as a rough draft may be very helpful in our early ventures in code generation because it is all too easy to miss something otherwise.

Anatomy of CodeDOM

Let us assume that we now know our "target" code to be generated. All our efforts would be concentrated on producing our desired code using the API provided by CodeDOM. Along the way, we would learn a few nitty-gritties of this process. Here is the structure of the solution we would be producing as we go:

• DynamicCodeGeneration.Base - This is the base class I talked about. Does nothing much. Our generated code will refer to this assembly and would have a class derived from this base class.
• DynamicCodeGeneration.CodeGenerator - This is our sweet little code generator which would provide us the UI and logic to generate the desired code using CodeDOM or IL.
• DynamicCodeGeneration.CustomAttributes - This project defines a few custom attributes which would apply to our target assembly/class/method while generating the code.

I found the application of custom attributes a little tricky so I thought I should include this in my sample.

Let's get going!

First things first!

Let us first write our base class.

using System;
using System.Collections.Generic;
using System.Text;

namespace DynamicCodeGeneration.Base
{
    /// <summary>
    /// Our base class
    /// </summary>
    public class Base
    {
        /// <summary>
        /// Base method which does nothing and would be overridden by derived class
        /// </summary>
        public virtual void Method()
        {
        }
    }
}

Simple :)

Custom attributes

I have written three custom attributes, the names are self-explanatory:

• AssemblyLevelAttribute - Sets AssemblyAuthorName and GenerationMode on the assembly
• ClassLevelAttribute - Sets the ClassAuthorName on the class
• MethodLevelAttribute - Sets the MethodAuthorName and ComplexityLevel (how complex is the method - isn't it great if by reading an attribute, we know before hand if the method is complex or not :)

These attributes would be applied to our generated assembly at the appropriate level.

CodeGenerator

I am just being lazy here in design. I have written an interface IGenerator which defines a method called GenerateAssembly. We would implement this interface twice - one for CodeDOM and a second time for IL. The presentation layer is the UI class (the WinForm which lets user input values for some of the parameters of custom attributes). It has a method which generates the assembly in the desired fashion using factory pattern. Simple to follow architecture (if we can call this architecture :).

Here is the IGenerator interface we would implement in a while.

Let us implement this interface for CodeDOM generation.

The class name I have chosen is CSharpGenerator. We would need the following references:

using System;
using System.Collections.Generic;
using System.Text;
using System.CodeDom;
using Microsoft.CSharp;
using System.CodeDom.Compiler; 
using System.IO;

Notice the ones in bold-italics - CodeDom and CodeDom.Compiler provide the necessary API to build the code graph and build it. Microsoft.CSharp provides the generator which uses the code graph to generate code in C#.

Let us come straight to the centre of the action - the GenerateAssembly method. Using CodeDOM, it is possible to generate code from various CodeDOM "constructs":

1. CompileUnit - Equivalent to Assembly - top level construct
2. Namespace
3. Type - Equivalent to class
4. Expression
5. Statement

Since we want to apply some attributes to the assembly, we would use the first method. This would require us to create the code graph having a code compile unit as the root, and other constructs (namespace, type, expression, etc.) would be nested within the unit.

A. CodeCompileUnit

A CodeCompileUnit provides a container for a CodeDOM program graph. We need to specify any referenced assemblies by adding to ReferencedAssemblies. We also need to provide any attributes by (creating and) adding to AssemblyCustomAttributes.

For adding any custom attribute, we need to first create it. The CodeAttributeDeclaration class takes charge of applying an attribute. It takes a CodeTypeReference (which is nothing but a reference to the class which defines the attribute; needs to be fully qualified) and an array of CodeAttributeArguments which would be passed to the constructor of the attribute class. They have to be in the same order as expected by the constructor of the custom attribute.

So the process in steps would be:

1. Define a new code compile unit.
2. Create a CodeAttributeArgument array of size as the number of parameters in the Attribute constructor.
3. Define each argument using the constructor of CodeAttributeArgument which takes a CodeExpression. CodeDOM provides several types of expressions.
4. Create a CoedAttributeDeclaration which would take the CodeTypeReference of the attribute class and the array of the code attribute arguments.
5. Add this CodeAttributeDeclaration to the code compile unit defined in #1.
6. Add all the referenced assemblies to the code compile unit.

#region Unit

CodeCompileUnit unit = new CodeCompileUnit();
CodeAttributeArgument[] arguments = new CodeAttributeArgument[2];
arguments[0] = new CodeAttributeArgument(
    new CodePrimitiveExpression(assemblyAuthorName));//Create parameter for attribute 

arguments[1] = new CodeAttributeArgument(
    new CodeSnippetExpression("DynamicCodeGeneration.CustomAttributes.GenerationMode.CodeDOM"));
CodeAttributeDeclaration assemblyLevelAttribute = new CodeAttributeDeclaration(
    new CodeTypeReference("DynamicCodeGeneration.CustomAttributes.AssemblyLevelAttribute"),
    arguments);//Create attribute to be added to assembly
unit.AssemblyCustomAttributes.Add(assemblyLevelAttribute);
unit.ReferencedAssemblies.Add("DynamicCodeGeneration.Base.dll");
unit.ReferencedAssemblies.Add("System.dll");
unit.ReferencedAssemblies.Add("DynamicCodeGeneration.CustomAttributes.dll");
unit.ReferencedAssemblies.Add("Microsoft.Crm.SdkTypeProxy.dll");

#endregion

B. Namespace

Next in hierarchy is the target namespace. This is defined as follows:

1. Define a new namespace using the CodeNamespace class. Takes the namespace names as the constructor parameter.
2. Add the namespace to the Namespaces collection of the code compile unit defined in section #A.
3. Add the required references by adding CodeNamespaceImport statements to the Imports collection of the code compile unit.

#region Namespace

CodeNamespace customEntityRoot = new CodeNamespace("DerivedRoot");//Create a namespace
unit.Namespaces.Add(customEntityRoot);

customEntityRoot.Imports.Add(new CodeNamespaceImport("System"));//Add references
customEntityRoot.Imports.Add(new CodeNamespaceImport(
                 "DynamicCodeGeneration.Base"));//Add references
customEntityRoot.Imports.Add(new CodeNamespaceImport(
                 "DynamicCodeGeneration.CustomAttributes"));//Add references

#endregion

C. Class (Type)

After defining the namespace, the next logical task is to define our derived class. We would also apply the class level custom attribute to this class. This class is derived from the base class defined above.

1. Create a new CodeTypeDeclaration. Its constructor takes the name of the type (classes are types).
2. Add this newly created type to the namespace defined in section #B.
3. Now we need to add a constructor to our class. For this:
1. Create a new CodeConstructor.
2. Set the attribtute(s) (scope - public/private etc.).
3. Add this code constructor to the Members collection of the type defined above.
4. Now we would apply the class level attribute to this type. Steps are similar to the ones we performed for defineing the assembly level attribute in section #A.
5. Finally, since this is a derived class, we need to add the base type which describes the class from which this type (class) is derived. For this, we add a new CodeTypeReference to the base class of the BaseTypes collection of the derived type (class).

#region Class

CodeTypeDeclaration derived = new CodeTypeDeclaration("Derived");//Create class
customEntityRoot.Types.Add(derived);//Add the class to namespace defined above

CodeConstructor derivedClassConstructor = new CodeConstructor();//Create constructor
derivedClassConstructor.Attributes = MemberAttributes.Public;
derived.Members.Add(derivedClassConstructor);//Add constructor to class

CodeAttributeArgument argument = new CodeAttributeArgument(
    new CodePrimitiveExpression(classAuthorName));

CodeAttributeDeclaration classLevelAttribute = 
    new CodeAttributeDeclaration(
        new CodeTypeReference("DynamicCodeGeneration.CustomAttributes.ClassLevelAttribute"), 
            argument);//Create attribute to be added to class
derived.CustomAttributes.Add(classLevelAttribute);

derived.BaseTypes.Add(new CodeTypeReference("Base"));

#endregion

D. Method

What use is a blank class? We now would add a method to our type defined above which would override the method in the base class.

1. Define a new CodeMemberMethod.
2. Add the required attributes (this is not the same as the meta data attribute - but defines the "prefixes" applied to the method definition). Since we need our method to be public and want it to override the base class' method, we would add MemberAttributes.Public and MemberAttributes.Override to the CodeMemberMethod defined above. This is a bit field and need a bitwise OR for adding multiple attributes.
3. We can also add CodeCommentStatements to the Comments collection of the method.
4. Set the Name of the method.
5. Define the return "type". Ours is void.
6. Define the method level attribute as we defined above in section #B and section #C.
7. Now that we have created the skeleton of the method ,we need to add muscle to it. For this, we can add multiple statements to the Statements collection of the CodeMemberMethod. Here we have added a new CodeSnippetStatement (this is literally sent to the output stream during code generation).
8. Finally, we need to add this CodeMemberMethod to the Members collection of the type defined in section #C.

#region Method

CodeMemberMethod derivedMethod = new CodeMemberMethod();
derivedMethod.Attributes = MemberAttributes.Public | MemberAttributes.Override;
 //Make this method an override of base class's method
derivedMethod.Comments.Add(new CodeCommentStatement(new CodeComment("TestComment")));
derivedMethod.Name = "Method";
derivedMethod.ReturnType = new CodeTypeReference(typeof(void));

arguments = new CodeAttributeArgument[2];
arguments[0] = new CodeAttributeArgument(
    new CodeSnippetExpression("ComplexityLevel.SuperComplex"));
    //Create parameter for attribute
arguments[1] = new CodeAttributeArgument(
    new CodePrimitiveExpression(methodAuthorName));
CodeAttributeDeclaration methodLevelAttribute = new CodeAttributeDeclaration(
    new CodeTypeReference("DynamicCodeGeneration.CustomAttributes.MethodLevelAttribute"), 
    arguments);//Create attribute to be added to method

derivedMethod.CustomAttributes.Add(methodLevelAttribute);//Add attribute to method

CodeSnippetStatement code = new CodeSnippetStatement("base.Method();");
derivedMethod.Statements.Add(code);
derived.Members.Add(derivedMethod);//Add method to the class

#endregion

E. Generation

Now our code graph is ready and we can generate code in the language of our choice. That is the beauty of CodeDOM. Even though we are writing the code generation code in C#, we can use the same code graph to generate code in VB.NET or other supported languages. Let us generate code in C# for now.

1. Create a C# code provider. This is going to provide us the generator method which would use our code graph to generate code in C#.
2. Create a code generator from the code provider.
3. We need a stream where the code generator would output the code. We are using a StringWriter and a StringBuilder here.
4. Create an object of CodeGeneratorOptions. This is used by the generator for settings like bracing style, etc. Bracing style "C" keeps the braces on the next line. I like it that way :)
5. Use one of the five methods to generate code. As I explained earlier, we can generate code from various constructs. In this example, we would use the root - code compile unit to generate code, and GenerateCodeFromCompileUnit would be the method used.
6. Build the generated code.

CSharpCodeProvider codeProvider = new CSharpCodeProvider();
ICodeGenerator codeGenerator = codeProvider.CreateGenerator();
StringBuilder generatedCode = new StringBuilder();
StringWriter codeWriter = new StringWriter(generatedCode);

CodeGeneratorOptions options = new CodeGeneratorOptions();
options.BracingStyle = "C";
    //Keep the braces on the line following the statement or 
    //declaration that they are associated with
codeGenerator.GenerateCodeFromCompileUnit(unit, codeWriter, options);

this.Code = generatedCode.ToString();

return BuildGeneratedCode();

G. Building the code

The last step is of course building the generated code into a .NET assembly.

1. Create a provider of the appropriate language. C# here.
2. Create the compiler.
3. Create the compiler parameters and add the referenced assemblies to the ReferencedAssemblies collection.
4. You have the option to generate the assembly in memory and execute it directly from memory. For now, we are going to generate the assembly on disk.
5. Use the CompileAssemblyFromSource method to build code.
6. Compiler results will tell if the build was successful or not.

CSharpCodeProvider codeProvider = new CSharpCodeProvider();
ICodeCompiler codeCompiler = codeProvider.CreateCompiler();

CompilerParameters parameters = new CompilerParameters();
parameters.ReferencedAssemblies.Add("DynamicCodeGeneration.Base.dll");
parameters.ReferencedAssemblies.Add("System.dll");
parameters.ReferencedAssemblies.Add("DynamicCodeGeneration.CustomAttributes.dll");
parameters.GenerateInMemory = false;

CompilerResults results = codeCompiler.CompileAssemblyFromSource(parameters, this.Code);
if (results.Errors.HasErrors)
{
    string errorMessage = "";
    errorMessage = results.Errors.Count.ToString() + " Errors:";
    for (int x = 0; x < results.Errors.Count; x++)
    {
        errorMessage = errorMessage + "\r\nLine: " + 
            results.Errors[x].Line.ToString() + " - " + results.Errors[x].ErrorText;
    }
    return errorMessage;
}
return results.PathToAssembly;

This concludes our endeavor. We now have the assembly which was built using the code we generated. The generated code will look like:

//------------------------------------------------------------------------------
// <auto-generated>
//     This code was generated by a tool.
//     Runtime Version:2.0.50727.42
//
//     Changes to this file may cause incorrect behavior and will be lost if
//     the code is regenerated.
// </auto-generated>
//------------------------------------------------------------------------------

[assembly: DynamicCodeGeneration.CustomAttributes.AssemblyLevelAttribute("Neo",
 DynamicCodeGeneration.CustomAttributes.GenerationMode.CodeDOM)]

namespace DerivedRoot
{
    using System;
    using DynamicCodeGeneration.Base;
    using DynamicCodeGeneration.CustomAttributes;
    
    
    [DynamicCodeGeneration.CustomAttributes.ClassLevelAttribute("Morpheus")]
    public class Derived : Base
    {
        public Derived()
        {
        }
        
        // TestComment
        [DynamicCodeGeneration.CustomAttributes.MethodLevelAttribute(
            ComplexityLevel.SuperComplex, "Trinity")]
        public override void Method()
        {
            base.Method();
        }
    }
}

The assembly would be generated in temp location.

This concludes the first part of this article in which we saw how we can leverage the CodeDOM API to generate code in a language of our choice with the same code graph.

References

1. Using the CodeDOM by Nick Harrison
2. Microsoft .NET CodeDom Technology - Part 2 by Brian J. Korzeniowski
3. Dynamically Executing Code in .NET by Rick Strahl
4. Compiling with CodeDOM by Gustavo Bonansea

Further Reading

1. CodeDOM patterns by Oman van Kloeten
2. Tonnes on articles at Code Generation Network
3. Introduction to Creating Dynamic Types with Reflection.Emit
4. EmitHelper

Disclaimer

While writing these articles, I have freely borrowed from several resources available online (freely). Though I have made every attempt not to leave any article in my references list, I might have inadvertently missed some citations. Please do bring to my notice such omissions and I would be glad to include the same in my article references.