Dynamic... But Fast: The Tale of Three Monkeys, A Wolf and the DynamicMethod and ILGenerator Classes

• Download source - 20.75 KB

Monkey Number One

Once upon a time, there were three little code monkeys. Monkey number one worked at the straw factory out on 7^th and Penn. One day the first monkey's boss, B. B. Wolf, brought him a new assignment. The HR department's application was being upgraded to C#. Monkey number one's job was to write the code to populate the Person class with data from the database. The monkey got straight to work and quickly produced code similar to the following (the actual code could not be used for legal reasons):

C#

public class ManualBuilder
{
    public Person Build(SqlDataReader reader)
    {
        Person person = new Person();

        if (!reader.IsDBNull(0))
        {
            person.ID = (Guid)reader[0];
        }

        if (!reader.IsDBNull(1))
        {
            person.Name = (string)reader[1];
        }

        if (!reader.IsDBNull(2))
        {
            person.Kids = (int)reader[2];
        }

        if (!reader.IsDBNull(3))
        {
            person.Active = (bool)reader[3];
        }

        if (!reader.IsDBNull(4))
        {
            person.DateOfBirth = (DateTime)reader[4];
        }

        return person;
    }
}

VB

Public Class ManualBuilder

    Public Function Build(ByVal reader As SqlDataReader) As Person
        Dim person As Person = New Person()

        If Not reader.IsDBNull(0) Then
            person.ID = CType(reader(0), Guid)
        End If

        If Not reader.IsDBNull(1) Then
            person.Name = CType(reader(1), String)
        End If

        If Not reader.IsDBNull(2) Then
            person.Kids = CType(reader(2), Integer)
        End If

        If Not reader.IsDBNull(3) Then
            person.Active = CType(reader(3), Boolean)
        End If

        If Not reader.IsDBNull(4) Then
            person.DateOfBirth = CType(reader(4), DateTime)
        End If

        Return person
    End Function

End Class

This code worked well at first. It was clean and really fast. However, the head of the HR department decided that the new software should really have a few additional features. Every new feature seemed to either require adding fields to the Person table, creating a new table, or moving fields from the Person table to one of the new tables. Every time a new feature was introduced, the monkey had to either edit his code or write mapping code for the new table. It seemed that the monkey was constantly the bottleneck for any new feature being worked on. One day, Mr. Wolf called the monkey into his office. Mr. Wolf huffed, and puffed, and laid the monkey off.

Monkey Number Two

Later, Mr. Wolf was let go from the straw factory due to allegations of improper conduct with Mrs. Pig. He ended up taking a new job at the lumber yard, which just happened to be where monkey number two worked. Mr. Wolf was hired by the lumber yard specifically because of his experience with upgrading HR applications which, interestingly, was exactly the type of project that the lumber yard was about to start. Unsurprisingly, monkey number two was given the task of writing the code to populate the Person class with the data from the database. Mr. Wolf informed monkey number two of the first monkey's fate and not-so-subtly implied that this would also be monkey number two's fate if he did not come up with a more flexible solution. The monkey thought about it for a while and produced something similar to the following:

C#

public class ReflectionBuilder
{
    private PropertyInfo[] properties;

    private ReflectionBuilder() { }

    public T Build(SqlDataReader reader)
    {
        T result = (T)Activator.CreateInstance(typeof(T));

        for (int i = 0; i < reader.FieldCount; i++)
        {
            if (properties[i] != null && !reader.IsDBNull(i))
            {
                properties[i].SetValue(result, reader[i], null);
            }
        }

        return result;
    }

    public static ReflectionBuilder<t> CreateBuilder(SqlDataReader reader)
    {
        ReflectionBuilder<t> result = new ReflectionBuilder<t>();

        result.properties = new PropertyInfo[reader.FieldCount];
        for (int i = 0; i < reader.FieldCount; i++)
        {
            result.properties[i] = typeof(T).GetProperty(reader.GetName(i));
        }

        return result;
    }
}

VB

Public Class ReflectionBuilder(Of T)
    Private properties() As PropertyInfo

    Private Sub ReflectionBuilder()

    End Sub

    Public Function Build(ByVal reader As SqlDataReader) As T
        Dim result As T = CType(Activator.CreateInstance(GetType(T)), T)
        Dim i As Integer

        For i = 0 To reader.FieldCount - 1
            If Not properties(i) Is Nothing And Not reader.IsDBNull(i) Then
                properties(i).SetValue(result, reader(i), Nothing)
            End If
        Next

        Return result
    End Function

    Public Shared Function CreateBuilder(ByVal reader As SqlDataReader) _
            As ReflectionBuilder(Of T)
        Dim result As ReflectionBuilder(Of T) = New ReflectionBuilder(Of T)()
        Dim i As Integer
        ReDim result.properties(0 To reader.FieldCount)

        For i = 0 To reader.FieldCount - 1
            result.properties(i) = GetType(T).GetProperty(reader.GetName(i))
        Next

        Return result

    End Function

End Class

This solution worked much better than the first monkey's solution. As you might have guessed, the requirements for the HR "upgrade" were constantly changing. "Add this feature," "remove that feature," "move this here," "move that there." None of this seemed to matter. Monkey number two's use of reflection meant that his code could automatically recognize the changes. Better yet, when new tables and objects were created, the same code could be used with no additional changes. All was going extremely well. Monkey number two felt certain that he was in line for a major promotion.

But then the unthinkable happened… the application went live. Suddenly Mr. Wolf was inundated with calls from unhappy HR employees complaining about how slooooow the new application was. After a few weeks, Mr. Wolf was let go for his incompetence and the lumber yard went back to using their old HR software. However, Mr. Wolf did manage to terminate monkey number two on his way out.

Monkey Number Three

Despite his apparent ineptitude as a project manager, Mr. Wolf quickly landed a new job down at the brick yard. The brick yard was in the process of migrating their old HR software and felt they could benefit from Mr. Wolf's "expertise." Coincidentally, the brick yard was also the employer of monkey number three. Once again, Mr. Wolf assigned the task of loading the Person class with data from the database to monkey number three and again he implied that the monkey's continued employment depended on not creating either of the issues that monkeys number one and two had created. Monkey number three did some research and stumbled upon the DynamicMethod and ILGenerator classes in .NET 2.0. These classes would allow the monkey to dynamically create and compile code at runtime. This would give him the best of both worlds. His code could be dynamic like monkey number two's, but since it was actually compiled, it would be as fast as monkey number one's.

He did some experimenting. The downside was that the dynamic code needed to be written using IL (intermediate language) instead of C#. However, with a small amount of Googling, some code decompiling using ildasm.exe from the .NET SDK, and some good old trial and error, the monkey was able to create code similar to the following:

C#

public class DynamicBuilder<T>
{
    private static readonly MethodInfo getValueMethod = 
        typeof(IDataRecord).GetMethod("get_Item", new Type[] { typeof(int) });
    private static readonly MethodInfo isDBNullMethod = 
        typeof(IDataRecord).GetMethod("IsDBNull", new Type[] { typeof(int) });
    private delegate T Load(IDataRecord dataRecord);
    private Load handler;

    private DynamicBuilder() { }

    public T Build(IDataRecord dataRecord)
    {
        return handler(dataRecord);
    }

    public static DynamicBuilder<T> CreateBuilder(IDataRecord dataRecord)
    {
        DynamicBuilder<T> dynamicBuilder = new DynamicBuilder<T>();

        DynamicMethod method = new DynamicMethod("DynamicCreate", typeof(T), 
                new Type[] { typeof(IDataRecord) }, typeof(T), true);
        ILGenerator generator = method.GetILGenerator();

        LocalBuilder result = generator.DeclareLocal(typeof(T));
        generator.Emit(OpCodes.Newobj, typeof(T).GetConstructor(Type.EmptyTypes));
        generator.Emit(OpCodes.Stloc, result);

        for (int i = 0; i < dataRecord.FieldCount; i++)
        {
            PropertyInfo propertyInfo = typeof(T).GetProperty(dataRecord.GetName(i));
            Label endIfLabel = generator.DefineLabel();

            if (propertyInfo != null && propertyInfo.GetSetMethod() != null)
            {
                generator.Emit(OpCodes.Ldarg_0);
                generator.Emit(OpCodes.Ldc_I4, i);
                generator.Emit(OpCodes.Callvirt, isDBNullMethod);
                generator.Emit(OpCodes.Brtrue, endIfLabel);

                generator.Emit(OpCodes.Ldloc, result);
                generator.Emit(OpCodes.Ldarg_0);
                generator.Emit(OpCodes.Ldc_I4, i);
                generator.Emit(OpCodes.Callvirt, getValueMethod);
                generator.Emit(OpCodes.Unbox_Any, dataRecord.GetFieldType(i));
                generator.Emit(OpCodes.Callvirt, propertyInfo.GetSetMethod());

                generator.MarkLabel(endIfLabel);
            }
        }

        generator.Emit(OpCodes.Ldloc, result);
        generator.Emit(OpCodes.Ret);

        dynamicBuilder.handler = (Load)method.CreateDelegate(typeof(Load));
        return dynamicBuilder;
    }
}

VB

Public Class DynamicBuilder(Of T)
    Private Shared ReadOnly getValueMethod As MethodInfo = _
        GetType(IDataRecord).GetMethod("get_Item", New Type() {GetType(Integer)})
    Private Shared ReadOnly isDBNullMethod As MethodInfo = _
        GetType(IDataRecord).GetMethod("IsDBNull", New Type() {GetType(Integer)})
    Private Delegate Function Load(ByVal dataRecord As IDataRecord) As T
    Private handler As Load

    Private Sub DynamicBuilder()

    End Sub

    Public Function Build(ByVal dataRecord As IDataRecord) As T
        Return handler(dataRecord)
    End Function


    Public Shared Function CreateBuilder(ByVal dataRecord As IDataRecord) _
        As DynamicBuilder(Of T)
        Dim dynamicBuilder As DynamicBuilder(Of T) = New DynamicBuilder(Of T)()
        Dim i As Integer

        Dim method As DynamicMethod = New DynamicMethod("DynamicCreate", GetType(T), _
            New Type() {GetType(IDataRecord)}, GetType(T), True)
        Dim generator As ILGenerator = method.GetILGenerator()

        Dim result As LocalBuilder = generator.DeclareLocal(GetType(T))
        generator.Emit(OpCodes.Newobj, GetType(T).GetConstructor(Type.EmptyTypes))
        generator.Emit(OpCodes.Stloc, result)

        For i = 0 To dataRecord.FieldCount - 1
            Dim propertyInfo As PropertyInfo = _
                GetType(T).GetProperty(dataRecord.GetName(i))
            Dim endIfLabel As Label = generator.DefineLabel()

            If Not propertyInfo Is Nothing Then
                If Not propertyInfo.GetSetMethod() Is Nothing Then
                    generator.Emit(OpCodes.Ldarg_0)
                    generator.Emit(OpCodes.Ldc_I4, i)
                    generator.Emit(OpCodes.Callvirt, isDBNullMethod)
                    generator.Emit(OpCodes.Brtrue, endIfLabel)

                    generator.Emit(OpCodes.Ldloc, result)
                    generator.Emit(OpCodes.Ldarg_0)
                    generator.Emit(OpCodes.Ldc_I4, i)
                    generator.Emit(OpCodes.Callvirt, getValueMethod)
                    generator.Emit(OpCodes.Unbox_Any, dataRecord.GetFieldType(i))
                    generator.Emit(OpCodes.Callvirt, propertyInfo.GetSetMethod())

                    generator.MarkLabel(endIfLabel)
                End If
            End If
        Next

        generator.Emit(OpCodes.Ldloc, result)
        generator.Emit(OpCodes.Ret)

        dynamicBuilder.handler = CType(method.CreateDelegate(GetType(Load)), Load)
        Return dynamicBuilder
    End Function
End Class

Mr. Wolf was skeptical, so monkey number three did his best to explain what was going on.

The first few lines of CreateBuilder instantiate the DynamicMethod and ILGenerator classes. In short, it's creating a new static method called DynamicCreate and adding that method to the object type that was passed in, i.e. the Person class in this example. The method takes SqlDataReader and returns an instance of the correct object. If this were non-dynamic code, you might call it like this:

C#

Person myPerson = Person.DynamicCreate(mySqlDataReader);

VB

dim myPerson as Person = Person.DynamicCreate(mySqlDataReader)

The next line of code generates a variable of the generic type. So this,

LocalBuilder result = generator.DeclareLocal(typeof(T));

in non-dynamic code would be this:

C#

Person myPerson;

VB

Dim myPerson as Person

The next piece of code instantiates the requested type of object and stores it in the local variable.

generator.Emit(OpCodes.Newobj, typeof(T).GetConstructor(Type.EmptyTypes));
generator.Emit(OpCodes.Stloc, result);

In non-dynamic code, it would be this:

C#

myPerson = new Person();

VB

myPerson = new Person()

The code then loops through the fields in the data reader, finding matching properties on the type passed in. When a match is found, the code checks to see if the value from the data reader is null.

C#

generator.Emit(OpCodes.Ldarg_0);
generator.Emit(OpCodes.Ldc_I4, i);
generator.Emit(OpCodes.Callvirt, isDBNullMethod);
generator.Emit(OpCodes.Brtrue, endIfLabel);

...

generator.MarkLabel(endIfLabel);

or

if (!mySqlDataReader.IsDBNull(1))
{
    ...
}

VB

If Not mySqlDataReader.IsDBNull(1) Then
    ...
End If

If the value in the data reader is not null, the code sets the value on the object.

generator.Emit(OpCodes.Ldloc, result);
generator.Emit(OpCodes.Ldarg_0);
generator.Emit(OpCodes.Ldc_I4, i);
generator.Emit(OpCodes.Callvirt, getValueMethod);
generator.Emit(OpCodes.Unbox_Any, dataRecord.GetFieldType(i));
generator.Emit(OpCodes.Callvirt, propertyInfo.GetSetMethod());

Again, in non-dynamic code, it would be this:

C#

myPerson.Name = (string)mySqlDataReader[1];

VB

myPerson.Name = CType(mySqlDataReader(1), string)

The last part of the code returns the value of the local variable:

C#

generator.Emit(OpCodes.Ldloc_0);
generator.Emit(OpCodes.Ret);

or

return myPerson;

VB

return myPerson

The code then returns a handler to a delegate. When this handler is invoked, it calls the dynamically generated code, which can be seen in this code:

C#

public T Build(SqlDataReader reader)
{
    return handler(reader);
}

VB

Public Function Build(ByVal dataRecord As IDataRecord) As T
    Return handler(dataRecord)
End Function

Mr. Wolf had no clue what any of this meant, but not wanting to look dumb, he said, "Sounds promising, but let's get some benchmarks before we move forward." Monkey number three quickly threw together a test to use all three approaches. Each sample would load three million rows out of the Person table. Here were the results:

Based on these results, Mr. Wolf had monkey number three implement his solution. Development went great. The live release went even better. The project was a huge success. It performed well and came in close to budget. Mr. Wolf received a huge bonus, retired early, and moved to a small private island. Monkey number three was later downsized and is currently unemployed.

Keep It Simple, Monkey

NOTE: This article is an extreme over-simplification. The code is intended to be an introduction to dynamic runtime code generation, not a full-blown solution. That being said, if you carefully and judiciously apply the ideas presented here, you should be able to be just as successful as monkey number three.

Historical Monkeys

• 7^th July, 2007 -- Original article version posted
• 13^th July, 2007 -- Article edited and moved to the main CodeProject article base
• 24^th July, 2007 -- Updated
• 20^th July, 2008 -- Updated to handle DBNull, to handle any IDataRecord (instead of just SqlDataReader), and added VB