Reflection Benchmark and Optimize

• Download source - 15.88 KB

Introduction

Reflection is one of the most powerful features of .NET. But it is also not forgiving regarding performance issues. This article tries to measure the performance penalties, by displaying average ticks per various operation kinds and their optimized versions. Generic types are included in the benchmark to measure MakeGenericType performance.

Background

For a project I have been developing over the last week, I have come to utilize a lot of reflection, generics and their combination using MakeGenericType.

So today being the last day, I began to optimize its performance based on my experience of reflection and the .NET world. I have used a lot of reflection so I believe my performance hunches were correct.

Assumptions

I assumed that:

1. It is a lot quicker to have a dictionary of Types instead of calling MakeGenericType
2. It is a lot quicker to have a dictionary of PropertyInfo instead of using GetPropetyInfo repeatedly
3. It is a lot quicker to have a dictionary of MethodInfo instead of using GetMethodInfo repeatedly
4. It is better to create static functions that will do your work as a single function, instead of calling through reflection the constructor and then the method

The Benchmark

So tonight, I created a Benchmarking program that proved I was right on all 4. What I didn't expect was the great time MakeGenericType needed to execute.

In the benchmark project, I have a TestType.cs without generics and a GenericType.cs with two generic parameters. Each class has a public constructor, an Execute method and a static ExecuteStatic which calls the constructor and the Execute method.

Because the operations are pretty simple, there will be no milliseconds but average ticks.

The names consist of the following:

1. Test or Generic class
2. With reflection or not
3. Instantiating and calling Execute or just call static ExecuteStatic
4. Optimized or not

So here are the results:

TestStatic->7
TestInstance->7
TestReflectionStatic->21
TestReflectionInstance->33
TestReflectionStaticOptimized->16
TestReflectionInstanceOptimized->24
GenericStatic->8
GenericInstance->7
GenericReflectionStatic->41
GenericReflectionInstance->75
GenericReflectionStaticOptimized->37
GenericReflectionInstanceOptimized->67
GenericReflectionMake->32
GenericReflectionMakeOptimized->12

The above example is with just one combination of generic parameters used. When I increase the number of generic combinations, the results become:

TestStatic->9
TestInstance->9
TestReflectionStatic->22
TestReflectionInstance->31
TestReflectionStaticOptimized->16
TestReflectionInstanceOptimized->23
GenericStatic->8
GenericInstance->9
GenericReflectionStatic->40
GenericReflectionInstance->71
GenericReflectionStaticOptimized->37
GenericReflectionInstanceOptimized->66
GenericReflectionMake->31
GenericReflectionMakeOptimized->19

Conclusion

As you can see, the performance weakens probably because of the dictionary and its long keys. String manipulation in .NET is not one of its strongest features.

Another thing is that in my classes, there is only one method, so minor as the difference in optimization as it may seem, keep in mind that the more methods you have in a type, the slower GetMethodInfo executes.

In my benchmark, I didn't implement test about properties but I firmly believe that they will not have any difference from methods.

Some Code

There is not much code in the post, because the optimization is pretty straight forward. Just reuse Type and MethodInfo as much as possible.

For example, the not-optimized way is...

public static void TestReflectionInstance()
        {
            Type type = typeof(TestType);
            object tt = type.GetConstructor(new Type[] { }).Invoke(null);
            type.GetMethod("Execute").Invoke(tt, null);
            Stopwatch sw = new Stopwatch();
            sw.Start();
            for (int i = 0; i < iterations; i++)
            {
                type = typeof(TestType);
                tt = type.GetConstructor(new Type[] { }).Invoke(null);
                type.GetMethod("Execute").Invoke(tt, null);
            }
            sw.Stop();
            if (Write)
                Console.WriteLine("TestReflectionInstance " + 
                        sw.ElapsedTicks.ToString());
            times["TestReflectionInstance"] += sw.ElapsedTicks;
        }

... and the optimized way is:

public static void TestReflectionInstanceOptimized()
        {
            Type type = typeof(TestType);
            ConstructorInfo ci = type.GetConstructor(new Type[] { });
            object tt = ci.Invoke(null);
            MethodInfo mi = type.GetMethod("Execute");
            mi.Invoke(tt, null);
            Stopwatch sw = new Stopwatch();
            sw.Start();
            for (int i = 0; i < iterations; i++)
            {
                tt = ci.Invoke(null);
                mi.Invoke(tt, null);
                //type.GetMethod("Execute").Invoke(tt, null);
            }
            sw.Stop();
            if (Write)
                Console.WriteLine("TestReflectionInstanceOptimized " + 
                        sw.ElapsedTicks.ToString());
            times["TestReflectionInstanceOptimized"] += sw.ElapsedTicks;
        }

What I believe?

My belief is that if you optimize your code well when using reflection, then the performance penalty is not so great but the advantages are many times more significant. I believe this must have been taken into account, when Microsoft developed WPF, WWF and LINQ.

Thoughts After This Article was Written

Thought 1

The Generic Benchmarking is not fair. The practical dilemma is that at any single point of execution, is it better to call MakeGenericType for a generic type or to find the type from a variety of pre-made types from a dictionary.

So I ran the test using the below codes:

Type type = typeof(GenericType<,>);

Type ii = makedTypes[type.FullName + "|int|int"];

The optimized version is as follows:

Type ii = makedTypes[type.FullName + "|int|int"];

The results were as follows:

GenericReflectionMake->26
GenericReflectionMakeOptimized->9

This shows that having pre-made generic types in a sort of list is much quicker than calling MakeGenericType.

Links

• Post from my blog

History

• 18^th April, 2008: Initial post