Click here to Skip to main content
13,900,427 members
Click here to Skip to main content
Add your own
alternative version

Tagged as


49 bookmarked
Posted 30 Jan 2011
Licenced CPOL

A High Performance Binary Serializer using Microsoft Common Intermediate Language

, 1 Feb 2011
Rate this:
Please Sign up or sign in to vote.
A high performance binary serializer using Microsoft Common Intermediate Language

Basic Concepts

  • Serialization: A mechanism to transform the state of an object into a persistable format.
  • Deserialization: Restore the state of an object from a persistable format.
  • Binary serialization: Serialization technique to transform the state of an object into a binary stream.


Microsoft .NET provides a binary serializer in the System.Runtime.Serialization.Formatters.Binary namespace. Here is a simple example of how that works:

using System;
using System.Text;
using System.IO;
using System.Runtime.Serialization.Formatters.Binary;
namespace ConsoleApplication1
    class TestClass
        public String Name;
    class Program
        static void Main(string[] args)
            TestClass oT = new TestClass();
            oT.Name = "Hello Bin Serializer";
            MemoryStream ms = new MemoryStream();
            BinaryFormatter bf = new BinaryFormatter();
            bf.Serialize(ms, oT);

The binary stream generated by this formatter is 173 bytes long, and when converted to characters looks like:

Version=, Culture=neutral, 
0Name\0\0\0\0\0\0Hello Bin Serializer\v"

173 bytes just to serialize a simple class !!

That may not be suitable for high performance applications with a low memory budget.


The good news is that you can write a custom surrogate class that can serialize and deserialize your class into/from a binary stream.

The bad news is that you need to write this surrogate for every class you need to serialize.

And this is where .NET IL and the System.Reflection.Emit.ILGenerator come to the rescue. Using this class and a good working knowledge of MSIL, you can auto generate serialization surrogates on the fly. Here are the basic steps:

  1. Define a custom attribute that you can use to tag the class you want to generate serialization surrogates for.
  2. During the startup of your assembly, walk through all the types that have this custom attribute and generate a serialization surrogate for them.
  3. Create a class with an interface similar to the binary formatter that internally delegates the call to the IL serialization surrogate.

Sounds pretty easy, but unfortunately, the hardest part is to code the serialization surrogate using IL. Well, don’t lose heart yet, for I will show you how to write one and also give a reference implementation free. What do you say? It is a good deal, right?

OK then, let's get started. First, let me give a quick tutorial on IL.

Quick IL Tutorial

Write a simple hello world app in C#:

class HelloIL
 public static void Main()
  System.Console.Writeline("Hello IL");

Compile it and then open the application in IL DASM (in Visual Studio, go to Tools/ILDasm). Double click the Main node to see the IL:


This is what the IL looks like:

.method public hidebysig static vod Main() cil managed
// Code size 13 (0xd)
.maxstack 8
IL_0000: nop
IL_0001: ldstr "Hello IL"
IL_0006: call void [mscorlib]System.Console::WriteLine(string)
IL_000b: nop
IL_000c: ret
} // end of method HelloIL::Main

Here is where the fun begins. Using the following classes in the System.Reflection.Emit namespace, you can generate IL at runtime in any .NET app.

Now coming back to creating our serialization surrogates using this namespace. Here are the steps.

Steps to Write an IL Binary Serializer

  1. Define an interface that our dynamic serialization surrogate will implement:
    public interface IHiPerfSerializationSurrogate
     void Serialize(BinaryWriter writer, object graph);
     object DeSerialize(BinaryReader reader);
  2. Using the AssemblyBuilder class, create a dynamic assembly within the current app domain:
    AssemblyBuilder myAsmBuilder = Thread.GetDomain().DefineDynamicAssembly(
             new AssemblyName("SomeName"),
  3. Within this assembly, now define a module:
    ModuleBuilder surrogateModule = 
  4. Within the module, now define your custom serialization surrogate:
    TypeBuilder surrogateTypeBuilder = surrogateModule.DefineType( 
                                "MyClass_EventSurrogate", TypeAttributes.Public);
  5. Make this type an implementation of IHiPerfSerializationSurrogate:
  6. Now define the Serialize method within the surrogate:
    Type[] dpParams = new Type[] { typeof(BinaryWriter), typeof(object) };
    MethodBuilder serializeMethod = surrogateTypeBuilder.DefineMethod(
               MethodAttributes.Public | MethodAttributes.Virtual,
  7. And then emit a getter method for each public property:
    ILGenerator serializeIL = serializeMethod.GetILGenerator();
    MethodInfo mi = EventType.GetMethod("get_" + pi.Name);
    MethodInfo brWrite = GetBinaryWriterMethod(pi.PropertyType);
    serializeIL.Emit(OpCodes.Ldarg_1);//PU binary writer
    serializeIL.Emit(OpCodes.Ldloc, tpmEvent);//PU load the event object
    serializeIL.EmitCall(OpCodes.Callvirt, mi, null);//PU get val of property
    serializeIL.EmitCall(OpCodes.Callvirt, brWrite, null);//PU
  8. Define the DeSerialize method within the surrogate:
    MethodBuilder deserializeMthd = surrogateTypeBuilder.DefineMethod(
                        MethodAttributes.Public | MethodAttributes.Virtual | 
                        MethodAttributes.HideBySig | MethodAttributes.Final | 
  9. And now emit a setter method for each property:
    ILGenerator deserializeIL = deserializeMthd.GetILGenerator();
    MethodInfo setProp = EventType.GetMethod("set_" + pi.Name);
    deserializeIL.Emit(OpCodes.Ldloc, tpmRetEvent);//load new obj on ES
    deserializeIL.Emit(OpCodes.Ldarg_1);//PU binary reader ,load BR on ES
    deserializeIL.EmitCall(OpCodes.Callvirt, brRead, null);//PU
    deserializeIL.EmitCall(OpCodes.Callvirt, setProp, null);//PU
  10. Emit the serializing surrogate:
    Type HiPerfSurrogate = surrogateTypeBuilder.CreateType();
  11. Now that we have a high performance serialization surrogate, it is time to use it. Here is how:
    IHiPerfSerializationSurrogate surrogate =Activator.CreateInstance(HiPerfSurrogate);
    BinaryWriter binaryWriter = new BinaryWriter(serializationStream);
    surrogate.Serialize(_binaryWriter, obj);


using System;
using System.Text;
using System.IO;
using System.Runtime.Serialization.Formatters.Binary;
namespace ConsoleApplication1
    public class TestClass
        public String Name;
    class Program
        static void Main(string[] args)
            int len = int.Parse(args[0]);
            TestClass oT = new TestClass();
            oT.Name = "Hello Bin Serializer";
            System.Diagnostics.Stopwatch w = System.Diagnostics.Stopwatch.StartNew();
            for (int i = 0; i < len; i++)
                MemoryStream ms = new MemoryStream();
                BinaryFormatter bf = new BinaryFormatter();
                bf.Serialize(ms, oT);
            Console.WriteLine("Time elapsed .net binary serializer= " 
				+ w.ElapsedMilliseconds);

            //now let us see how our high performance serializer performs
            w = System.Diagnostics.Stopwatch.StartNew();
            for (int i = 0; i < len; i++)
                MemoryStream ms = new MemoryStream();
                ILSerialization.Formatters.HiPerfBinaryFormatter hpSer = 
                    new ILSerialization.Formatters.HiPerfBinaryFormatter();
                hpSer.Serialize(ms, oT);
            Console.WriteLine("Time elapsed IL hi perf serializer= " 
				+ w.ElapsedMilliseconds);

Serializing the TestClass defined in the problem section gives the following results:

  • Byte stream size: 1/3rd the size of the .NET binary serializer (51 bytes)
  • Performance: 5 times faster (for 1000000 runs, the .NET serializer took 6602 ms, our high performance serializer took 1261 ms)

Reference Implementation

"HiPerf_IL_CustomSerializer" is a reference implementation of the high performance binary serializer that is 5 times faster than the .NET binary serializer with 1/3rd the size of the serialized stream.


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

asheesh goja
United States United States
I work as an software architect in NJ USA.

Here is my technical blog.

You may also be interested in...


Comments and Discussions

SuggestionAlso check other formats (JSON/protobuf/msgpack) Pin
Maxim Novak31-Mar-14 23:38
memberMaxim Novak31-Mar-14 23:38 
Questioneven slower then .Net BinaryFormatter when serializing List<Object> in the object Pin
Member 818589026-Jan-14 15:07
memberMember 818589026-Jan-14 15:07 
QuestionNot so well for complex types Pin
Trancos5-Jun-12 18:52
memberTrancos5-Jun-12 18:52 
GeneralThank you very much ! Pin
Mazen el Senih21-Mar-12 5:13
professionalMazen el Senih21-Mar-12 5:13 
Bugwhy?? Pin
jieky peng17-Oct-11 19:59
memberjieky peng17-Oct-11 19:59 
GeneralMy vote of 5 Pin
Armando Airo'3-Feb-11 0:35
memberArmando Airo'3-Feb-11 0:35 
GeneralRe: My vote of 5 Pin
asheesh goja10-Feb-11 11:31
memberasheesh goja10-Feb-11 11:31 
Questionwhy Pin
jaan33us1-Feb-11 9:48
memberjaan33us1-Feb-11 9:48 
AnswerRe: why Pin
Mike Stoffregen23-Feb-11 14:16
memberMike Stoffregen23-Feb-11 14:16 
GeneralInteresting!!! Pin
shakil030400331-Jan-11 21:56
membershakil030400331-Jan-11 21:56 
QuestionHow do you deal with referential integrity ? Pin
leppie31-Jan-11 20:31
memberleppie31-Jan-11 20:31 
AnswerRe: How do you deal with referential integrity ? Pin
asheesh goja8-Feb-11 10:21
memberasheesh goja8-Feb-11 10:21 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Cookies | Terms of Use | Mobile
Web02 | 2.8.190306.1 | Last Updated 1 Feb 2011
Article Copyright 2011 by asheesh goja
Everything else Copyright © CodeProject, 1999-2019
Layout: fixed | fluid