Click here to Skip to main content
13,348,874 members (79,738 online)
Click here to Skip to main content
Add your own
alternative version


174 bookmarked
Posted 22 Feb 2006

A Fast/Compact Serialization Framework

, 13 Oct 2010
Rate this:
Please Sign up or sign in to vote.
A framework for object serializiation/deserialization that is many times faster and yields a compact output.


NxSerialization is an easy to use object serialization framework that replaces the functionality of the default serialization providers for .NET and Mono. The binary formatter for NxSerialization can be up to 50 times faster than the default binary formatter for .NET and Mono. This is evident from the screenshot of the benchmark application shown above. There are three main benefits this framework provides to applications that serialize objects. The main benefits being that of increased space and time performance, and enhanced security comes as a byproduct.

Quick Facts

The figure below contains benchmark results using the sample application shipped with NxSerialization for CLI. The important values are given in bold. The time measured was for 100 iterations of 100 runs each. In each run, an object of the specified type was serialized and then deserialized. These results may vary depending upon the system configuration; however, the important thing to consider is the relative difference or the performance factors between the native and the NxSerializer.

Warning: These stats are from the previous release, and do not reflect comparison with latest native formatters.

Size based comparison of .NET and NxSerialization formatters

Size based comparison of .NET and NxSerialization formatters

Time based comparison of .NET and NxSerialization formatters

Time based comparison of .NET and NxSerialization formatters

What is New in 3.0?

There is nothing substantially new in this release, except the inclusion of the Remoting sub-system and a few unfinished features. After an extended long period of inactivity and quite some queries to release the Remoting specific portions, I have finally decided to release all that I had in my dev folders, and it's probably going to be the last release ever.

An interesting observation is that the latest versions of CLR have much improved native formatters, and what used to be on the average >5 times speed gain in the past is now much reduced. The stats above are therefore not representative of comparison with latest .NET versions. It also follows that the toolkit has probably seen its time :)

Unfinished Features

EAR - (Emit Avoid Reflection)

Some of the surrogates have an EAR property that when set uses dynamic IL to facilitate creation of objects and avoids the abhorred Activator.CreateInstance that is not known to be a super-fast way. The support is in early stages, and not rigorously tested, and therefore issues may popup. Moreover, there is no way to configure EAR externally, and source modifications are needed should you want to try it.


The ability to use NxSerialization in Remoting sinks should theoretically speedup Remoting code - though the network latency may overshadow it - but surprisingly, the results have always been quite the opposite (which is why I never released it). There are also issues with HTTP channels (some functionality is missing), as well as Channel security that does not work at all.

Surrogates for System.Data.*

Still unimplemented - even though a straightforward task.

I would love to know if anyone still finds it useful and could spot the shortcomings in Remoting slowdown and suggest a fix. As always, your feedback is highly welcome!

Using the Framework

Application objects can be integrated with the framework in two ways. By writing a surrogate for the object type and registering the surrogate with the framework, or by implementing INxSerializable. The framework provides a built-in surrogate for types that implement INxSerializable. For unknown types, native .NET serialization is used.

The following sample of code demonstrates a type that implements INxSerializable. Note the line at the bottom that registers the type with the framework.

// Sample class that implements INxSerializable
class SampleCompactableClass : INxSerializable
   private String title = "SampleCompactableClass";

   void INxSerializable.Serialize(INxBinaryWriter w)

   void INxSerializable.Deserialize(INxBinaryReader r)
      title = r.ReadString();

// Register the class with the framework.

The following sample of code demonstrates a sample surrogate for another type that does not implement INxSerializable. Using surrogates is the only way the framework is able to compactly serialize .NET native types.

// Sample surrogate for SampleSurrogatedClass
class SampleSurrogate : NxSerializationSurrogate
   public SampleSurrogate() : base(typeof(SampleSurrogatedClass)) {}

   public override object Read(INxBinaryReader r)
      SampleSurrogatedClass obj = new SampleSurrogatedClass();
      obj.title = r.ReadString();
      return obj;

   public override void Write(INxBinaryWriter w, object graph)
      SampleSurrogatedClass obj = (SampleSurrogatedClass) graph;

// Sample class that does not implement INxSerializable
class SampleSurrogatedClass
   internal string title = "SampleSurrogatedClass";

// Register the surrogate with the framework.
NxTypeSurrogateSelectorNative.Default.Register(new SampleSurrogate());

Everything else is pretty much self-explanatory. For more information, look at the sample benchmark application provided with the source code.


Please note that for objects where the actual data size to type-info size ratio is very large, not much memory reduction will occur. Try a byte array of size 100K. It is also possible to come up with a case where the native serializer is actually more efficient in terms of CPU.

Among other possibilities with the framework are:

  • Enhanced security as custom serialization protects your object's data from prying eyes. Excluding the possibilities of complete reverse engineering, objects cannot be deserialized from persistent streams.
  • .NET CLR 1.x objects can be deserialized into 2.0 objects. Objects of type A can be deserialized to objects of type B etc.


OpenNxSerialization 2.0 (August 08, 2008)

Changes in this version include:

  • Arrays and collections serialization is now significantly faster.
  • New surrogates for a lot of built-in types.
  • Support for serialization of containers in the System.Collections.Generic namespace.
  • Support for serialization of BitVector32, BitArray and KeyValuePair objects.
  • Support for serialization of Type objects.
  • Surrogate redirection support now provided.
  • Dynamic (on the fly) surrogates now supported.
  • Major refactoring of the API.
  • Quite a few enhancements and utilities everywhere.

OpenNxSerialization 1.5 (March 12, 2008)

Changes in this version include:

  • NxFormatter now implements IRemotingFormatter.
  • New surrogates for a lot of built-in types.
  • Support for serialization of ISerializable objects.
  • Support for serialization of MarshalByRef objects.
  • Support for generic versions of SerializeAs and DeserializeAs functions.
  • Streaming context can now contain application specific items.
  • Quite a few enhancements and utilities everywhere.

OpenNxSerialization 1.0 (CompactSerialization 2.5) (July 21, 2007)

Once again, thanks to all contributors. Changes in this version include:

  • CompactSerialization 2.5 is now OpenNxSerialization 1.0.
  • Support for multiple instances of TypeSurrogateSelector.
  • Support for SerializeAs and DeserializeAs functions (faster and more compact).
  • Reader does not close the base stream.
  • Support to configure types using a config file.
  • Quite a few enhancements and utilities everywhere.

CompactSerialization 2.0 (May 17, 2006)

This has been possible due to the wonderful feedback I've received. Thanks to all contributors. Changes in this version include:

  • Support for .NET 2.0 Nullable types.
  • Circular and shared references are now handled wisely.
  • Support for permanent/hard type handles.
  • Support for enumerations, SortedList etc.
  • Major refactoring of the internal and public APIs.
  • Improved performance at places, and decreased at places :).

CompactSerialization 1.0 (Feb 15, 2006)

  • Released the initial version of the framework.


This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)


About the Author

Pakistan Pakistan
Let a = b ....... (1)
a - b = a - b
a^2 - ab = a^2 - ab
a^2 - ab = a^2 - b^2 (from 1)
a (a - b) = (a + b) (a - b)
a = (a + b) ...... (2)

if a = 1
1 = (1 + 1) (from 1 & 2)
1 = 2 !!

You may also be interested in...


Comments and Discussions

AnswerRe: Cross Platform Serialization Pin
.Shoaib16-Sep-08 21:02
member.Shoaib16-Sep-08 21:02 
GeneralXmlSerialization Pin
jklBruin_nyc3-Jul-08 4:18
memberjklBruin_nyc3-Jul-08 4:18 
AnswerRe: XmlSerialization Pin
.Shoaib3-Jul-08 20:17
member.Shoaib3-Jul-08 20:17 
GeneralRe: XmlSerialization [modified] Pin
jboarman7-Jul-08 16:10
memberjboarman7-Jul-08 16:10 
QuestionRe: XmlSerialization Pin
.Shoaib20-Jul-08 19:53
member.Shoaib20-Jul-08 19:53 
AnswerRe: XmlSerialization Pin
jboarman21-Jul-08 7:22
memberjboarman21-Jul-08 7:22 
GeneralOpenNxSerialization within Shared Cache Pin
schuetz25-Apr-08 7:13
memberschuetz25-Apr-08 7:13 
AnswerRe: OpenNxSerialization within Shared Cache Pin
.Shoaib28-Apr-08 22:32
member.Shoaib28-Apr-08 22:32 
Obviously yes and in two ways.

1) SharedCache uses the default serialization for internal objects (control/container objects) and performance can be improved a lot. This is not just a wishful claim, but based upon past experiences with other products of this kind. If you read the article you should find out that the background sections discuss specifically applications similar to sharedcache.

2) As for user objects (those that the client application will have to work with), sharedcache is not responsible for their serialization because it accepts byte arrays only (I could be wrong though because I didn't dig deep). It would definitely be a great help to the overall performance of the system if the user objects serialization times are also reduced. The strategy could either be by having the user opt for it independently or by integrating opennxserialization with sharedcache and providing integrated services for fast serialization.
GeneralSurrogate Emitter Pin
Aaron Jackson11-Apr-08 9:01
memberAaron Jackson11-Apr-08 9:01 
AnswerRe: Surrogate Emitter Pin
.Shoaib13-Apr-08 20:48
member.Shoaib13-Apr-08 20:48 
GeneralRe: Surrogate Emitter Pin
Derek Viljoen18-Jul-08 6:08
memberDerek Viljoen18-Jul-08 6:08 
AnswerRe: Surrogate Emitter Pin
.Shoaib20-Jul-08 19:50
member.Shoaib20-Jul-08 19:50 
GeneralRe: Surrogate Emitter Pin
Aaron Jackson23-Jul-08 9:48
memberAaron Jackson23-Jul-08 9:48 
GeneralRe: Surrogate Emitter [modified] Pin
Alex_120-Apr-08 21:02
memberAlex_120-Apr-08 21:02 
GeneralFix the license please Pin
Jean-Paul Mikkers2-Apr-08 12:29
memberJean-Paul Mikkers2-Apr-08 12:29 
NewsRe: Fix the license please Pin
.Shoaib3-Apr-08 22:34
member.Shoaib3-Apr-08 22:34 
GeneralThe download link is wrong Pin
kjetilroe26-Jul-07 0:15
memberkjetilroe26-Jul-07 0:15 
GeneralIt would be nice if TypeSurrogateSelector wasn't static Pin
patperry11-Jul-07 13:31
memberpatperry11-Jul-07 13:31 
AnswerRe: It would be nice if TypeSurrogateSelector wasn't static Pin
.Shoaib11-Jul-07 20:27
member.Shoaib11-Jul-07 20:27 
GeneralCommercial License Pin
Kay Herzam10-Jul-07 2:44
memberKay Herzam10-Jul-07 2:44 
AnswerRe: Commercial License Pin
.Shoaib10-Jul-07 3:51
member.Shoaib10-Jul-07 3:51 
QuestionCan you verify compatibility with Mono 1.2.4? Pin
Felipe Decroaux28-May-07 4:59
memberFelipe Decroaux28-May-07 4:59 
AnswerRe: Can you verify compatibility with Mono 1.2.4? Pin
.Shoaib4-Jun-07 1:17
member.Shoaib4-Jun-07 1:17 
AnswerRe: Can you verify compatibility with Mono 1.2.4? Pin
patperry11-Jul-07 11:14
memberpatperry11-Jul-07 11:14 
GeneralRe: Can you verify compatibility with Mono 1.2.4? Pin
patperry11-Jul-07 11:17
memberpatperry11-Jul-07 11:17 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web02 | 2.8.180111.1 | Last Updated 13 Oct 2010
Article Copyright 2006 by .Shoaib
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid