Click here to Skip to main content
Click here to Skip to main content

fastJSON

By , 24 May 2013
 

Preface 

The code is now on CodePlex at http://fastjson.codeplex.com/ . I will do my best to keep this article and the source code on CodePlex in sync.

Introduction  

This is the smallest and fastest polymorphic JSON serializer, smallest because it's only 25kb when compiled, fastest because most of the time it is (see performance test section) and polymorphic because it can serialize and deserialize the following situation correctly at run-time with what ever object you throw at it:

class animal { public string Name { get; set;} }
class cat: animal { public int legs { get; set;} }
class dog : animal { public bool tail { get; set;} }
class zoo { public List<animal> animals { get; set;} }

var zoo1 = new zoo();

zoo1.animals = new List<animal>();
zoo1.animals.Add(new cat());
zoo1.animals.Add(new dog());

This is a very important point because it simplifies your coding immensely and is a cornerstone of object orientated programming, strangely few serializers handle this situation, even the  XmlSerializer in .NET  doesn't do this and you have to jump through hoops to get it to work. Also this is a must if you want to replace the BinaryFormatter serializer which what most transport protocols use in applications and can handle any .NET object structure (see my WCF Killer article).

The What and Why of JSON

JSON (Java Script Object Notation) is a text or human readable format invented by Douglas Crockford around 1999 primarily as a data exchange format for web applications (see www.JSON.org). The benefits of which are ( in regards to XML which was used before):

  • Structured data format like XML
  • High signal to noise ratio in other words it does away with extra characters which are not conclusive to the data ( angle brackets and slashes in XML)
  • Compact data format
  • Simple parsing rules which makes the processing of data easy and fast

So its good for the following scenarios:

  • Data exchange between same or different platforms like Java, .NET services over the wire.
  • Data storage: MongoDB (www.mongodb.org) uses JSON as an internal storage format.

Features of this implementation

  • Just 3 classes + 2 helpers : 1158 lines of code
  • JSON standard compliant with the following additions
    • "$type" is used to denote object type information [ Json.NET does this as well ].
    • "$schema" is used to denote the dataset schema information
    • "$map" is used for post processing runtime types when assigned to the object type.
    • "$types" is used for global type definition where the instances reference this dictionary of types via a number ( reduces JSON size for large number of embedded types)
  • Works on .NET 2.0+ : some implementations in the list of alternatives below require at least .NET 3.5
  • Extremely small size : 25kb when compiled
  • Blazingly fast (see the performance tests section)
  • Can dynamically create types
  • Handles Guid, Dataset, Dictionary, Hashtable and Generic lists
  • Handles Nullable types
  • Handles byte arrays as base64 strings
  • Handles polymorphic collections of objects 
  • Thread safe   
  • Handles value type arrays (e.g. int[] char[] etc.)
  • Handles value type generic lists (e.g. List<int> etc.) 
  • Handles special case List<object[]> (useful for bulk data transfer)
  • Handles Embedded Classes (e.g. Sales.Customer)
  • Handles polymorphic object type deserialized to original type (e.g object ReturnEntity = Guid, DataSet, valuetype, new object[] { object1, object2 } ) [needed for wire communications]. 
  • Ability to disable extensions when serializing for the JSON purists (e.g. no $type, $map in the output). 
  • Ability to deserialize standard JSON into a type you give to the deserializer, no polymorphism is guaranteed.
  • Special case optimized output for Dictionary<string,string>.
  • Override null value outputs. 
  • Handles XmlIgnore attributes on properties.
  • Datatable support.
  • Indented JSON output via IndentOutput property. 
  • Support for SilverLight 4.0+. 
  • RegisterCustomType() for user defined and non-standard types that are not built into fastJSON (like TimeSpan, Point, etc.).
    • This feature must be enabled via the CUSTOMTYPE compiler directive as there is about a 1% performance hit.
    • You supply the serializer and deserializer routines as delegates.
  • Added support for public Fields.
  • Added ShowReadOnlyProperties to control the output of readonly properties (default is false = won't be outputted).
  • Automatic UTC datetime conversion if the date ends in "Z" (JSON standard compliant now).
  • Added UseUTCDateTime property to control the output of UTC datetimes.
  • Dictionary<string, > are now stored optimally not in K V format. 
  • Support for Anonymous Types in the serializer (deserializer is not possible at the moment)

Limitations 

  • Currently can't deserialize value type array properties (e.g. int[] char[] etc.)
  • Currently can't handle multi dimensional arrays.
  • Silverlight 4.0+ support lacks HashTable, DataSet, DataTable as it is not part of the runtime.

What's out there

In this section I will discuss some of the JSON alternatives that I have personally used. Although I can't say it is a comprehensive list, it does however showcase the best of what is out there.

XML

If you are using XML, then don't. It's too slow and bloated, it does deserve an honorable mention as being the first thing everyone uses, but seriously don't. It's about 50 times slower than the slowest JSON in this list. The upside is that you can convert to and from JSON easily.

BinaryFormatter

Probably the most robust format for computer to computer data transfer. It has a pretty good performance although some implementation here beat it.

Pros Cons
  • Can handle anything with a Serializable attribute on it
  • Pretty compact output
  • Version unfriendly : must be deserialized into the exact class that was serialized
  • Not good for storing of data because of the versioning problem
  • Not human readable
  • Not for communication outside of the same platform (e.g. both sides must be .NET)

Json.NET

The most referenced JSON serializer for the .NET framework is Json.NET from (http://JSON.codeplex.com/) and the blog site (http://james.newtonking.com/pages/JSON-net.aspx). It was the first JSON implementation I used in my own applications.

Pros Cons
  • Robust output which can handle datasets
  • First implementation I saw which could handle polymorphic object collections
  • Large dll size ~320kb
  • Slow in comparison to the rest in the list
  • Source code is hard to follow as it is large

LitJSON

I had to look around a lot to find this gem (http://litjson.sourceforge.NET/), which is still at version 0.5 since 2007. This was what I was using before my own implementation and it replaced the previous JSON serializer which was Json.NET. Admittedly I had to change the original to fit the requirements stated above.

Pros Cons
  • Can do all that Json.NET does (after my changes).
  • Small dll size ~57kb
  • Relatively fast
  • Didn't handle datasets in the original source code ( I wrote it my self afterwards in my own application)
  • The lexer class is difficult to follow
  • Requires .NET 3.5 ( Got around this limitation by implementing a Linqbridge class which works with .NET 2.0)

ServiceStack Serializer

An amazingly fast JSON serializer from Demis Bellot found at (http://www.servicestack.NET/mythz_blog/?p=344). The serializer speed is astonishing, although it does not support what is needed from the serializer. I have included it here as a measure of performance.

Pros     Cons  
  • Amazingly fast serializer
  • Pretty small dll size ~91kb  
  • Can't handle polymorphic object collections
  • Requires at least .NET 3.5
  • Fails on Nullable types
  • Fails on Datasets
  • Fails on other "exotic" types like dictionaries, hash tables etc.

Microsoft Json Serializer  (v1.7 update)

By popular demand and my previous ignorance about the Microsoft JSON implementation and thanks to everyone who pointed this out to me, I have added this here.

Pros      Cons
  • Included in the framework
  • Can serialize basic polymorphic objects
  • Can't deserialize polymorphic objects
  • Fails on Datasets
  • Fails on other "exotic" types like dictionaries, hash tables etc.
  • 4x slower that fastJSON in serialization

Using the code

To use the code do the following:

// to serialize an object to string
string jsonText = fastJSON.JSON.Instance.ToJSON(c);

// to deserialize a string to an object
var newobj = fastJSON.JSON.Instance.ToObject(jsonText);

The main class is JSON which is implemented as a singleton so it can cache type and property information for speed. 

Additions in v1.7.5

// you can set the defaults for the Instance which will be used for all calls
JSON.Instance.UseOptimizedDatasetSchema = true; // you can control the serializer dataset schema
JSON.Instance.UseFastGuid = true;               // enable disable fast GUID serialization
JSON.Instance.UseSerializerExtension = true;    // enable disable the $type and $map inn the output

// you can do the same as the above on a per call basis
public string ToJSON(object obj, bool enableSerializerExtensions)
public string ToJSON(object obj, bool enableSerializerExtensions, bool enableFastGuid)
public string ToJSON(object obj, bool enableSerializerExtensions, bool enableFastGuid, bool enableOptimizedDatasetSchema)

// Parse will give you a Dictionary<string,object> with ArrayList representation of the JSON input
public object Parse(string json)

// if you have disabled extensions or are getting JSON from other sources then you must specify
// the deserialization type in one of the following ways
public T ToObject<T>(string json)
public object ToObject(string json, Type type)

Additions v1.7.6

JSON.Instance.SerializeNullValues = true;    // enable disable null values to output

public string ToJSON(object obj, bool enableSerializerExtensions, bool enableFastGuid, bool enableOptimizedDatasetSchema, bool serializeNulls)
 

Additions v1.8

For all those who requested why there is no support for type "X", I have implemented a open closed principal extension to fastJSON which allows you to implement your own routines for types not supported without going through the code.

To allow this extension you must compile with CUSTOMTYPE compiler directive as there is a performance hit associated with it.

public void main()
{
     fastJSON.JSON.Instance.RegisterCustomType(typeof(TimeSpan), tsser, tsdes);
     // do some work as normal
}

private static string tsser(object data)
{
     return ((TimeSpan)data).Ticks.ToString();
}

private static object tsdes(string data)
{
     return new TimeSpan(long.Parse(data))
}

Performance Tests 

All test were run on the following computer:

  • AMD K625 1.5Ghz Processor
  • 4Gb Ram DDR2
  • Windows 7 Home Premium 64bit
  • Windows Rating of 3.9

The tests were conducted under three different .NET compilation versions

  • .NET 3.5
  • .NET 4 with processor type set to auto
  • .NET 4 with processor type set to x86

The Excel screen shots below are the results of these test with the following descriptions:

  • The numbers are elapsed time in milliseconds.
  • The more red the background the slower the times
  • The more green the background the faster the times.
  • 5 tests were conducted for each serializer.
  • The "AVG" column is the average for the last 4 tests excluding the first test which is basically the serializer setting up its internal caching structures, and the times are off.
  • The "min" row is the minimum numbers in the respective columns below.
  • The Json.NET serializer was tested with two version of 3.5r6 and 4.0r1 which is the current one.
  • "bin" is the BinaryFormatter tests which for reference.
  • The test structure is the code below which is a 5 time loop with an inner processing of 1000 objects.
  • Some data types were removed from the test data structure so all serializers could work.

The test code template

The following is the basic test code template, as you can see it is a loop of 5 tests of what we want to test each done count time (1000 times). The elapsed time is written out to the console with tab formatting so you can pipe it to a file for easier viewing in an Excel spreadsheet.

int count = 1000;
private static void fastjson_serialize()
{
	Console.WriteLine();
	Console.Write("fastjson serialize");
	for (int tests = 0; tests < 5; tests++)
	{
		DateTime st = DateTime.Now;
		colclass c;
		string jsonText = null;
		c = CreateObject();
		for (int i = 0; i < count; i++)
		{
			jsonText = fastJSON.JSON.Instance.ToJSON(c);
		}
		Console.Write("\t" + DateTime.Now.Subtract(st).TotalMilliseconds + "\t");
	}
}

The test data structure

The test data are the following classes which show the polymorphic nature we want to test. The "colclass" is a collection of these data structures. In the attached source files more exotic data structures like Hashtables, Dictionaries, Datasets etc. are included.

[Serializable()]
public class baseclass
{
    public string Name { get; set; }
    public string Code { get; set; }
}

[Serializable()]
public class class1 : baseclass
{
    public Guid guid { get; set; }
}

[Serializable()]
public class class2 : baseclass
{
    public string description { get; set; }
}

[Serializable()]
public class colclass
{
    public colclass()
    {
        items = new
List<baseclass>();
        date = DateTime.Now;
        multilineString = @"
        AJKLjaskljLA
   ahjksjkAHJKS
   AJKHSKJhaksjhAHSJKa
   AJKSHajkhsjkHKSJKash
   ASJKhasjkKASJKahsjk
        ";
        gggg = Guid.NewGuid();
        //hash = new Hashtable();
        isNew = true;
        done= true;
    }
    public bool done { get; set; }
    public DateTime date {get; set;}
    //public DataSet ds { get; set; }
    public string multilineString { get; set; }
    public List<baseclass> items { get; set; }
    public Guid gggg {get; set;}
    public decimal? dec {get; set;}
    public bool isNew { get; set; }
    //public Hashtable hash { get; set; }

}

.NET 3.5 Serialize

  • fastJSON is second place in this test by a margin of nearly 35% slower than Stacks.
  • fastJSON is nearly 2.9x faster than binary formatter.
  • Json.NET is nearly 1.9x slower in the new version 4.0r1 against its previous version of 3.5r6
  • Json.NET v3.5r6 is nearly 20% faster than binary formatter.

.NET 3.5 Deserialize

  • fastJSON is first place in this test to Stacks by a margin of 10%.
  • fastJSON is nearly 4x faster than nearest other JSON.
  • Json.NET is nearly 1.5x faster in version 4.0r1 than its previous version of 3.5r6

.NET 4 Auto Serialize

  • fastJSON is first place in this test by a margin of nearly 20% against Stacks.
  • fastJSON is nearly 4.9x faster than binary formatter.
  • Json.NET v3.5r6 is on par with binary formatter.

.NET 4 Auto Deserialize

  • fastJSON is first place by a margin of 11%.
  • fastJSON is 1.7x faster than binary formatter.
  • Json.NET v4 1.5x faster than its previous version.

.NET 4 x86 Serialize

  • fastJSON is first place in this test by a margin of nearly 21% against Stacks.
  • fastJSON is 4x faster than binary formatter.
  • Json.NET v3.5r6 1.7x faster than the previuos version.

.NET 4 x86 Deserialize

  • fastJSON is first place by a margin of 5% against Stacks.
  • fastJSON is 1.7x faster than binary formatter which is third.

Exotic data type tests

In this section we will see the performance results for exotic data types like datasets, hash tables, dictionaries, etc.. The comparison is between fastJSON and the BinaryFormatter as most of the other serializers can't handle these data types. These include the following:

  • Datasets
  • Nullable types
  • Hashtables
  • Dictionaries

fastJSON/exotic.png

  • fastJSON is 5x faster than BinaryFormatter in serialization
  • fastJSON is 20% faster than BinaryFormatter in deserialization
  • Datasets are performance killers by a factor of 10  

Performance Conclusions

  • fastJSON is faster in all test except the when running the serializer under .NET 3.5 for which Stacks is faster by only 35% (note must be made that Stacks is not polymorphic and can't handle all types so it is not outputting data correctly within the tests).
  • .NET 4 is faster than .NET 3.5 by around 15% in these test except for the fastJSON serializer which is 90% faster..
  • You can replace BinaryFormatter with fastJSON with a huge performance boost ( this lean way lends it self to compression techniques on the text output also).
  • Start up costs for fastJSON is on average 2x faster than Stacks and consistently faster than everyone else.   

Performance Conclusions v1.4

fastJSON/v1.4.png

As you can see from the above picture v1.4 is noticably faster. The speed boost make fastJSON faster than SerializerStack in all tests even on .net v3.5.

  • fastJSON serializer is 6.7x faster than binary with a dataset. 
  • fastJSON deserializer is 2.1x faster than binary with a dataset.
  • fastJSON serializer is 6.9x faster than binary without a dataset.
  • fastJSON deserializer is 1.6x faster than binary without a dataset.

Performance Conclusions v1.5

fastJSON/v1.5.png

  • The numbers speak for themselves fastJSON serializer 6.65x faster without dataset and 6.88x faster than binary, the deserializer is 2.7x faster than binary.
  • The difference in numbers in v1.5 which is slower than v1.4 is because of extra properties in the test for Enums etc.

Performance Conclusions v1.6

fastJSON/v1.6.png

  • Guid are 2x faster now with base64 encoding you can revert back to old style with the UseFastGuid = false on the JSON.Instance
  • Datasets are ~40% smaller and ~35% faster.
  • fastJSON serializer is now ~2.3x faster than deserializer and the limit seems to be 2x.

Performance Conclusions v1.7

fastJSON/v1.7.png

  • int, long parse are 4x faster.
  • unicode string optimizations, reading and writing non english strings are faster.
  • ChangeType method optimized 
  • Dictionary optimized  using TryGetValue

Points of Interest 

I did a lot of performance tuning with a profiler and here are my results:

  • Always use a StringBuilder and never strings concats.
  • Never do the following stringbuilder.append("string1 + "string2") because it kills performance, replace it with two stringbuilder appends. This point blew my mind and was 50% faster in my tests with the profiler.
  • Never give the stringbuilder a capacity value to start with e.g. var stringbuilder = new StringBuilder(4096); . Strange but it is faster without it.
  • I tried replacing the StringBuiler with a MemoryStream but it was too slow (100% slower).  
  • The simplest and the most direct way is probably the fastest as well, case in point reading values as opposed to lexer parser implementations.
  • Always use cached reflection properties on objects.

Appendix v1.9.8

Some reformatting was done to make the use of fastJSON easier in this release which will break some code but is ultimately better in the long run. To use the serializer in this version you can do the following :

// per call customization of the serializer
string str = fastJSON.JSON.Instance.ToJSON(obj, 
                 new fastJSON.JSONParamters { EnableAnonymousTypes = true }); // using the parameters

fastJSON.JSON.Instance.Parameters.UseExtensions = false; // set globally

This removes a lot of the ToJSON overloads and gives you more readable code.

Also in this release support for anonymous types has been added, this will give you a JSON string for the type, but deserialization is not possible at the moment since anonymous types are compiler generated.

DeepCopy has been added which allows you to create an exact copy of your objects which is useful for business application rollback/cancel semantics.

Appendix v2.0.0

Finally got round to adding Unit Tests to the project (mostly because of some embarrassing bugs that showed up in the changes), hopefully the tests cover the majority of use cases, and I will add more in the future. 

Also by popular demand you can now deseialize root level basic value types, Lists and Dictionaries. So you can use the following style code : 

var o = fastJSON.JSON.Instance.ToObject<List<Retclass>>(s); // return a generic list

var o = fastJSON.JSON.Instance.ToObject<Dictionary<Retstruct, Retclass>>(s); // return a dictionary

A breaking change in this version is the Parse() method now returns number formats as long and decimal not string values, this was necessary for array returns and compliance with the json format (keep the type information in the original json, and not loose it to strings).  So the following code is now working :

List<int> ls = new List<int>();
ls.AddRange(new int[] { 1, 2, 3, 4, 5, 10 }); 
var s = fastJSON.JSON.Instance.ToJSON(ls);
var o = fastJSON.JSON.Instance.ToObject(s); // long[] {1,2,3,4,5,10}

Be aware that if you do not supply the type information the return will be longs not ints.  To get what you expect use the following style code:

var o = fastJSON.JSON.Instance.ToObject<List<int>>(s); // you get List<int>

Check the unit test project for sample code regarding the above cases.

Appendix v2.0.3 -  Silverlight Support

Microsoft in their infinate wisdom has removed some functionality which was in Silverlight4 from Silverlight5. So fastJSON will not build or work on Silverlight5.

Appendix v2.0.10 - MonoDroid Support

In this release I have added a MonoDroid project file and fastJSON now compiles and works on Android devices running the excellent work done by Miguel de Icaza and his team at Xamarin. This is what Silverlight should have been and I am really excited about this as it will open a lot of opportunities one of which is the new RaptorDB

Appendix v2.0.11 - Unicode Changes

My apologies to everyone regarding my misreading of the JSON standard regarding Unicode, my interpretation was that the output should be in ASCII format and hence all non ASCII characters should be in the \uxxxx format.  

In this version you can control the output format with the UseEscapedUnicode parameter and all the strings will be in Unicode format (no \uxxxx), the default is true for backward compatibility. 

Appendix - fastJSON vs Json.net rematch 

After being contacted by James Newton King for a retest with his new version of Json.net which is v5r2, I redid the tests and here is the results (times are in milliseconds): 

As you can see there are 5 test and the AVG column is the average of the last 4 tests so to exclude the startup of each library, the DIFF column is the difference between the two libraries and fastJSON being the base of the test.

Things to note :

  • fastJSON is about 2x faster than Json.net in both serialize and deserialize.
  • Json.net is about 1.5-2x faster that it's previous versions which is a great job of optimizatons done and congratualtions in order.

History     

  • Initial Release : 2011/02/20  
  • Update v1.1 : 26% performance boost on dataset deserialization, corrected ServiceStack name
  • Update v1.2 : System.DBNull serialized to null, CultureInfo fix for numbers, Readonly properties handled correctly
  • Update v1.3 : Removed unused code (lines now at 780), Property comma fix
  • Update v1.4 : Heavy optimizations (serializer 3% faster, deserializer  50% faster, dataset serializer 46% faster, dataset deserializer 26% faster) [ now officially faster than the serializer ServiceStack in all test even on .net 3.5]
  • Update v1.5 : Heavy optimizations (deserializer ~50% faster than v1.4), Enum fix, Max Depth property for serializer. Special thanks and credits to Simon Hewitt for optimizations in this version. 
  • Update v1.6 :
    • value type arrays handled 
    • guid 2x faster
    • datasets ~40% smaller
    • serializer ~2% to 11% faster
    • deserializer ~6% to 38% faster
  • Update v1.7 :
    • added microsoft json evaluation
    • added consoletest project to downloads for testing newer exotic types
    • bug fix dictionary deserialize
    • special case handles List<object[]> 
    • int and long parse 4x faster
    • unicode string optimize
    • changetype optimize
    • dictionary optimize
    • deserialize embeded class e.g. Sales.Customer
    • safedictionary check before add
    • handles object ReturnEntity = new object[] { object1, object2 }
    • handles object ReturnEntity = Guid, Dataset, valuetype
  • Update v1.7.5 :
    • ability to serialize without extensions
    • overloaded methods for serialize and deserialize
    • the deserializer will do its best to deserialize the input with or without extensions with no gaurantee on polymorphism 
  • Update v1.7.6 :
    • XmlIgnore handled : thanks to Patrik Oscarsson for the idea
    • special case optimized output for dictionary of string,string
    • bug fix year 1 date output as 0000 string
    • override serialize nulls to output : thanks again to Patrik
  • Update v1.7.7 :
    • Indented output
    • Datatable support
    • bug fix
  • Update v1.7.7 Silverlight4 : 4th June 2011
    • A new project added for silverlight4, currently in testing phase will add to main zip when all ok.
    • Silverlight lacks arraylist, dataset, datatable, hashtable support
    • #if statements in source files for silverlight4 support. 
  • Update v1.8 :  9th June 2011
    • Silverlight code merged into the project
    • Seperate Silverlight project 
    • RegisterCustomType extension for user defined serialization routines
    • CUSTOMTYPE compiler directive
  • Update v1.9 : 28th June 2011
    • added support for public fields
  • Update v1.9.1 : 30th June 2011
    • fixed a shameful bug when SerializeNullValues = false, special thanks to Grant Birchmeier for testing 
  • Update v1.9.2 : 10th July 2011
    • fixed to fullname instead of name when searching for types in property cache (namespace1.myclass , namespace2.myclass are now different) thanks to alex211b
  • Update v1.9.3 : 31st July 2011
    • UTC datetime handling via UseUTCDateTime = true property thanks to mrkappa
    • added support for enum as key in dictionary thanks to Grant Birchmeier
  • Update v1.9.4 : 23rd September 2011
    • ShowReadOnlyProperties added for exporting readonly properties (default = false)
    • if datetime value ends in "Z" then automatic UTC time calculated
    • if using UTC datetime the output end in a "Z" (standards compliant)
  • Update v1.9.6 : 26th November 2011
    • bug fix datatable schema serialize & deserialize
    • added a $types extension for global type definitions which reduce the size of the output json thanks to Marc Bayé for the idea
    • added UsingGlobalTypes config for controling the above (default = true)
    • bug fix datatable commas between arrays and table definitions (less lint complaining)
    • string key dictionaries are serialized optimally now (not K V format)
  • Update v1.9.7 : 10th May 2012 
    • bug fix SilverLight version to support GlobalTypes
    • removed indent logic from serializer
    • added Beautify(json) method to JSON credits to Mark http://stackoverflow.com/users/65387/mark
    • added locks on SafeDictionary
    • added FillObject(obj,json) for filling an existing object
  • Update v1.9.8 : 17th May 2012
    • added DeepCopy(obj) and DeepCopy<T>(obj)
    • refactored code to JSONParameters and removed the JSON overloads
    • added support to serialize anonymous types (deserialize is not possible at the moment) 
    • bug fix $types output with non object root
  • Update v1.9.9 : 24th July 2012
    • spelling mistake on JSONParameters
    • bug fix Parameter initialization
    • bug fix char and string ToString
    • refactored reflection code into Reflection class
    • added support for top level struct object serialize/deserialize
  • Update v2.0.0 : 4th August 2012
    • bug fix reflection code
    • added unit tests
    • deserialize root level arrays (int[] etc.)
    • deserialize root level value types (int,long,decimal,string)
    • deserialize ToObject< Dictionary<T,V> > 
    • deserialize ToObject< List<T> >
    • * breaking change in Parse , numbers are returned as decimals and longs not strings
  • Update v2.0.1 : 10th August 2012
    • bug fix preserve internal objects when FillObject called
    • changed ArrayList to List<object> and consolidated silverlight code
    • added more tests
    • speed increase when using global types ($types)
  • Update v2.0.2 : 16th August 2012
    • bug fix $types and arrays
  • Update v2.0.3 : 27th August 2012
    • readonly property checking on deserialize (thanks to Slava Pocheptsov)
    • bug fix deserialize nested types with unit test (thanks to Slava Pocheptsov)
    • fix the silverlight4 project build (silverlight5 is not supported)
  • Update v2.0.4 : 7th September 2012
    • fixed null objects -> returns "null"
    • added sealed keyword to classes
    • bug fix SerializeNullValues=false and an extra comma at the end
    • UseExtensions=false will disable global types also  (thanks to qio94, donat.hutter, softwarejaeger)
    • fixed parameters setting for Parse()
  • Update v2.0.5 : 17th September 2012 
    • fixed number parsing for invariant format
    • added a test for German locale number testing (,. problems)
  • Update v2.0.6 : 19th September 2012  
    • singleton uses [ThreadStatic] for concurrency (thanks to Philip Jander)
    • bug fix extra comma in the output when only 1 property in the object (thanks to Philip Jander)
  • Update v2.0.7 : 5th October 2012  
    • bug fix missing comma with single property and extensions enabled
  • Update v2.0.8 : 13th October 2012    
    • bug fix big number conversions  (thanks to  Alex .µZ Hg.
    • * breaking change Parse will return longs and doubles instead of longs and decimal
    • ToObject on value types will auto convert the data (e.g ToObject<decimal>() )
  • Update v2.0.9 : 24th October 2012  
    •  added support for root level DataSet and DataTable deserialize (you have to do ToObject<DataSet>(...) ) 
    • added dataset tests
  • Update v2.0.10 : 15th November 2012  
    • added MonoDroid project  
  • Update v2.0.11 : 7th December 2012 
    • bug fix single char number json
    • added UseEscapedUnicode parameter for controlling string output in \uxxxx for unicode/utf8 format
    • bug fix null and generic ToObject<>()
    • bug fix List<> of custom types
  • Update v2.0.12 : 3rd January 2013
    • bug fix nested generic types (thanks to Zambiorix)
    • bug fix comma edge cases with nulls
  • Update v2.0.13 : 9th January 2013 
    • bug fix comma edge cases with nulls
    • unified DynamicMethod calls with SilverLight4 code
    • test cases for silverlight
  • Article Update : 12th April 2013
    • rematch between fastJSON and Json.net v5r2
  • Update v2.0.14 : 19th April 2013
    •   Optimizations done by Sean Cooper
         - using Stopwatch instead of DateTime for timings
         - myPropInfo using enum instead of boolean
         - using switch instead of linked if statements
         - parsing DateTime optimized
         - StringBuilder using single char output instead of strings for \" chars etc
  • Update v2.0.15 : 24th May 2013 
    • removed CUSTOMTYPE directives from code
    • fix for writing enumerable object

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Mehdi Gholam
Architect
United Kingdom United Kingdom
Mehdi first started programming when he was 8 on BBC+128k machine in 6512 processor language, after various hardware and software changes he eventually came across .net and c# which he has been using since v1.0.
He is formally educated as a system analyst Industrial engineer, but his programming passion continues.
 
* Mehdi is the 5th person to get 6 out of 7 Platinums on CodeProject (13th Jan'12)

Sign Up to vote   Poor Excellent
Add a reason or comment to your vote: x
Votes of 3 or less require a comment

Comments and Discussions

 
You must Sign In to use this message board.
Search this forum  
    Spacing  Noise  Layout  Per page   
GeneralRe: Nested classes handled?memberGhanashyamL2-May-13 11:03 
Thanks! That worked!
QuestionWhat is Stacks?memberahmed zahmed19-Apr-13 9:17 
Good job on the article. Haven't looked at the code, but I'm intrigued. I've written some xml serializers before so I'm interested to checkout your code for JSON.
 
But I don't know what Stacks is? Can you provide a link? And yes, I've googled and found some promising links but I'm unsure if it's what you're referring to.
 
Thanks.
If your actions inspire others to dream more, learn more, do more and become more, you are a leader.-John Q. Adams
You must accept one of two basic premises: Either we are alone in the universe, or we are not alone in the universe. And either way, the implications are staggering.-Wernher von Braun
Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.-Albert Einstein

AnswerRe: What is Stacks?mvpMehdi Gholam19-Apr-13 9:24 
Thanks Ahmed!
 
If you mean ServiceStacks then the link is in the article, and it is a json/csv serializer.
Its the man, not the machine - Chuck Yeager
If at first you don't succeed... get a better publicist
If the final destination is death, then we should enjoy every second of the journey.

GeneralRe: What is Stacks?memberahmed zahmed19-Apr-13 9:28 
I was referring to the performance table where you list "Stacks". ServiceStacks is what I found when I googled "JSON stacks".
 
Where is the "$type" and other "special" keywords documented for the JSON "standard".
If your actions inspire others to dream more, learn more, do more and become more, you are a leader.-John Q. Adams
You must accept one of two basic premises: Either we are alone in the universe, or we are not alone in the universe. And either way, the implications are staggering.-Wernher von Braun
Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.-Albert Einstein

GeneralRe: What is Stacks?mvpMehdi Gholam19-Apr-13 9:38 
Ah! I abbreviated the name in the excel sheets, but it is referring to ServiceStacks.
 
$type is not part of the JSON standard, it is an extension first used by JSON.net which kind of stuck. $types and $map are my extensions, and normally you would not need to work with them as they are handled internally.
Its the man, not the machine - Chuck Yeager
If at first you don't succeed... get a better publicist
If the final destination is death, then we should enjoy every second of the journey.

GeneralRe: What is Stacks?memberahmed zahmed19-Apr-13 9:40 
ok, thanks for the info.
 
Cheers!
If your actions inspire others to dream more, learn more, do more and become more, you are a leader.-John Q. Adams
You must accept one of two basic premises: Either we are alone in the universe, or we are not alone in the universe. And either way, the implications are staggering.-Wernher von Braun
Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.-Albert Einstein

QuestionLINQ Expressions vs ILGenerator [modified]memberkornman0013-Apr-13 12:47 
Have you done any comparisons of using LINQ Expressions for generating your getters/setters code instead of using ILGenerator (which also requires the code using it to have elevated security privileges too if I'm not mistaken, where LINQ doesn't). Eg, I used LINQ Expressions to create accessors for private members, which you can see in this article[^]. I'd figure this would also allow the VM to optimize it since it's the one generating the IL from your high level expressions.
 
Also, why is 'myPropInfo' implemented as a struct (value type)? Not counting alignment padding, the structure is going has consume almost 50 bytes on a 32-bit platform (could be lessened with a Flags enum, but that still doesn't address my question about why it's a struct).
 
Every time you add or query an myPropInfo from your _propertycache, by definition the item retrieved should be a by-value copy of the stored 'myPropInfo', instead of a reference to. That just seems wasteful in terms of memory and execution performance, but if you have evidence to counter this assumption I'd like to see it (just as you've shown with all your other perf comparisons Cool | :cool: ).
 
edit: Also, wouldn't it be a tad more efficient to implement your CreateLong(string) method with an index and count parameter? In CreateDateTime you do a couple of value.Substring() calls. I would figure that you could just lose the extra string allocation from Substring and instead operating on the string's characters using its indexer instead of working on an enumerator.
 
It may also be better to have CreateInteger(out int/long num, string...) variants, at least for 32-bit platforms. Just to get rid of the extraneous 64-bit operations when you're dealing with ints (32-bits) in CreateDateTime (at least).
 
edit2: Looking more at the myPropInfo code, you have booleans for conditions which would be better suited in a 'type' Enum. Eg, isGuid vs isLong. A probably will only ever be one of those. Then in your 'ParseDictionary' method you could instead have a switch() statement that works on this hypothetical 'type' Enum's member for most of the if/else conditions, which would probably result in faster condition execution (assuming the JIT compiler generates an actual switch table in the assembly code)
 
edit3: I notice in some places you perform StringBuilder.Append calls on 1 character strings, instead of say an actual character (_output.Append("-") vs _output.Append('-')). Looking at the StringBuilder implementation for Append(string) and Append(char), it may prove to be slightly faster to use the char overload. While the string overload optimizes for strings which have 2 or less characters (and when there's enough free characters to append them), you would probably save a few cycles with the char overload due to fewer operations being done before the character is Append'ed
 
See for yourself:
public unsafe StringBuilder Append(string value)
{
    if (value != null)
    {
        char[] chunkChars = this.m_ChunkChars;
        int chunkLength = this.m_ChunkLength;
        int length = value.Length;
        int num3 = chunkLength + length;
        if (num3 < chunkChars.Length)
        {
            if (length <= 0x2)
            {
                if (length > 0x0)
                {
                    chunkChars[chunkLength] = value[0x0];
 
public StringBuilder Append(char value)
{
    if (this.m_ChunkLength < this.m_ChunkChars.Length)
    {
        this.m_ChunkChars[this.m_ChunkLength++] = value;
    }
    else
    {
        this.Append(value, 0x1);
    }


modified 13-Apr-13 20:25pm.

AnswerRe: LINQ Expressions vs ILGeneratormvpMehdi Gholam13-Apr-13 21:20 
Thanks!
 
A couple of points:
 
1) fastJSON was designed to work with .net2+ so using LINQ and lambdas was not an option at the time (these requirement might not be valid now, but just the same).
2) IL gen is bare metal, and I doubt that you can get faster.
3) myPropInfo is a struct because it was faster (don't ask me why! I leave that to people with more time than me to figure out Smile | :) )
4) fastJSON was again designed to be as simple and as fast as possible given technical limitations, and has been optimized to mere percentages (in my opinion) to the current design.
 
Please feel free to change the code and try out new things and let me know, you never know I could be wrong...
 
Cheers
Its the man, not the machine - Chuck Yeager
If at first you don't succeed... get a better publicist
If the final destination is death, then we should enjoy every second of the journey.

GeneralRe: LINQ Expressions vs ILGenerator [modified]memberkornman0014-Apr-13 6:16 
Thanks for the reply. I'll try and look into testing the changes and will report back if I find anything of interest Smile | :)

modified 15-Apr-13 13:04pm.

GeneralRe: LINQ Expressions vs ILGeneratormvpMehdi Gholam15-Apr-13 6:40 
Please add new comments instead of editing old ones, since editing does not notify me via email to take a look so I will probably miss them unless I open the article page.
Its the man, not the machine - Chuck Yeager
If at first you don't succeed... get a better publicist
If the final destination is death, then we should enjoy every second of the journey.

AnswerResults from changesmemberkornman0015-Apr-13 7:11 
Sorry about that! Since there was no response yet I just edited the post (the usual etiquette on other forums with simpler natures). Didn't consider how it handled emails (or lack thereof) Here's what I added in the edit yesterday:
 
The following are results from
1) Restructuring myPropInfo to use one Enum type and flags member instead of an assortment of bool's. On my x64 machine (with AnyCPU) myPropInfo was originally 0x98 bytes. After grouping all reference fields together and using the type/flags setup it boiled down to 0x40 bytes. At this stage, changing it to a class didn't seem to impact performance good or bad.
2) Restructuring 'ParseDictionary' to primarily use a switch() table instead of a block of if/else conditions
3) Reimplementing CreateLong as a static CreateInteger which uses an out parameter to deduce the integer type (there are overloads for int and long, with CreateDateTime using the int overload) and with a string/char[] index/count parameters instead of foreach'ing over a string.
4) ParseNumber in JsonParser uses the above implementation, avoiding a 'new string' allocation when the number doesn't have a decimal
5) Going through JsonSerializer and changing all 1-character string Append() calls to Append(char) calls.
6) Reimplementing consoletest to use Stopwatch instead of DateTime
7) .NET 4, AnyCPU (machine is natively x64) release build (and outside a debugger)
 
e-dataset
fastjson serialize      70      52      52      52      53
fastjson deserialize    107     91      88      88      87
 
+dataset
fastjson serialize      334     324     327     328     324
fastjson deserialize    680     676     675     680     678
 
e-dataset
fastjson serialize      71      52      52      52      53
fastjson deserialize    107     89      89      91      88
 
+dataset
fastjson serialize      330     324     326     324     325
fastjson deserialize    677     703     694     694     699
 
e-dataset
fastjson serialize      70      52      52      54      54
fastjson deserialize    113     89      89      89      89
 
+dataset
fastjson serialize      328     328     324     325     325
fastjson deserialize    672     669     671     676     674
 
e-dataset
fastjson serialize      70      52      52      52      52
fastjson deserialize    112     90      91      90      90
 
+dataset
fastjson serialize      329     322     322     322     322
fastjson deserialize    678     681     678     672     669
 
 
Original code (Stopwatch modifications only)
e-dataset
fastjson serialize      70      53      53      52      52
fastjson deserialize    114     92      90      92      91
 
+dataset
fastjson serialize      338     337     338     338     340
fastjson deserialize    697     695     695     695     698
 
e-dataset
fastjson serialize      70      53      55      54      55
fastjson deserialize    114     91      90      91      92
 
+dataset
fastjson serialize      341     339     340     336     338
fastjson deserialize    697     698     695     694     695
 
e-dataset
fastjson serialize      70      52      52      52      53
fastjson deserialize    115     92      91      92      92
 
+dataset
fastjson serialize      337     337     336     335     340
fastjson deserialize    675     674     678     673     672
 
Overall, my changes saw a consistent performance increase in the deserialization process. For +dataset though, there's a pretty nice improvement in serialize, while -dataset remains pretty much the same.
 
I think I have a CodePlex account so I can work on getting a clone up with these changes if you'd like, so you can investigate further.
GeneralRe: Results from changesmvpMehdi Gholam15-Apr-13 7:25 
Very nice! send me a diff when you can so I can try things out.
 
I remember using a switch without any speedups a long time ago.
Its the man, not the machine - Chuck Yeager
If at first you don't succeed... get a better publicist
If the final destination is death, then we should enjoy every second of the journey.

GeneralRe: Results from changesmemberkornman0015-Apr-13 7:54 
I put the unified diff from 2.0.13 up on pastebin http://pastebin.com/Y0uGjLSy[^]
 
I use tabs instead of spaces, so that may irk you. VS's "format document" should take care of that for you though Poke tongue | ;-P
 
Also forgot to mention: Right now in your Debug builds the CUSTOMTYPE macro has a '-' before it. I assuming this was intentional to make the compiler ignore it, but to still keep the macro definition there?
 
Release builds don't have it, so my timings don't factor CUSTOMTYPE support in
GeneralRe: Results from changesmvpMehdi Gholam15-Apr-13 8:04 
Thanks I will check it out.
 
Yep the customtype is there as a placeholder and not enabled, release build should have it also, although debug build seems to be faster than release for some strange reason (checked a long time ago so I don't know if it is the same now).
 
Try running the unit tests with your changes to make sure things still work.
Its the man, not the machine - Chuck Yeager
If at first you don't succeed... get a better publicist
If the final destination is death, then we should enjoy every second of the journey.

GeneralRe: Results from changesmemberkornman0015-Apr-13 10:34 
I just downloaded and installed NUnit to run the unit tests you have and I'm happy to report that I didn't break anything! All 34 tests ran without error
GeneralmyPropInfoType/Flags as an enum vs const intsmemberkornman0016-Apr-13 12:15 
I noticed in fastBinary that you use const bytes instead of an enum for your token types. I wanted to see if going this route made any impact with my changes. The following are results from changing the myPropInfoType/Flags to const ints (.NET 4, AnyCPU on a 64bit machine, Release) . Diff here: http://pastebin.com/30faGq0u[^] (still includes my original changes)
 
const int:
e-dataset
fastjson serialize      71      53      54      53      53
fastjson deserialize    112     90      89      89      90
 
+dataset
fastjson serialize      327     338     333     327     325
fastjson deserialize    674     669     702     677     671
 

e-dataset
fastjson serialize      70      53      53      52      53
fastjson deserialize    113     90      89      90      89
 
+dataset
fastjson serialize      326     322     322     336     322
fastjson deserialize    675     700     669     673     670
 

e-dataset
fastjson serialize      73      53      57      58      52
fastjson deserialize    112     90      90      90      89
 
+dataset
fastjson serialize      321     320     321     318     322
fastjson deserialize    681     684     696     694     681
 
enum:
e-dataset
fastjson serialize      74      69      63      56      52
fastjson deserialize    138     103     91      108     90
 
+dataset
fastjson serialize      324     332     334     327     324
fastjson deserialize    677     682     675     688     675
 

e-dataset
fastjson serialize      73      53      56      52      53
fastjson deserialize    112     90      90      90      90
 
+dataset
fastjson serialize      327     324     320     319     323
fastjson deserialize    673     668     672     667     676
 

e-dataset
fastjson serialize      70      52      53      52      52
fastjson deserialize    111     89      90      89      91
 
+dataset
fastjson serialize      336     333     346     340     334
fastjson deserialize    678     675     685     683     674
 
There was no real consistent difference between the two. Which is good because I like enums. I would have got a little depressed if there was a performance hit in using them.
GeneralRe: myPropInfoType/Flags as an enum vs const intsmvpMehdi Gholam17-Apr-13 5:57 
In fastBinaryJSON the TOKENS are used in the output so I can't change them to enum (performance hit in the conversion to output).
Its the man, not the machine - Chuck Yeager
If at first you don't succeed... get a better publicist
If the final destination is death, then we should enjoy every second of the journey.

Generalside note...mvpMehdi Gholam15-Apr-13 8:12 
... take a look at the binary version here : fastBinaryJSON[^]
Its the man, not the machine - Chuck Yeager
If at first you don't succeed... get a better publicist
If the final destination is death, then we should enjoy every second of the journey.

GeneralRe: side note...memberkornman0015-Apr-13 10:46 
Ah yes, completely forgot about your BSON version Smile | :) . Will do. Have you thought about combining them under one repo, or having BSON be a subrepo, while having something like a fastJSON.Core for the shared code (along with making some use of InternalsVisableToAttribute)? Briefly scanning the code I saw the myPropInfo struct again, so I'm going to go out on a limb and assume that most of the reflection utilities are copy&pasted to BSON. If someone wanted to use both for whatever reason (eg, a game dev who develops with text but then ships using binary), they would end up with more memory overhead as both JSON and BSON libs would be duplicating that reflection code/data (at the very least).
 
Your Helper class in BSON seems to duplicate the GetBytes/ToXxx functions from BitConverter. I realize that you provide byte swapping functionality, but have you done speed comparisons to see if re-implementing those functions was faster than calling BitConverter's? Well, yours doesn't do any contract verification (bytes != null, etc) like BitConverter so that probably shaves off some cycles.
GeneralRe: side note...mvpMehdi Gholam15-Apr-13 18:30 
Well it's not BSON but a binary JSON, and yes most of the helper code is for performance.
 
As for combining the libraries, this is certainly desirable from my stand point in maintaining them both but the problem is the users of fastJSON are probably web devs and don't need binary, and the only people who need them both are network protocol and storage users which are few and trying to combine them will probably introduce extra layers and slow things down (speculating here).
 
I use them both in RaptorDB and there I factor out the same files and only use one.
Its the man, not the machine - Chuck Yeager
If at first you don't succeed... get a better publicist
If the final destination is death, then we should enjoy every second of the journey.

GeneralSystem.Diagnostics.Stopwatch [modified]memberkornman0014-Apr-13 6:46 
While going through your consoletest project, I noticed that you use DateTime for timing. Using Stopwatch is more precise in both timing and control over where you're timing. Eg (using bin_deserialize, line 211):
 
var stopwatch = new Stopwatch();
            for (int pp = 0; pp < tcount; pp++)
            {
                BinaryFormatter bf = new BinaryFormatter();
                MemoryStream ms = new MemoryStream();
                colclass deserializedStore = null;
                stopwatch.Restart();
                bf.Serialize(ms, c);
                for (int i = 0; i < count; i++)
                {
                    stopwatch.Stop(); // we stop then resume the stopwatch here so we don't factor in Seek()'s execution
                    ms.Seek(0L, SeekOrigin.Begin);
                    stopwatch.Start();
                    deserializedStore = (colclass)bf.Deserialize(ms);
                }
                stopwatch.Stop();
                Console.Write("\t" + stopwatch.ElapsedMilliseconds);
 
Just for being efficient in the test code itself, I created the stopwatch before the loop body (it's a reference type, not value like DateTime). From there, I start/restart (calling Restart is the same as calling Start on a fresh stopwatch) it after non-interesting operations (the MemoryStream ctor, etc) then essentially pause the stopwatch while Seek executes. Granted, it still means part of the for() loop itself is being timed, but overall it's a much better representation of your, or a 'rival's, API.
 
edit: Not sure if you noticed this (though you do have the bin_* methods commented out in Main) but none of the classes in dataobjects.cs are marked with [Serializable] so the bin_* methods will throw exceptions.

modified 14-Apr-13 12:54pm.

GeneralRe: System.Diagnostics.StopwatchmvpMehdi Gholam18-Apr-13 2:35 
Please send me your full name so I can credit you in the next release.
Its the man, not the machine - Chuck Yeager
If at first you don't succeed... get a better publicist
If the final destination is death, then we should enjoy every second of the journey.

QuestionUpdate fastJSON benchmarksmemberJames Newton-King7-Apr-13 13:29 
Hi
 
Could you update your benchmarks to either benchmark against the latest version of Json.NET or remove Json.NET from them all together.
 

Thanks
James
AnswerRe: Update fastJSON benchmarksmvpMehdi Gholam7-Apr-13 20:12 
Certainly, I will run the tests as soon as possible.
 
Thanks James!
Its the man, not the machine - Chuck Yeager
If at first you don't succeed... get a better publicist
If the final destination is death, then we should enjoy every second of the journey.

QuestionSerialization of custom class containing overloaded index operator property [modified]memberbarbrady123428-Mar-13 9:50 
I have a class defined with an overloaded [] operator...for example:
 
public int this[int iX, int iY] { get { return m_aiData[iX,iY]; } set { m_aiData[iX,iY] = value;} }
 
However, the serializer seems to have a problem with this, giving me the exception "Common Language Runtime detected an invalid program."
 
I tested a variety of scenarios and this fails on *any* property of this type, unless of course I remove the "set", since it becomes read-only in that case and is ignored by fastJSON.
 
Is there any way around this? Is this something that can't be serialized? Of course I can make the property read-only and create a seperate function to "set" the values, but I'd really prefer not to, if possible. Thanks! Confused | :confused:
 
Edit: I forgot it doesn't handle n-dimension array, so I'll have to change some stuff anway Frown | :( But, I'd still like to have a definite answer. Smile | :)

modified 28-Mar-13 16:46pm.

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Permalink | Advertise | Privacy | Mobile
Web03 | 2.6.130617.1 | Last Updated 24 May 2013
Article Copyright 2011 by Mehdi Gholam
Everything else Copyright © CodeProject, 1999-2013
Terms of Use
Layout: fixed | fluid