Deserializing Microsoft Ajax ClientScript JSON in Managed Code

Overview

In a previous post, I proposed a means of deserializing JSON returned from calls to ClientScript endpoints such as XML WebServices decorated with [ScriptService] or [ScriptMethod] attributes, Ajax-enabled WCF Services and WCF services created with WebScriptServiceHostFactory.

The use case that prompted this requirement is testing endpoints called by client-side JavaScript. Calling WebScript endpoints in managed code using HttpWebRequest is easy, the trick is to specify content-type 'application/json'. And this is where the problems start.

Problem Domain

When a WebScript endpoint responds to a request with content-type 'application/json', it understandably assumes that it is being called from JavaScript and, as of 3.5sp1, wraps the actual return value in an object named 'd' and decorates each object with a '__type' property.

Listing 1: .NET 3.5sp1 JSON response

{
    "d": {
        "__type": "Result:#HttpLibArticleSite",
        "Message": "Value pull OK.",
        "Session": "rmyykw45zbkxxxzdun0juyfr",
        "Value": "foo"
    }
}

A Solution

Deserializing this JSON in managed code presents a few challenges.

The first is the 'd' wrapper. You could create wrapper classes for all types you expect to deserialize or, as shown in the previous post, create a generic wrapper similar to that listed below.

Listing 2: Ajax Wrapper

public class AjaxWrapper<T>
{
    public T d;   
}

The next challenge is that without a custom JavaScriptTypeResolver to resolve the '__type' value into a managed Type, neither the JavaScriptSerializer nor DataContractJsonSerializer will consume the JSON. Attempting to do so results in an ArgumentNull exception when it cannot resolve the '__type' value as no JavaScriptTypeResolver was supplied. This is in disregard of the type we supply as the generic argument of .Serialize<T>. Only if the JSON is not decorated with a JavaScriptSerializer.ServerTypeFieldName (__type) is the type argument used to instantiate the instance into which the JSON is stuffed.

We could create a custom JavaScriptTypeResolver but that would entail resolving types and maintaining lists of registered types, just as the runtime does. This is entirely beyond the intended scope of our requirements. We simply wish to enjoy the benefits of the default JavaScriptSerializer behavior when dealing with simple JSON.

So, one option is to use a JSON library such as JSON.net which does not recognize the '__type' property and happily deserializes wrapped JSON into the AjaxWrapper class.

Listing 3: Using JSON.Net and AjaxWrapper

HttpLibArticleSite.Result result = Newtonsoft.Json.JsonConvert.DeserializeObject
	<AjaxWrapper<HttpLibArticleSite.Result>>(wrappedJson).d;

If you are already using JSON.NET in your project or do not mind the dependency, then you are good to go.

A Better Solution

Recently, in developing a new library for testing JSON endpoints, the dependency on JSON.NET became undesirable and I embarked on another attempt at deserializing wrapped JSON using only my code and intrinsic framework types.

As mentioned before, parsing wrapped JSON with the .NET serializers, with or without the wrapper, will fail without a custom JavaScriptTypeResolver. Since we have determined that a custom JavaScriptTypeResolver is not indicated by the requirements, another approach is required.

The obstacles have already been identified:

• The 'd' wrapper
• The '__type' property

The obvious solution is to eliminate the offending text from the JSON. Generally, I find munging text to be a perilous venture, but in this case the text complies with the JSON spec so using a Regex will confidently satisfy the requirement.

After extracting the inner JSON, we can replace the '__type' with an empty string and proceed to deserialize the JSON into any similarly shaped CLR type.

Listed below is the final solution. ClientScriptJsonUtilities is static class that provides an extension method on JavaScriptSerializer, which, by the way, has been un-obsoleted in 3.5sp1.

NOTE

Some endpoint configurations omit the '__type' property while still wrapping in a 'd' while other configurations will respond with a 'bare' result not wrapped in a 'd' but still containing '__type' fields and yet other configurations will respond with POJO JSON.

This class will properly consume each of these types of returns so CleanAndDeserialize<T> can be treated as a backwards compatible replacement for Deserialize<T> throughout your code.

Anonymous Types

Now consider a scenario in which you need to deserialize some JSON for which you have no CLR Type. You could simply define a new class that is shaped like the JSON into which you could deserialize.

In a situation in which you will be using this type often or need to pass the response around, this may be the best approach. Other times, when the class you would be defining could be considered a temp, a more flexible approach using anonymous types may be appropriate.

With a little help from Jacob Carpenter's Blog, I have added an overload to CleanAndDeserialize that will accept an anonymous prototype. See Listing 6.

Listing 4: ClientScriptJsonUtilities.cs

// /*!
//  * Project: Salient.Web.HttpLib
//  * http://salient.codeplex.com
//  */

#region

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text.RegularExpressions;
using System.Web.Script.Serialization;

#endregion

namespace Salient.Web.HttpLib
{
    public static class ClientScriptJsonUtilities
    {
        private static readonly Regex RxMsAjaxJsonInner = 
	new Regex("^{\\s*\"d\"\\s*:(.*)}$", RegexOptions.Compiled);

        private static readonly Regex RxMsAjaxJsonInnerType = 
	new Regex("\\s*\"__type\"\\s*:\\s*\"[^\"]*\"\\s*,\\s*", RegexOptions.Compiled);

        /// <summary>
        /// Pre-processes <paramref name="json"/>, if necessary, 
        /// to extract the inner object from a "d:" 
        /// wrapped MsAjax response and removing "__type" 
        /// properties to allow deserialization with JavaScriptSerializer
        /// into an instance of <typeparamref name="T"/>.
        /// 
        /// Note: this method is not limited to MsAjax responses, 
        /// it will capably deserialize any valid JSON.
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="serializer"></param>
        /// <param name="json"></param>
        /// <returns></returns>
        public static T CleanAndDeserialize<T>
		(this JavaScriptSerializer serializer, string json)
        {
            string innerJson = CleanWebScriptJson(json);
            return serializer.Deserialize<T>(innerJson);
        }

        /// <summary>
        /// Pre-processes <paramref name="json"/>, if necessary, 
        /// to extract the inner object from a "d:" 
        /// wrapped MsAjax response and removing "__type" properties 
        /// to allow deserialization with JavaScriptSerializer
        /// into an instance of anonymous type <typeparamref name="T"/>.
        /// 
        /// Note: this method is not limited to MsAjax responses, 
        /// it will capably deserialize any valid JSON.
        /// </summary>
        /// <typeparam name="T">The anonymous type defined by 
        /// <paramref name="anonymousPrototype"/> </typeparam>
        /// <param name="serializer"></param>
        /// <param name="json"></param>
        /// <param name="anonymousPrototype">
        /// An instance of the anonymous type into which 
        /// you would like to stuff this JSON. It simply needs to be
        /// shaped like the JSON object. 
        /// <example>
        /// string json = "{ \"name\": \"Joe\" }";
        /// var jsob = new JavaScriptSerializer().CleanAndDeserialize
        /// (json, new { name=default(string) });
        /// Debug.Assert(jsob.name=="Joe");
        /// </example>
        /// </param>
        /// <returns></returns>
        public static T CleanAndDeserialize<T>
	(this JavaScriptSerializer serializer, string json, T anonymousPrototype)
        {
            json = CleanWebScriptJson(json);
            Dictionary<string, object> dict = (Dictionary<string, 
			object>)serializer.DeserializeObject(json);
            return dict.ToAnonymousType(anonymousPrototype);
        }

        /// <summary>
        /// Extracts the inner JSON of an MS Ajax 'd' result and 
        /// removes embedded '__type' properties.
        /// </summary>
        /// <param name="json"></param>
        /// <returns>The inner JSON</returns>
        private static string CleanWebScriptJson(string json)
        {
            if (string.IsNullOrEmpty(json))
            {
                throw new ArgumentNullException("json");
            }

            Match match = RxMsAjaxJsonInner.Match(json);
            string innerJson = match.Success ? match.Groups[1].Value : json;
            return RxMsAjaxJsonInnerType.Replace(innerJson, string.Empty);
        }

        #region Dictionary to Anonymous Type

        /* An entry on Jacob Carpenter saved me from having to work this out for myself.
         * Thanks Jacob.
         * http://jacobcarpenter.wordpress.com/2008/03/13/dictionary-to-anonymous-type/
         */

        /// <summary>
        /// 
        /// </summary>
        /// <typeparam name="TKey"></typeparam>
        /// <typeparam name="TValue"></typeparam>
        /// <param name="dict"></param>
        /// <param name="key"></param>
        /// <returns></returns>
        private static TValue GetValueOrDefault<TKey, TValue>
		(this IDictionary<TKey, TValue> dict, TKey key)
        {
            TValue result;
            dict.TryGetValue(key, out result);
            return result;
        }

        private static T ToAnonymousType<T, TValue>
		(this IDictionary<string, TValue> dict, T anonymousPrototype)
        {

            // get the sole constructor
            var ctor = anonymousPrototype.GetType().GetConstructors().Single();

            // conveniently named constructor parameters make this all possible...
            // TODO: sky: i think the conditional assignment could be improved
            // ReSharper disable CompareNonConstrainedGenericWithNull
            // In our typical use of this method, we are deserializing valid json, 
            // which should not contain
            // nulls for value types. So this is not a problem.
            var args = from p in ctor.GetParameters()
                       let val = dict.GetValueOrDefault(p.Name)
                       select val != null && 
			p.ParameterType.IsAssignableFrom(val.GetType()) ?
                       (object)val : null;
            // ReSharper restore CompareNonConstrainedGenericWithNull
            return (T)ctor.Invoke(args.ToArray());
        }
        #endregion
    }
}

Listing 5: CleanAndDeserialize<T>() Usage

Result result = new JavaScriptSerializer().CleanAndDeserialize<Result>(responseText);

Listing 6: CleanAndDeserialize() Usage with Anonymous Types

[Test]
public void CleanAndDeserializeToAnonymousType()
{
    // To demonstrate deserializing to an anonymous type consider this:

    // This is an example of a TestClass response
    // {"d":{"__type":"TestClass:#Salient.Web.HttpLib.TestSite",
    // "Date":"\/Date(1271275580882)\/","Header":"","IntVal":99,"Name":"sky"}}

    const string responseText = 
        "{\"d\":{\"__type\":\"TestClass:#Salient.Web.HttpLib.TestSite\",\"Date\":\"\\
	/Date(1271275580882)\\/\",\"Header\":\"\",\"IntVal\":99,\"Name\":\"sky\"}}";

    // imagine for a moment that we do not have a reference to 
    // Salient.Web.HttpLib.TestSite from which
    // to deserialize this JSON into an instance of TestClass.

    // this is what an anonymous prototype of TestClass looks like

    var testClassPrototype = new
        {
            Name = default(string),
            Header = default(string),
            Date = default(DateTime),
            IntVal = default(int)
        };

    // just pass this prototype to deserialize to get a strongly typed instance 
    var jsob = new JavaScriptSerializer().CleanAndDeserialize
		(responseText, testClassPrototype);

    Assert.AreEqual("sky", jsob.Name);
    Assert.AreEqual(99, jsob.IntVal);

    // now also imagine that we are interested in only part of a JSON response,
    var prototypeOfInterestingData = new
        {
            Name = default(string)

        };

    var partialJsob = new JavaScriptSerializer().CleanAndDeserialize
			(responseText, prototypeOfInterestingData);

    Assert.AreEqual("sky", partialJsob.Name);

    // one thing to keep in mind is that anonymous types are read-only.
}

History

• 04-15-2010 - Added anonymous type support
• 04-18-2010 - Removed licensing restriction

You can find the latest source and tests @ http://salient.codeplex.com.