WebService DIME Bridge

• Download source - 19.1 Kb
• Download installer - 146 Kb

Contents

• Introduction
• Features
• Concept
• Usage
• Implementation
• Conclusion

Introduction

Communication to the web service over Internet is based on the text formatted protocol, where binary fields are using the Base64 encoded text embedded in the body of the message. The Return object from the web method can be a simple type value or a custom object with more complex types such as binary arrays, etc. The logical connectivity between the web service and its consumer is encapsulated into the proxy/stub infrastructure allowing strongly type access from the business layer. The communication proxy/stub channel provides all magic over wire using the HTTP transport layer, including encoding/decoding binary images in the Base64 text pattern.

Introducing the DIME (Direct Internet Message Encapsulation) in the WSE 1.0 +, the binary images can be sent over Internet as binary attachments. Using the WSE Dime namespace classes, the DIME programming is very straightforward at both ends. Each attachment represents one record behind the first one - record 0 (reserved for a SoapMessage).

This article describes the solution allowing to transfer the web service response (return value) in the binary format using the WSE-DIME feature. This is a fully transparent loosely coupled solution injected into the message process pipeline at the properly stage.

The DIME Bridge encapsulates a tightly coupled programming of the binary attachments from the business layer, which it can be very useful for migrating the business layers to the next communication model such as Indigo. (Notice that the Indigo will not use the DIME feature.)

Features

The DIME Bridge has the following features:

• Fast (binary) response time from the web service
• Eliminating a binary encoding/decoding using the Base64 fashion
• Encapsulating a business layer from the DIME attachment handling
• Loosely coupled plug-in

Concept

The concept of the DIME Bridge is based on binary transferring a web service response (return object) via the Attachment. On the server side - before the DIME bridge, the Return value is serialized into the DIME attachment and its reference in the SoapMessage is setup to null, which will produce very lightweight XML text formatted SoapMessage. On the client side - after the DIME bridge, the situation is reversed. The DIME Attachment is de-serialized into the Return object and its reference is set-up in the SoapMessage (replacing the null value). From the client point of view, all processes via the DIME Bridge is fully transparent and loosely coupled in the SoapExtension message pipeline.

The following picture shows this scenario:

The client invokes the WebMethod via the WSClientProxy - at the stage before serialize (1), the SoapMessage is extended for unknown soap header to indicate a server side that the DIME Bridge can be used for the response (4). The SoapServerMessage before the serialize stage is holding the reference of the Return object, so it can be simply serialized and its stream be stored into the DIME Attachment. After that, we can "kill" the return object in the SoapMessage in the same manner like web method, returning value null.

The following flowchart shows the stage (4):

The SoapMessage is encapsulated into the DIME records. In our case, the result value (return object) is first an Attachment, so the DIME Bridge on the wire (5) will have the following pattern, which represents the core of the concept:

As you can see, the above server side stages are straightforward supported by MS WSE-DIME SoapExtension, which has to be plugged into the SoapExtension pipeline close to the wire (priority="1", group="0").

Situation on the client side is a little bit different, because the DIME Bridge can be injected also in the "legacy" client proxy (no WSE features).

Let's look at this scenario.

The SoapClientMessage at the BeforeDeserialize stage has a responsibility to handle a network stream in the case of the "legacy" proxy. The first DIME record represents the SoapMessage and must be passed through the SoapExtension pipeline like in a normal text/XML way. The next (second) DIME record is our stream of the Return value. Its reference is temporarily stored for the next - final stage.

The following flowchart shows the message workflow at the BeforeDeserialize stage:

The following flowchart is a final stage of the SoapClientMessage processing. Based on the existing WSE proxy, the first Attachment is retrieved and temporarily stored. The next workflow is a common part for any type of proxy - replacing the nullvalue of the Return object in the SoapMessage by the deserialized stream. That's the final step, and from now the SoapMessage can go to the business layer like in the case without the DIME Bridge.

That's all there is in the magic of the DIME Bridge.

Based on the above concept, the DIME Bridge can be extended very easily for full duplex included the output parameters or for a message body streaming.

The following picture shows an idea for streaming data without using the DOM, where SoapHeaders are encapsulated from the content (body) based on the described concept. This design pattern allows to scan the SOAP headers and forward content to the proper endpoint.

Usage

Using the DIME Bridge requires installing the WSE 2.0 in prior. After that, the downloaded DIME Bridge MSI file can be run. The setup will install the DimeBridge assembly into the GAC including its source code in the specified location. Note that the DIME Bridge must be installed on both sides: the server and its consumer (client). The next step is to plug the bridge into the SoapExtension pipeline.

Configuration

The following snippet shows a configuration part on the server side:

<webServices>
   <soapExtensionTypes>
      <add type="Microsoft.Web.Services2.WebServicesExtension, 
                 Microsoft.Web.Services2, 
                 Version=2.0.0.0, Culture=neutral, 
                 PublicKeyToken=31bf3856ad364e35" 
                 priority="1" group="0" />
      <add type="RKiss.SoapExtensionUtil.DimeBridge, DimeBridge, 
                 Version=1.0.0.0, Culture=neutral, 
                 PublicKeyToken=3445381e5e540793" 
                 priority="2" group="0" />
   </soapExtensionTypes>
</webServices>

and the following snippet on the client side:

<webServices>
  <soapExtensionTypes>
     <add type="RKiss.SoapExtensionUtil.DimeBridge, DimeBridge, 
                Version=1.0.0.0, Culture=neutral, 
                PublicKeyToken=3445381e5e540793" 
                priority="1" group="0" />
  </soapExtensionTypes>
</webServices>

Note that the priority of the DIME Bridge has to be properly set-up in the case of using multiple SoapExtensions. On the server side, the bridge's priority should be behind the WSE, but on the client side, must be close to the wire (NetworkStream).

The following DIME Bridge properties can be configured in the appSettings section:

<appSettings>
   <add key="DimeBridge.Name" value="__DimeBridgeRequest" />
   <add key="DimeBridge.Enable" value="true" />
   <add key="DimeBridge.SizeThreshold" value="0" />
</appSettings>

After this configuration step, the DIME Bridge is ready to transfer the web service result in binary manner. Of course, there is also capability to set-up the "private bridge" attributing the specific method. Note that this is a tightly coupled solution - incorporated (hard coded) into the client/service implementation.

Trace Output

The trace of the checkpoints in the DIME Bridge can be displayed, for instance, on the DbgView program (http://www.sysinternals.com/). Here is its result, for attaching the result (338656 bytes) to the DIME record:

DimeBridge ctor at C:/MyProject/WebApp/web.config: Enable = True,
     Name = __DimeBridgeRequest, Threshold = 5000bytes
DimeBridge.SoapClientMessageBeforeSerialize - DimeBridgeRequest
     for ReturnType=DataContract.GetOrders
DimeBridge ctor at C:/MyProject/WebServices/web.config: Enable = True,
     Name = __DimeBridgeRequest, Threshold = 5000bytes
DimeBridge.SoapServerMessageBeforeSerialize -
     Attaching 'DataContract.GetOrders' [size=338656]
DimeBridge.SoapClientMessageBeforeDeserialize-
     Detaching DimeRecord stream [length=338656]
DimeBridge.SoapClientMessageAfterDeserialize -
     Object 'DataContract.GetOrders' has been returned

and in the case of by-passing the bridge (the object size is under the threshold [908 < 5000]):

DimeBridge ctor at C:/MyProject/WebApp/web.config: Enable = True,
     Name = __DimeBridgeRequest, Threshold = 5000bytes
DimeBridge.SoapClientMessageBeforeSerialize - DimeBridgeRequest
     for ReturnType=DataContract.GetCatalog
DimeBridge ctor at C:/MyProject/WebServices/web.config: Enable = True,
     Name = __DimeBridgeRequest, Threshold = 5000bytes
DimeBridge.SoapServerMessageBeforeSerialize - No attaching,
     the object size is under the threshold [908 < 5000]

Performance

Using the DIME Bridge for Return object with embedded binary images can significantly increase a performance over Internet. For instance, the streamed Return object with the length 5MB can be transferred ~300% faster than the XML formatted one. On the other hand, the small streamed return object like 2KB increase performance approximately about 12%.

Implementation

Implementation of the DIME Bridge is encapsulated into the base class - NopSoapExtension and its overrides. The base class represents the "empty" SoapExtension class with virtual public methods for each SoapMessage stage. If this class is plugged into the SoapExtension pipeline, the SoapMessage is processing though all stages without any action (nop operation).

public class NopSoapExtension : WebServicesExtension 
{
   
   #region virtuals
   public virtual void 
     SoapClientMessageBeforeSerialize(SoapClientMessage message){}
   public virtual void 
     SoapClientMessageAfterSerialize(SoapClientMessage message)
     { Copy(_newStream, _oldStream);}

   public virtual void 
     SoapServerMessageBeforeDeserialize(SoapServerMessage message)
     { Copy(_oldStream, _newStream); }
   public virtual void 
     SoapServerMessageAfterDeserialize(SoapServerMessage message){}

   public virtual void 
     SoapServerMessageBeforeSerialize(SoapServerMessage message){}
   public virtual void 
     SoapServerMessageAfterSerialize(SoapServerMessage message)
     { Copy(_newStream, _oldStream); }

   public virtual void 
     SoapClientMessageBeforeDeserialize(SoapClientMessage message)
     { Copy(_oldStream, _newStream); } 
   public virtual void 
     SoapClientMessageAfterDeserialize(SoapClientMessage message){}
   #endregion

   #region Properties
   public Stream OldStream 
     { get { return _oldStream; } set {_oldStream = value; } }
   public Stream NewStream 
     { get { return _newStream; } set {_newStream = value; } }
   public object Initializer 
     { get { return _initializer; } set {_initializer = value; } }
   #endregion
}

Overriding a proper stage method by the derived class, the SoapMessage workflow is modified based on the application logic. The DimeBridge class inherited this base class to perform the described flowcharts overriding the following methods - stages:

The SoapServerMessageBeforeSerialize method:

public override void 
   SoapServerMessageBeforeSerialize(SoapServerMessage message) 
{
   #region Dime Server - Attaching a return value
   // Check the primary conditions for DIME Response

   if(!bDimeBridgeEnable || !bIsDimeBridgeRequest 
      || message.MethodInfo.IsVoid ||
      ResponseSoapContext.Current == null 
      || (ResponseSoapContext.Current != null && 
      ResponseSoapContext.Current.Attachments.Count != 0))
        return; 

   // get the return value

   object[] parameterValues = 
     message.GetType().BaseType.GetField("parameterValues", 
     binding).GetValue(message) as object[];
   object retval = (parameterValues != null && 
      parameterValues.Length > 0) ? parameterValues[0] : null;

   // Check the secondary conditions for DIME Response

   if(retval == null || (retval != null && 
        (retval.GetType().IsValueType || retval is string)))
      return;

   // serialize a return value

   BinaryFormatter bf = new BinaryFormatter();
   MemoryStream stream = new MemoryStream();
   bf.Serialize(stream, retval);

   // Check the configuration condition for DIME Response

   if(stream.Length > intDimeBridgeSizeThreshold)
   {
      // pass the return value via a Dime record 

      DimeAttachment attachment = new DimeAttachment("image/object", 
                                TypeFormat.MediaType, stream);
      ResponseSoapContext.Current.Attachments.Add(attachment);

      // set the null for return value!

      parameterValues[0] = null;
      Trace.WriteLine(string.Format(@"DimeBridge." + 
         @"SoapServerMessageBeforeSerialize - Attaching '{0}' 
         [size={1}]", retval.GetType().FullName, stream.Length));
   }
   else
   {
      Trace.WriteLine(string.Format("DimeBridge." + 
         "SoapServerMessageBeforeSerialize - " + 
         "No attaching, the object size is under the threshold [{0} < {1}]", 
         stream.Length, intDimeBridgeSizeThreshold));

      // clean-up, the return value is a part of the SoapEnvelope

      stream.Close();
      stream = null;
   }
   #endregion
}

The SoapClientMessageBeforeDeserialize method:

public override void 
  SoapClientMessageBeforeDeserialize(SoapClientMessage message) 
{
   #region Dime Client - Detaching Preprocessor 
   #region validation 
   // clean-up

   FirstAttachment = null;

   // check the message content type

   if(message.ContentType != "application/dime") 
   {
      base.SoapClientMessageBeforeDeserialize(message);
      return;
   }

   // check the WSE client proxy

   WebServicesClientProtocol wsp = 
         message.Client as WebServicesClientProtocol;
   if(wsp != null && wsp.ResponseSoapContext != null)
   {
      base.SoapClientMessageBeforeDeserialize(message);
      return;
   }

   // check the DimeBridge position in the message workflow 

   if(OldStream.CanSeek == true)
      throw new 
        Exception("The DimeBridge is not configured close to the wire.");
   #endregion

   #region Get the SoapEnvelope and FirstAttachment
   // get the network stream length

   object WireStreamLength = OldStream.GetType().GetField("m_ReadBytes", 
                                             binding).GetValue(OldStream);

   // get the network stream

   using(BinaryReader binreader = 
        new BinaryReader(OldStream, Encoding.UTF8)) 
   { 
      MemoryStream ms = new 
       MemoryStream(binreader.ReadBytes(Convert.ToInt32(WireStreamLength)));
      DimeReader dimereader = new DimeReader(ms);

      if(dimereader.CanRead) 
      {
         // SoapEnvelope (the return value is null!)

         base.Copy(dimereader.ReadRecord().BodyStream, NewStream);

         // our first Attachment - return value (body)

         DimeRecord dimerecord = dimereader.ReadRecord();
         if(dimerecord.Type == "image/object") 
         {
            FirstAttachment = dimerecord.BodyStream;
            message.ContentType = "text/xml; image/object";
            Trace.WriteLine(string.Format(@"DimeBridge." + 
               @"SoapClientMessageBeforeDeserialize- " + 
               @"Detaching DimeRecord stream [length={0}]", 
               dimerecord.ContentLength)); 
         }
         else
            throw new Exception("DimeBridge " + 
            "Internal Error - wrong attachment type."); 
      }
   }
   #endregion
   #endregion
}

The SoapClientMessageAfterSerialize method:

public override void 
  SoapClientMessageAfterDeserialize(SoapClientMessage message) 
{
   #region Dime Client - Detaching a return value
   // validation

   if(message.MethodInfo.IsVoid)
      return;

   // get the stream based on the WSE client proxy

   WebServicesClientProtocol wsp = 
           message.Client as WebServicesClientProtocol;
   if(wsp != null && wsp.ResponseSoapContext != null && 
      wsp.ResponseSoapContext.Attachments.Count == 1 
      && FirstAttachment == null) 
   {
      DimeAttachment attachment = 
        wsp.ResponseSoapContext.Attachments[0] as DimeAttachment;
      if(attachment.ContentType == "image/object") 
         FirstAttachment = attachment.Stream;
      else
         throw new Exception("DimeBridge" + 
         " Internal Error - wrong attachment type.");
   }

   if(FirstAttachment != null) 
   {
      using(FirstAttachment) 
      {
         // get parameterValues 

         object[] parameterValues = 
           message.GetType().BaseType.GetField("parameterValues", 
           binding).GetValue(message) as object[];

         // the return value has a mandatory index 0!

         if(parameterValues[0] == null) 
         {
            BinaryFormatter bf = new BinaryFormatter();
            parameterValues[0] = bf.Deserialize(FirstAttachment);

            Trace.WriteLine(string.Format(@"DimeBridge." + 
              @"SoapClientMessageAfterDeserialize - " + 
              @"Object '{0}' has been returned", 
              parameterValues[0].ToString()));
         }
         else
            throw new Exception("DimeBridge Internal Error" + 
            " - the parameterValues[0] is not null");
      }
      FirstAttachment = null;
   }
   #endregion
}

As you can see, the implementation of the DIME Bridge at the message stages is straightforward using the classes from the Microsoft.Web.Services2.Dime namespace. As I described earlier, the "concept core" is based on overwriting the reference of the Return value. This reference is obtained using reflection.

Conclusion

In this article, I showed you a SoapMessage encapsulation into the DIME records based on the message content, for instance, a return value from the WebMethod. The WSE-DIME feature allows to split the SoapHeaders from the message body and handle them separately (like Indigo messages). This concept can be used also for creating your own protocol for router, streaming, etc. in a loosely coupled manner. Included DIME Bridge is the example of this concept, how to significantly increase the response from the web service without touching either the client or server. I hope you will enjoy it.