Adding a zip filter to web services

• Download source files - 9.97 Kb
• Download demo project - 42.6 Kb

Introduction

Hold on, I know there is no picture involved, give it a chance all the same, especially if you are a Web services developer ;-)

While writing WebServices, in cases where the client and the server have to communicate through the web, the cost of transferring data through the network is becoming an issue, costly both in the aspect of general responsiveness and the network load.

This lib Zips the SOAP envelope body at the server side, and unzips it back in the client side, behind the scenes, through the use of SOAP filters.

Background

There are some common recommendations that one should follow when writing such an app, amongst which you can find the following list:

1. Strive to call your server once per page - usually, the actual cost is accessing back and forth to the server, and not passing the actual data.
2. Cache data on the client side to reduce the need to access the server whenever possible.
3. Design your app to pass only the exact necessary info (i.e., don't make a "all suppliers" screen, instead make a "search a specific supplier" screen).

Having that said, sometimes you do need to pass a big chunk of data through the web, usually a chubby DataSet, with thousands of rows or alike. This is where my code comes into the picture.

I have written a Zip and a corresponding Unzip filter that you very simply add to your Web service (adding a reference and a filter declaration in the web.config file) and to your WinForms client (again, a reference and a line of code), and you're set.

There is absolutely no code changes to make nor any other overhead.

The code description

The code is a DLL that should be referenced by both projects - client and server, and consists of 3 simple classes:

The ZipWrapper

ZipWrapper is a wrapper class around the ICSharpZipLib library (a freeware code library for Zip utilities taken from here), through which you can select your desired zipping method (BZ2, GZip, tar, etc.) while working with the same interface. (The code for this class was taken from this link, and was mildly amended to fit my needs.)

The Zip type I'm using is GZip; although not the best compression, it is extremely faster than its brother the BZip2, and proved more desired in our case.

The ZipFilter

ZipFilter is a SoapOutputFilter class that is in charge of zipping the body of the SOAP envelope. It also exposes two properties:

• MinFilterSizeKB: the minimum size of body to start zipping, not all messages are worth zipping.
• Enabled: is the filter enabled or not.

This class can also determine the zipping method.

The UnZipFilter

UnZipFilter is a SoapInputFilter class that reverses the zipped body back into its original form. The unzip filter will only operate if the Zip filter made its mark on the envelope, and hence doesn't require any configuration params.

Code snippet

We must override the ProcessMessage to change the body of the envelope through the pipeline of the SOAP mechanism. First, we verify that the filter is enabled and the message is big enough to Zip. Then, we create a custom header to announce the client that the message should be unzipped. Next, we create a zipped stream of the envelope's body element. And lastly, we replace the body with the new zipped object.

public override void ProcessMessage(SoapEnvelope envelope)
{
   if ( !m_bEnabled )
      return;

   //adding an attribute to specify that the filter has been 

   //applied on this envelope.

   XmlElement soapHeader = envelope.CreateHeader();

   if ( envelope.Body.InnerText.Length < ( m_MinFilterSize ) )
      return;
   else
      soapHeader.AppendChild(CreateCustomHeader(soapHeader, "1" ));

   //compress the body element.

   MemoryStream result = new 
      MemoryStream( ZipWrapper.Compress( 
      Encoding.UTF8.GetBytes( envelope.Body.InnerXml ) ) );

   //Attach zipped result to the envelope.

   Microsoft.Web.Services2.Attachments.Attachment attch = 
      new Microsoft.Web.Services2.Attachments.Attachment( 
      "APPLICATION/OCTET-STREAM",result );

   envelope.Context.Attachments.Add( attch );

   //remove old body.

   XmlElement newBody = envelope.CreateBody();
   newBody.RemoveAll();

   envelope.SetBodyObject( newBody );
}

Using the lib

Prerequisites

• ICSharpZipLib.dll - look for the link in the Resources section, WSE 2.0 - Microsoft's web services enhancements package.

Server side usage

1. Create a web service project.
2. Right click the project in the Solution Explorer, and choose the WSE 2.0 settings.
3. In the General tab, enable both checkboxes.
4. Add a reference to WebServiceZipFilter.dll.
5. In the web.config file, add under the <microsoft.web.services2> tag:

   <filters>
      <output>
         <add type="WebServiceZipFilter.ZipFilter, WebServiceZipFilter" />
      </output>
   </filters>

In order to config the filter, you should set its attributes:

// setting minimum zip size requirement to 10KB.

WebServiceZipFilter.ZipFilter.MinFilterSizeKB = 10; 
// enabling the filter

WebServiceZipFilter.ZipFilter.Enabled = true;

Please take note: it is strongly recommended to set these values through values from a configurable part in the web.config, so it can be dynamically changed according to the tuning requirements.

Client side usage

Important note: Many users out there have repeatedly asked me if it is possible to work in duplex mode and if the client side can be configured as the server does. The answer is yes and yes. The list below is an alternative code to add the filter to the pipeline, but you can add the necessary sections to the app.config file and get it done without coding (see the "Server side usage" section above). Regarding the duplex mode, both sides can zip and unzip without a worry!

1. Create a WinForms client project.
2. Right click the project in the Solution Explorer, and choose the WSE 2.0 settings.
3. In the General tab, enable the first checkbox.
4. Add the web reference of your server.
5. Create a server proxy (instantiate the ServerNameWSE proxy class).
6. Add the following code right after the proxy server creation:

   // creating the server
   
   TestServer.Service1Wse myServer = 
      new TestClient.TestServer.Service1Wse();
   
   // Adding the unzip filter
   
   myServer.Pipeline.InputFilters.Add( 
      new WebServiceZipFilter.UnZipFilter() );

Some empirical results

I've conducted my tests on a server, dedicated to me only, that is located in a web farm in my hometown, providing a 5MB download and 1 MB upload, using an ADSL 750 KB connection on a relaxed weekend noon, so the conditions were optimal for the non-zipping tests, and still the differences are quite notable:

Sending 570 records of heterogeneous data took around 2.281 seconds without zipping, compared with 1.843 seconds with zipping (20% reduction).

Sending 10570 records (570 heterogeneous data + 10000 different although similar dummy records) took around 29.43 seconds without zipping, compared with 6.04 seconds with zipping (80% reduction !!!). A completely heterogeneous data would have made the difference negligible.

Future versions

Will there be any? Apparently not. Is this good news? Yes! Why? Because folks at Redmond realized the compression feature is missing, and a few good men released the WCF implementation, and those same guys also released a version for WSE 3.0 here, so my mission is quite done. Everyone is happy.

Resources

• Compression helper class.
• ICSharpZipLib.
• Download WSE 2.0.
• WSE 3.0 implementation of a Zip filter.
• WCF implementation of a Zip filter.

History

• 18-7-2004 v.1.0 - first release.
• 12-5-2006 v.1.1 - fixed bug reported by tl11, zips according to byte count, not text length.

You are strongly urged to review and comment.

Happy coding.