Click here to Skip to main content
Click here to Skip to main content

Adding a zip filter to web services

, 16 May 2006
Rate this:
Please Sign up or sign in to vote.
Passing large amounts of data through web services can become a huge bottle-neck in a WAN application architecture (i.e., server is on the web), and in any case, a real load on the network traffic. This is one solution for downsizing the network costs.

Introduction

Hold on, I know there is no picture involved, give it a chance all the same, especially if you are a Web services developer Wink | ;-)

While writing WebServices, in cases where the client and the server have to communicate through the web, the cost of transferring data through the network is becoming an issue, costly both in the aspect of general responsiveness and the network load.

This lib Zips the SOAP envelope body at the server side, and unzips it back in the client side, behind the scenes, through the use of SOAP filters.

Background

There are some common recommendations that one should follow when writing such an app, amongst which you can find the following list:

  1. Strive to call your server once per page - usually, the actual cost is accessing back and forth to the server, and not passing the actual data.
  2. Cache data on the client side to reduce the need to access the server whenever possible.
  3. Design your app to pass only the exact necessary info (i.e., don't make a "all suppliers" screen, instead make a "search a specific supplier" screen).

Having that said, sometimes you do need to pass a big chunk of data through the web, usually a chubby DataSet, with thousands of rows or alike. This is where my code comes into the picture.

I have written a Zip and a corresponding Unzip filter that you very simply add to your Web service (adding a reference and a filter declaration in the web.config file) and to your WinForms client (again, a reference and a line of code), and you're set.

There is absolutely no code changes to make nor any other overhead.

The code description

The code is a DLL that should be referenced by both projects - client and server, and consists of 3 simple classes:

The ZipWrapper

ZipWrapper is a wrapper class around the ICSharpZipLib library (a freeware code library for Zip utilities taken from here), through which you can select your desired zipping method (BZ2, GZip, tar, etc.) while working with the same interface. (The code for this class was taken from this link, and was mildly amended to fit my needs.)

The Zip type I'm using is GZip; although not the best compression, it is extremely faster than its brother the BZip2, and proved more desired in our case.

The ZipFilter

ZipFilter is a SoapOutputFilter class that is in charge of zipping the body of the SOAP envelope. It also exposes two properties:

  • MinFilterSizeKB: the minimum size of body to start zipping, not all messages are worth zipping.
  • Enabled: is the filter enabled or not.

This class can also determine the zipping method.

The UnZipFilter

UnZipFilter is a SoapInputFilter class that reverses the zipped body back into its original form. The unzip filter will only operate if the Zip filter made its mark on the envelope, and hence doesn't require any configuration params.

Code snippet

We must override the ProcessMessage to change the body of the envelope through the pipeline of the SOAP mechanism. First, we verify that the filter is enabled and the message is big enough to Zip. Then, we create a custom header to announce the client that the message should be unzipped. Next, we create a zipped stream of the envelope's body element. And lastly, we replace the body with the new zipped object.

public override void ProcessMessage(SoapEnvelope envelope)
{
   if ( !m_bEnabled )
      return;

   //adding an attribute to specify that the filter has been 
   //applied on this envelope.
   XmlElement soapHeader = envelope.CreateHeader();

   if ( envelope.Body.InnerText.Length < ( m_MinFilterSize ) )
      return;
   else
      soapHeader.AppendChild(CreateCustomHeader(soapHeader, "1" ));

   //compress the body element.
   MemoryStream result = new 
      MemoryStream( ZipWrapper.Compress( 
      Encoding.UTF8.GetBytes( envelope.Body.InnerXml ) ) );

   //Attach zipped result to the envelope.
   Microsoft.Web.Services2.Attachments.Attachment attch = 
      new Microsoft.Web.Services2.Attachments.Attachment( 
      "APPLICATION/OCTET-STREAM",result );

   envelope.Context.Attachments.Add( attch );

   //remove old body.
   XmlElement newBody = envelope.CreateBody();
   newBody.RemoveAll();

   envelope.SetBodyObject( newBody );
}

Using the lib

Prerequisites

  • ICSharpZipLib.dll - look for the link in the Resources section, WSE 2.0 - Microsoft's web services enhancements package.

Server side usage

  1. Create a web service project.
  2. Right click the project in the Solution Explorer, and choose the WSE 2.0 settings.
  3. In the General tab, enable both checkboxes.
  4. Add a reference to WebServiceZipFilter.dll.
  5. In the web.config file, add under the <microsoft.web.services2> tag:
    <filters>
       <output>
          <add type="WebServiceZipFilter.ZipFilter, WebServiceZipFilter" />
       </output>
    </filters>

In order to config the filter, you should set its attributes:

// setting minimum zip size requirement to 10KB.
WebServiceZipFilter.ZipFilter.MinFilterSizeKB = 10; 
// enabling the filter
WebServiceZipFilter.ZipFilter.Enabled = true;

Please take note: it is strongly recommended to set these values through values from a configurable part in the web.config, so it can be dynamically changed according to the tuning requirements.

Client side usage

Important note: Many users out there have repeatedly asked me if it is possible to work in duplex mode and if the client side can be configured as the server does. The answer is yes and yes. The list below is an alternative code to add the filter to the pipeline, but you can add the necessary sections to the app.config file and get it done without coding (see the "Server side usage" section above). Regarding the duplex mode, both sides can zip and unzip without a worry!

  1. Create a WinForms client project.
  2. Right click the project in the Solution Explorer, and choose the WSE 2.0 settings.
  3. In the General tab, enable the first checkbox.
  4. Add the web reference of your server.
  5. Create a server proxy (instantiate the ServerNameWSE proxy class).
  6. Add the following code right after the proxy server creation:
    // creating the server
    TestServer.Service1Wse myServer = 
       new TestClient.TestServer.Service1Wse();
    
    // Adding the unzip filter
    myServer.Pipeline.InputFilters.Add( 
       new WebServiceZipFilter.UnZipFilter() );

Some empirical results

I've conducted my tests on a server, dedicated to me only, that is located in a web farm in my hometown, providing a 5MB download and 1 MB upload, using an ADSL 750 KB connection on a relaxed weekend noon, so the conditions were optimal for the non-zipping tests, and still the differences are quite notable:

Sending 570 records of heterogeneous data took around 2.281 seconds without zipping, compared with 1.843 seconds with zipping (20% reduction).

Sending 10570 records (570 heterogeneous data + 10000 different although similar dummy records) took around 29.43 seconds without zipping, compared with 6.04 seconds with zipping (80% reduction !!!). A completely heterogeneous data would have made the difference negligible.

Future versions

Will there be any? Apparently not. Is this good news? Yes! Why? Because folks at Redmond realized the compression feature is missing, and a few good men released the WCF implementation, and those same guys also released a version for WSE 3.0 here, so my mission is quite done. Everyone is happy.

Resources

History

  • 18-7-2004 v.1.0 - first release.
  • 12-5-2006 v.1.1 - fixed bug reported by tl11, zips according to byte count, not text length.

You are strongly urged to review and comment.

Happy coding.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author


Comments and Discussions

 
QuestionThis way VS System.io.compression?? Pinmemberzorro91128-May-08 9:41 
GeneralGreat work - you got my 5! PinmemberEli Gazit3-Sep-07 1:09 
GeneralRe: Great work - you got my 5! PinmemberDaberElay3-Sep-07 1:45 
GeneralWSE 2.0 and WSE 3.0 Interoperability/compatibility Pinmembertcbarton15-Nov-06 15:24 
GeneralRe: WSE 2.0 and WSE 3.0 Interoperability/compatibility PinmemberDaberElay19-Nov-06 1:17 
GeneralRe: WSE 2.0 and WSE 3.0 Interoperability/compatibility Pinmembertcbarton20-Nov-06 9:52 
GeneralPipeline Error Pinmemberolkor8-Aug-06 4:27 
GeneralRe: Pipeline Error [modified] PinmemberDaberElay9-Aug-06 23:31 
GeneralThanks! Pinmemberanichin19-Jun-06 12:41 
Generaltime taken to download without compression is faster than compressed request Pinmemberkkiran2-Jun-06 10:29 
GeneralRe: time taken to download without compression is faster than compressed request PinmemberDaberElay2-Jun-06 16:44 
GeneralGetting Exception during use. Pinmember88Keys15-May-06 16:12 
GeneralRe: Getting Exception during use. PinmemberDaberElay16-May-06 7:05 
GeneralRe: Getting Exception during use. Pinmember88Keys16-May-06 10:12 
GeneralHTTP SOAP Remoting PinmemberTim Friesen14-May-06 16:23 
Questionwhy not "Content-Encoding: gzip" in http response? PinmemberNewSilence12-May-06 16:35 
AnswerRe: why not "Content-Encoding: gzip" in http response? PinmemberJon Rista12-May-06 22:47 
GeneralRe: why not "Content-Encoding: gzip" in http response? Pinmemberpblse29-Apr-07 5:41 
GeneralRe: why not "Content-Encoding: gzip" in http response? PinmemberSimon McKenzie30-Nov-11 17:53 
QuestionHow to change the code to work with WSE 3.0? Pinmemberfewfewfewfgwew7-May-06 12:19 
AnswerRe: How to change the code to work with WSE 3.0? PinmemberDaberElay14-May-06 2:29 
QuestionDIME Attachments? PinmemberOskar Austegard5-May-06 5:55 
AnswerRe: DIME Attachments? PinmemberDaberElay7-May-06 0:54 
QuestionRe: DIME Attachments? PinmemberOskar Austegard16-May-06 5:08 
AnswerRe: DIME Attachments? CORRECTED PinmemberDaberElay16-May-06 7:28 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web03 | 2.8.141220.1 | Last Updated 16 May 2006
Article Copyright 2004 by DaberElay
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid