Click here to Skip to main content
15,867,834 members
Articles / Web Development / IIS
Article

Adding a zip filter to web services

Rate me:
Please Sign up or sign in to vote.
4.88/5 (74 votes)
16 May 20065 min read 393.4K   1K   177   158
Passing large amounts of data through web services can become a huge bottle-neck in a WAN application architecture (i.e., server is on the web), and in any case, a real load on the network traffic. This is one solution for downsizing the network costs.

Introduction

Hold on, I know there is no picture involved, give it a chance all the same, especially if you are a Web services developer ;-)

While writing WebServices, in cases where the client and the server have to communicate through the web, the cost of transferring data through the network is becoming an issue, costly both in the aspect of general responsiveness and the network load.

This lib Zips the SOAP envelope body at the server side, and unzips it back in the client side, behind the scenes, through the use of SOAP filters.

Background

There are some common recommendations that one should follow when writing such an app, amongst which you can find the following list:

  1. Strive to call your server once per page - usually, the actual cost is accessing back and forth to the server, and not passing the actual data.
  2. Cache data on the client side to reduce the need to access the server whenever possible.
  3. Design your app to pass only the exact necessary info (i.e., don't make a "all suppliers" screen, instead make a "search a specific supplier" screen).

Having that said, sometimes you do need to pass a big chunk of data through the web, usually a chubby DataSet, with thousands of rows or alike. This is where my code comes into the picture.

I have written a Zip and a corresponding Unzip filter that you very simply add to your Web service (adding a reference and a filter declaration in the web.config file) and to your WinForms client (again, a reference and a line of code), and you're set.

There is absolutely no code changes to make nor any other overhead.

The code description

The code is a DLL that should be referenced by both projects - client and server, and consists of 3 simple classes:

The ZipWrapper

ZipWrapper is a wrapper class around the ICSharpZipLib library (a freeware code library for Zip utilities taken from here), through which you can select your desired zipping method (BZ2, GZip, tar, etc.) while working with the same interface. (The code for this class was taken from this link, and was mildly amended to fit my needs.)

The Zip type I'm using is GZip; although not the best compression, it is extremely faster than its brother the BZip2, and proved more desired in our case.

The ZipFilter

ZipFilter is a SoapOutputFilter class that is in charge of zipping the body of the SOAP envelope. It also exposes two properties:

  • MinFilterSizeKB: the minimum size of body to start zipping, not all messages are worth zipping.
  • Enabled: is the filter enabled or not.

This class can also determine the zipping method.

The UnZipFilter

UnZipFilter is a SoapInputFilter class that reverses the zipped body back into its original form. The unzip filter will only operate if the Zip filter made its mark on the envelope, and hence doesn't require any configuration params.

Code snippet

We must override the ProcessMessage to change the body of the envelope through the pipeline of the SOAP mechanism. First, we verify that the filter is enabled and the message is big enough to Zip. Then, we create a custom header to announce the client that the message should be unzipped. Next, we create a zipped stream of the envelope's body element. And lastly, we replace the body with the new zipped object.

C#
public override void ProcessMessage(SoapEnvelope envelope)
{
   if ( !m_bEnabled )
      return;

   //adding an attribute to specify that the filter has been 
   //applied on this envelope.
   XmlElement soapHeader = envelope.CreateHeader();

   if ( envelope.Body.InnerText.Length < ( m_MinFilterSize ) )
      return;
   else
      soapHeader.AppendChild(CreateCustomHeader(soapHeader, "1" ));

   //compress the body element.
   MemoryStream result = new 
      MemoryStream( ZipWrapper.Compress( 
      Encoding.UTF8.GetBytes( envelope.Body.InnerXml ) ) );

   //Attach zipped result to the envelope.
   Microsoft.Web.Services2.Attachments.Attachment attch = 
      new Microsoft.Web.Services2.Attachments.Attachment( 
      "APPLICATION/OCTET-STREAM",result );

   envelope.Context.Attachments.Add( attch );

   //remove old body.
   XmlElement newBody = envelope.CreateBody();
   newBody.RemoveAll();

   envelope.SetBodyObject( newBody );
}

Using the lib

Prerequisites

  • ICSharpZipLib.dll - look for the link in the Resources section, WSE 2.0 - Microsoft's web services enhancements package.

Server side usage

  1. Create a web service project.
  2. Right click the project in the Solution Explorer, and choose the WSE 2.0 settings.
  3. In the General tab, enable both checkboxes.
  4. Add a reference to WebServiceZipFilter.dll.
  5. In the web.config file, add under the <microsoft.web.services2> tag:
    XML
    <filters>
       <output>
          <add type="WebServiceZipFilter.ZipFilter, WebServiceZipFilter" />
       </output>
    </filters>

In order to config the filter, you should set its attributes:

C#
// setting minimum zip size requirement to 10KB.
WebServiceZipFilter.ZipFilter.MinFilterSizeKB = 10; 
// enabling the filter
WebServiceZipFilter.ZipFilter.Enabled = true;

Please take note: it is strongly recommended to set these values through values from a configurable part in the web.config, so it can be dynamically changed according to the tuning requirements.

Client side usage

Important note: Many users out there have repeatedly asked me if it is possible to work in duplex mode and if the client side can be configured as the server does. The answer is yes and yes. The list below is an alternative code to add the filter to the pipeline, but you can add the necessary sections to the app.config file and get it done without coding (see the "Server side usage" section above). Regarding the duplex mode, both sides can zip and unzip without a worry!

  1. Create a WinForms client project.
  2. Right click the project in the Solution Explorer, and choose the WSE 2.0 settings.
  3. In the General tab, enable the first checkbox.
  4. Add the web reference of your server.
  5. Create a server proxy (instantiate the ServerNameWSE proxy class).
  6. Add the following code right after the proxy server creation:
    C#
    // creating the server
    TestServer.Service1Wse myServer = 
       new TestClient.TestServer.Service1Wse();
    
    // Adding the unzip filter
    myServer.Pipeline.InputFilters.Add( 
       new WebServiceZipFilter.UnZipFilter() );

Some empirical results

I've conducted my tests on a server, dedicated to me only, that is located in a web farm in my hometown, providing a 5MB download and 1 MB upload, using an ADSL 750 KB connection on a relaxed weekend noon, so the conditions were optimal for the non-zipping tests, and still the differences are quite notable:

Sending 570 records of heterogeneous data took around 2.281 seconds without zipping, compared with 1.843 seconds with zipping (20% reduction).

Sending 10570 records (570 heterogeneous data + 10000 different although similar dummy records) took around 29.43 seconds without zipping, compared with 6.04 seconds with zipping (80% reduction !!!). A completely heterogeneous data would have made the difference negligible.

Future versions

Will there be any? Apparently not. Is this good news? Yes! Why? Because folks at Redmond realized the compression feature is missing, and a few good men released the WCF implementation, and those same guys also released a version for WSE 3.0 here, so my mission is quite done. Everyone is happy.

Resources

History

  • 18-7-2004 v.1.0 - first release.
  • 12-5-2006 v.1.1 - fixed bug reported by tl11, zips according to byte count, not text length.

You are strongly urged to review and comment.

Happy coding.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here



Comments and Discussions

 
GeneralRe: problem with FilterSize check Pin
tl118-May-06 5:57
tl118-May-06 5:57 
GeneralRe: problem with FilterSize check Pin
DaberElay11-May-06 12:14
DaberElay11-May-06 12:14 
GeneralRe: problem with FilterSize check Pin
tl1112-May-06 3:30
tl1112-May-06 3:30 
GeneralRe: problem with FilterSize check Pin
DaberElay12-May-06 4:33
DaberElay12-May-06 4:33 
GeneralRe: problem with FilterSize check CORRECTED Pin
DaberElay14-May-06 1:26
DaberElay14-May-06 1:26 
GeneralNewbie Question. Pin
tobikage5-Apr-06 23:09
tobikage5-Apr-06 23:09 
GeneralCompressing client request sent to server Pin
deepakisonline@yahoo.com30-Mar-06 20:33
deepakisonline@yahoo.com30-Mar-06 20:33 
GeneralRe: Compressing client request sent to server Pin
DaberElay24-Apr-06 10:14
DaberElay24-Apr-06 10:14 
sorry for the late answer, but the answer for you and all future inquirers is that the component works both ways.
you just need to configure the output filter section of the client's app.config and input filter section of the server's web config as demonstrated.

DaberElay.
GeneralThanks Pin
Eniac016-Mar-06 10:51
Eniac016-Mar-06 10:51 
GeneralRe: Thanks Pin
DaberElay18-Mar-06 5:31
DaberElay18-Mar-06 5:31 
GeneralThe other SOAP Extension - Compression Extension Pin
fewfewfewfgwew10-Mar-06 6:02
fewfewfewfgwew10-Mar-06 6:02 
GeneralRe: The other SOAP Extension - Compression Extension Pin
DaberElay10-Mar-06 6:49
DaberElay10-Mar-06 6:49 
GeneralRe: The other SOAP Extension - Compression Extension Pin
fewfewfewfgwew10-Mar-06 6:58
fewfewfewfgwew10-Mar-06 6:58 
QuestionRe: The other SOAP Extension - Compression Extension Pin
kkiran2-Jun-06 9:18
kkiran2-Jun-06 9:18 
AnswerRe: The other SOAP Extension - Compression Extension Pin
DaberElay3-Jun-06 0:02
DaberElay3-Jun-06 0:02 
GeneralWSE question Pin
fewfewfewfgwew4-Mar-06 1:05
fewfewfewfgwew4-Mar-06 1:05 
GeneralRe: WSE question Pin
DaberElay4-Mar-06 10:32
DaberElay4-Mar-06 10:32 
GeneralThanks and a question. Pin
e_Kevin21-Feb-06 21:45
e_Kevin21-Feb-06 21:45 
GeneralRe: Thanks and a question. Pin
DaberElay23-Feb-06 23:04
DaberElay23-Feb-06 23:04 
GeneralThanks + 1 question Pin
SpArtA9-Feb-06 7:17
SpArtA9-Feb-06 7:17 
GeneralRe: Thanks + 1 question Pin
DaberElay9-Feb-06 20:50
DaberElay9-Feb-06 20:50 
GeneralRe: Thanks + 1 question Pin
SpArtA11-Feb-06 4:52
SpArtA11-Feb-06 4:52 
AnswerRe: Thanks + 1 question Pin
SpArtA11-Feb-06 13:26
SpArtA11-Feb-06 13:26 
GeneralAbout WSE 3.0 Pin
szmankiw6-Feb-06 15:09
szmankiw6-Feb-06 15:09 
QuestionWSE filter section? Pin
hungpvtn16-Jan-06 13:00
hungpvtn16-Jan-06 13:00 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.