Secure Web Services via Message oriented Middleware

• Download source files and executables - 792Kb

Introduction

This article describes the exploitation of the pluggable transport infrastructure provided as part of ASP.NET Web Services, which allows developers to deliver Soap messages built in the document/literal format to be delivered to an end point via mechanisms other than default HTTP transport.

In this article, a framework that supports the addition of alternative physical delivery mechanisms is presented, and two such implementations are included – MSMQ and Websphere MQ are chosen to illustrate a store-forward capability for a more 'guaranteed' Web Services request delivery. The framework that will be developed integrates the Web Services Enhancements (WSE) 1.0 for secure 2-way transport of messages over non-HTTP mechanisms, thereby giving the developer a choice of 'trusted' (clear message) or 'untrusted' (secure message) delivery to the chosen endpoint.

Soap was designed as a transport neutral protocol to be used in combination with a variety of transport protocols such as HTTP(S), SMTP, FTP etc. to deliver structured and typed information between two participants. A great deal of work has gone into making ASP.NET Web Services support Soap message generation in a way that promotes interoperability within heterogeneous environments (for example .NET consuming a BEA Weblogic Server hosted Web Service) but to date the standard access to Web Services within the Microsoft arena has been almost exclusively geared to HTTP.

Soap over HTTP

There are of course strengths and weaknesses of the HTTP transport – a definite plus point is the maturity of the associated security infrastructure, which has risen on the back of the ubiquity of the web in general. Conversely, a weakness is seen to be the lack of any form of guaranteed and reliable (single message only) delivery within the protocol, which has spawned solutions such as HTTPR from IBM. Add to this the fact that the back-end Service has to physically be on line for the request to be received over HTTP, and there are some pretty compelling reasons to want to find alternative ways to "get the message across".

Soap over alternative transports

While ASP.NET does a lot to promote the HTTP-centric binding to Web Services, HTTP is not the only protocol available for transporting Soap. Indeed, a little talked-of area is the Web Services infrastructure in place within the .NET framework that allows message delivery mechanism and endpoint to be altered either statically (design time) or dynamically (run time). The real power of this infrastructure – named "pluggable protocols" by Microsoft – is that the transport method can be altered underneath the logical client request to direct Soap via any sensible medium the developer cares to implement.

What this case study will demonstrate is the ability to effectively switch transports underneath ASP.NET client between HTTP, MSMQ and Websphere MQ. It will demonstrate how trivial it is to alter the 'target' of a Soap Web Service request from a standard IIS hosted variant, to a listener waiting on an MSMQ or MQ queue. We'll demonstrate the exercising of API using standard types like strings, arrays, binary data and custom serializable types. We'll also show how this support can be achieved in an asynchronous fashion.

Securing Soap

It is likely that the queuing style transports would find use within the enterprise i.e. within the trusted network of an organization – with their commensurate (typically reduced) security requirements. However, by allowing non-HTTP delivery options, we've potentially taken away a layer of proven security that we ought to investigate a replacement for. Fortunately, a great deal of work has been undertaken in developing some joint standards for a new generation of web services protocols covering (amongst others) security, message routing and message attachments. This push really centers round support for securing messages between multiple processing nodes involved in a business transaction in a transport-agnostic way – without having to rely on technologies like SSL which only work well point-to-point. This effort has culminated recently in the release of Microsoft's implementation of the aforementioned standards in the form of the Web Services Enhancements (WSE) 1.0 package. So, the case study will also show how the implementation of WS-Security within the WSE 1.0 can be employed to secure (in our case digitally sign and symmetrically encrypt) the messages produced by ASP.NET clients and Servers instead of using the native security mechanisms inherent in the two queuing products. We'll also look at the DIME support provided by the WSE as an alternative way to send binary data, which was developed for transmission of large binary data streams.

Article goals

Hopefully by the end of the case study the following aspects will be clarified:

• How to write new delivery protocols within the ASP.NET pluggable protocol infrastructure
• How to access MSMQ and Websphere MQ from C# from within this infrastructure
• How to integrate WSE 1.0 with Web Services at both a high and a low (direct use of WSE filer pipelines) level to secure data
• How to use the WSE when transferring binary data
• How to drive queued delivery of Soap messages asynchronously from the client

Context for work

The original context for the work involved a Proof of Concept project designed to determine how easily a .NET client could access an existing Java-oriented backend using Web Services for interoperability, but via IBM Websphere MQ rather than HTTP. This article provides a .NET oriented back-end on the grounds that in doing so we can keep the skill sets covering the article constrained to .NET and can demonstrate secure communications more easily by virtue of having WSE at both ends. However, idiosyncrasies of interoperation with a Java service implementation will be identified at appropriate points throughout the article.

System Requirements

The built solution will require the following deployment environment:

1. Windows 2000 SP2 or Windows XP Pro with MSMQ and IIS installed

2. .NET Framework 1.0 (SP1 or above)

3. WSE 1.0 SP1 runtime

4. (For MQ support only ) Websphere MQ 5.3 Trial Release [2]

5. (For MQ support only) VB6 runtime [10]

In order to rebuild the solution binaries, you will require the following additional tools:

6. VS.NET (C#)

7. WSE 1.0 SP1 – full installation recommended

8. VS.NET WSE Settings Tool (optional) [8]

Skills requirement

From a skills perspective, the reader should have:

• familiarity with C# to an intermediate level. exposure to writing and consuming a basic Web Service in .NET, since we will be examining the client side infrastructure (including the WSDL-generated proxy) in some detail.
• some appreciation of the work being done in the "second-generation" Web Services arena like WS-Security and WS-Attachments / DIME would be advantageous.
• if wanting to use Websphere MQ as a transport (it can be disabled in the sample application configuration file), it would be highly beneficial to have installed the trial release [2] and configured the standard set up.
• a basic understanding of Unified Modeling Language (UML) notation.

Article goals

Essentially we aim to make progress in the following areas:

• To split what at first sight seems a tightly coupled relationship between the Soap serialization of the message and the delivery of said message via http to an end point.
• We'd like to reuse the .NET Web Services serialization portion such that we never have to hand-craft Soap ourselves.
• We aim to tap into the delivery portion of the infrastructure with the goal being to deliver Soap requests to a queue and to receive Soap responses from a queue. We want to support MSMQ and Websphere MQ in particular.
• To do this, we'd like the client to be able to "express" the queue it wishes to deliver requests to in as standard a way as possible i.e. with minimal change from the way in which one might describe a URL for example.
• Having achieved this, we'd like to see what impact / benefits the use of the WSE to condition Soap messages will bring for us – in securing message flows, and in allowing the transfer of binary attachments "outside" the Soap envelope.
• Ideally we'd like to be able to package the resultant code in a way that minimizes work for developers who wish to include queuing-oriented transports in their solutions.
• We want to ensure that the basic framework operates in a multiple concurrent usage environment.

WSDL and Web Services proxy generation in .NET

Interface contracts are described using WSDL (akin to IDL for COM). Once this WSDL is available at a URL or within a file, a Web Services proxy class can be generated either by directly using the standalone tool WSDL.EXE shipped with the SDK or implicitly when using the Visual Studio .NET 'Add Web Reference' facility:

Figure 1: The WSDL-2-Proxy process

The generated proxy file (usually called reference.cs) will live in a subfolder of the project off a subfolder called \Web References and includes a class typically derived from SoapHttpClientProtocol. The generated class does a very good job of hiding much of the complexity of the serialization and transport specifics pertinent to the standard XML Web Services invocation. Within the resultant generated proxy file, one may typically see a combination of synchronous and asynchronous forms of methods - for example the following shows a basic generated (synchronous) method:

[SoapDocumentMethodAttribute(...)]
public string IsCollaboratorRegistered(string vstrEnvUUID)
{
  object[] results = this.Invoke("IsCollaboratorRegistered", new object[]
                     { vstrEnvUUID });
  return ((string)(results[0]));
}

With the standard proxy in place, clients can the access the Web Service at the appropriate URL using code similar to the following:

// Get an instance of the proxy

localhost.RegService objWSReg = localhost.RegService();    
// Endpoint can be varied at runtime etc.

objWSReg.Url = "http://192.168.32.1/MyService.asmx";    
// Make the call

string strRetXML 
     = objWSReg.IsCollaboratorRegistered("12345678901234567890123456789012");

Being able to vary the URL and to describe this URL in terms of a "protocol scheme" and an "end point" is the key aspect of the approach.

At runtime, the single line within the proxy containing the call this.Invoke( … ) hides a lot of work that is segmented into the following basic pieces:

• Transport selection / confirmation of support
• Soap serialization of API call
• Transport protocol driving

We'll break this down further later in the article to show how the pluggable protocol infrastructure is combined with Soap serialization to build and deliver messages. In particular there are several methods that can be overridden within the proxy to tap into or even replace the message transport protocol process.

Defining Internet and other Network Transports for Web Services

While the de-facto delivery protocol for Soap messages is HTTP, there is an infrastructure in place within the .NET framework that allows message delivery mechanisms to be altered either statically (design time) or dynamically (run time).

The .NET Framework uses three specific classes to provide the information required to access Internet / Network resources through a request / response model:

1. the Uri class, which contains the URI of the Internet resource you are seeking (including the protocol scheme e.g. http://www.microsoft.com/webservices/stockcheck.asmx ),

2. the WebRequest class, which encapsulates a request for access to a network resource,

3. the WebResponse class, which provides a container for the incoming response from the network resource.

Client applications create WebRequest instances by passing the URI of the network resource to the WebRequest.Create() method. This static method creates a WebRequest instance for a specific protocol, such as HTTP. The WebRequest instance that is returned provides access to properties that control both the request to the server and access to the data stream that is sent when the request is made. The GetResponse() method on the WebRequest instance sends the request from the client application to the server identified in the URI.

[NOTE: A Uniform Resource Identifier (URI) is a compact string of characters for identifying an abstract or physical resource as defined in RFC2396. The "scheme" is a standard part of the URI, which can be broken down into a number of distinct segments. All network identifiers are capable of mapping into one of the parts. The scheme is typically the protocol identifier e.g. http, ftp etc.]

Out of the box there is support for three specific protocol schemes - "http:", "https:" and "file:". We will create new versions of the WebRequest and WebResponse classes to provide support for a further two:

• "msmq:" Support a request response model where the target is a local or remote MSMQ Queue
• "mq:" Support a request response model where the target is an MQ Queue Manager / Queue Name combination

One can imagine that additional support could be added for other schemes such as "mailto:" or "ftp:" for SMTP and FTP respectively.

Note at this point that attempting to create a WebRequest for a Uri involving a scheme that is unsupported:

Uri uri = "mq://accountq1";
WebRequest newWebRequest = WebRequest.Create(uri);

will yield an exception of the form NotSupportedException which indicates that the request scheme specified in requestUri is not registered. It is this issue we must address next in order to successfully introduce new schemes. It should be becoming clear now that our goal here is to allow a client to be able to identify a queue as the Uri of a request and be able to execute a method call against that Uri as they would against an http-based end point.

Defining custom transports

The WebRequest and WebResponse classes form the basis of the "pluggable protocols" infrastructure. That means that in order to specify a new protocol like "xxx://", one should create at minimum derivations of each of these base classes, plus a class that implements IWebRequestCreate . The following diagram depicts the basic model involved in production of new protocols:

Figure 2: The Object model for pluggable protocols

The main operations and attributes have been included to show the key aspects of any implementation.

Common Aspects

Several of the attributes and operations that can be overridden by descendants of the WebRequest and WebResponse classes are fairly redundant when it comes to implementing non-internet-specific transports. As well as this, there is a large degree of commonality in the two queue implementations. This gives us the opportunity to abstract away those commonalities and redundancies into a single intermediate class. As an example, one could imagine that the Method property that is used as part of every web type request (GET, POST etc.) would find no relevance in the new scheme implementations we are proposing.

We define the following new derived types to extend the hierarchy:

Type	Base Type	Purpose
MyBaseWebRequest	WebRequest	Contains overrides common to MQ and MSMQ implementations
MyBaseWebResponse	WebResponse	Contains overrides common to MQ and MSMQ implementations
MSMQWebRequest	MyBaseWebRequest	MSMQ implementation of WebRequest (sealed)
MSMQWebResponse	MyBaseWebResponse	MSMQ implementation of WebResponse (sealed)
MQWebRequest	MyBaseWebRequest	MQ implementation of WebRequest (sealed)
MQWebResponse	MyBaseWebResponse	MQ implementation of WebResponse (sealed)

Three criteria must be met in order for a protocol-specific class to be used as a pluggable protocol:

1. Preparing for scheme registration

The implementation must incorporate a version of the IWebRequestCreate interface e.g. for MSMQ:

public class MSMQRequestCreator : IWebRequestCreate
{
  public WebRequest Create(Uri uri)
  {
    // Create a new MSMQ request object

    return new MSMQWebRequest(uri);
  }
}
            

2. Scheme registration and Uri creation

Descendant classes of WebRequest that wish to implement new protocols must be registered with the WebRequest class in order to be allowed to manage the details of making the actual connections to non-standard network resources like Queues.

This is usually achieved in a similar fashion to the following:

// Register the "mq:" prefix here for WebRequest

MQRequestCreator objCreator = new MQRequestCreator();
WebRequest.RegisterPrefix("mq", objCreator);

This effectively allows the subsequent hooks up of the Create code for an instance of the new scheme, such that the code we mentioned above:

Uri uri = "mq://accountq1";
WebRequest newWebRequest = WebRequest.Create(uri);

will then work since the new scheme is known to .NET Web Services infrastructure.

3. Implementation of key overridden members

The derived classes must override the abstract methods and properties of WebRequest and WebResponse to provide the pluggable interface. These members are well documented for .NET Framework 1.0 - the relevant subset are shown here:

Member	Purpose
Method	Usually associated with a form of protocol method to use in this request e.g. PUT, GET etc. Not used here.
Headers	Gets or sets the collection of header name/value pairs associated with the request. Not used here.
ContentLength	Contains the number of bytes of data sent to the Internet resource by the WebRequest instance.
ContentType	When overridden in a descendant class, gets or sets the content type of the request data being sent. In our case always text/xml.
Credentials	Gets or sets the network credentials used for authenticating the request with the Internet resource. Unused in our case.
PreAuthenticate	Indicates whether to preauthenticate the request. Unused in our case.
Proxy	Gets or sets the network proxy to use to access this Internet resource. Unused in our case.
GetRequestStream	Typically creates and returns a Stream for writing data (Soap message) to the Internet resource. Implemented in our base class.
BeginGetRequestStream	Asynchronous version of the above. Not supported in our case.
EndGetRequestStream	Asynchronous version of the above. Not supported in our case.
GetResponse	Returns a response to a request. This override forms the work horse of our implementations of MSMQ and MQ as this is where the protocol-specific code is accessed.
BeginGetResponse	Asynchronous version of the above. Not supported in our case.
EndGetResponse	Asynchronous version of the above. Not supported in our case.
Abort	Cancels an asynchronous request started with the BeginGetResponse method. Not supported in our case.

As implied above, we'll implement any overrides common to MQ and MSMQ versions in MyBaseWebRequest / MyBaseWebResponse , and the specifics in the appropriate derived classes.

Execution of a WebRequest-derived instance

Here we'll attempt to map the general client-side serialization and transportation process onto the discrete calls made by the .NET Web Services infrastructure during a synchronous web method call.

The diagram depicts the following sequence of events on the “request” execution path:

Figure 3: Interaction diagram for WebRequest

Key facets of operation include:

1. Overriding GetWebRequest in our generated Soap proxy (derived from SoapHttpClientProtocol) allows us to hook in a registration (for schemes of msmq / mq) and creation of the appropriate WebRequest instance. If the scheme is "msmq", this creation is achieved via the MSMQRequestCreator instance.
2. The .NET Web Services infrastructure calls a private method, BeforeSerialize(), which triggers the getting and setting of various member data within the instance of our common WebRequest-derived class, setting the method (irrelevant for us but we have to support the operation - returning a NotSupported exception causes the .NET Web Services infrastructure to reject the whole request!) and getting the (empty) web headers collection.
3. Next, the GetRequestStream() method is called on our WebRequest-derived class where we simply create a stream (a MemoryStream in this case) and return it to the infrastructure.
4. At this point, the infrastructure begins the Soap serialization process and deposits the results into our stream. A very huge problem at this point is that in order to get the serialized stream pushed out onto the wire for transmission, the standard processing involves closing the stream! Given we've yet to reach the point where we want to use the stream to push at a queue this is at first glance a bit of a showstopper!! So, let's conveniently ignore it for now :-). The section entitled "Tapping into the serialized Soap stream" explains the issues and possible solutions.
5. Once the stream has been prepared, the infrastructure then calls our overridden GetResponse() in the MSMQWebRequest class instance. Here the specifics of dealing with queues via the appropriate mechanism (System.Messaging for MSMQ or the IBM-provided COM library for MQ) are employed. This will undoubtedly involve writing the Soap message and awaiting a response to it from the back end service. The specifics of MSMQ and MQ transports are covered in the next section. Yet again we conveniently glossed over access to the stream at this point...
6. Whatever the specifics of the MSMQ / MQ work, the result is to either deliver a response object of the appropriate type (e.g. MSMQWebResponse), or raise an instance of a WebException if a failure has occurred in the transport. These types of exception are discussed later in the article.

If you do get the chance, I'd strongly recommend a little trawl through the debugger, setting breakpoints all over the code as it really is instructive as to the ordering of events in the overall process.

Implementing MSMQ and Websphere MQ transports

Assuming we can access the serialized Soap message (which we deal with below), we need to hook in a protocol handler, based on WebRequest. This section highlights the key facets of the implementation and expands the information given in the Sequence diagram above.

Miscellaneous common Request aspects

Several overridden properties are not required for operation and so may be allowed to throw a NotSupported exception e.g.:

public override IWebProxy Proxy 
{
  /* override */ get { throw new NotSupportedException(); }
  /* override */ set { throw new NotSupportedException(); }
}            
            

Others, like ContentLength, ContentType and Timeout etc. are required and are set up correctly:

public override int Timeout
{
  /* override */ get { return m_intTimeout; }
  /* override */ set { m_intTimeout = value; }
}            
            

These aspects, shared by MSMQ and MQ implementations, are abstracted into the MyBaseWebRequest class.

Providing a Stream for serialization

MyBaseWebRequest contains a protected data member (accessible in derived classes) called m_RequestStream, which is set up when called by the client-side Web Services infrastructure:

public override Stream GetRequestStream()
{    
    if (m_RequestStream == null)
        m_RequestStream = new MemoryStream();
    else
        throw new InvalidOperationException("Request stream already retrieved");
    return m_RequestStream;        
}

As we would not expect the stream to be provided twice in the course of a single Web Services method call, and we also do not anticipate ever sharing WebRequest instances amongst multiple proxies, we provide a little protection against this happening.

NOTE: I hope this is obvious but just in case, we don't do the serialization ourselves. We simply provide a container (stream) for the Web Services infrastructure to do so.

Miscellaneous common Response aspects

In addition, MyBaseWebResponse is utilized unchanged by both MSMQ and MQ implementations. This class contains information more akin to the Internet resource world (status code, status description etc), and in fact since all errors are thrown as exceptions up the stack, the only key methods within this class are SetDownloadStream(), which allows a WebRequest caller to deposit a response Soap stream within the class instance, and GetResponseStream() which returns this response stream to the calling .NET Web Services response processing infrastructure. Assuming a successful call, once the stream has been returned standard Soap de-serialization takes place.

Common facets of a Queue-oriented transport

There are several operational aspects we would like to incorporate into our queue management.

A "Request Timeout"

The standard generated proxy includes a timeout data member, which has a default value and can be set prior to invocation of the actual Web Service call. We'd like that timeout to be honored by the Queuing implementations and we'd like them to raise exceptions like the standard protocol handlers would in the event of a timeout.

Request-Response matching (at the protocol level)

In a multi user scenario, it is likely that many requests will be outstanding at any one time, and that responses to those requests will be received concurrently. In this situation there is a requirement to be able to match up the response to the associated request. There are principally two options available to support this feature:

1. Support some form of Request ID at the application level API (i.e. in WSDL) that the Service can act on.

2. Use any native support provided by the infrastructure to correlate the request to the response.

While the first option would give us the ability to support a transport-neutral way of correlating messages, it would be at the expense of the queue protocol handlers, which would need to effectively peek at all the messages in the queue to identify a response. Taking option 1 also perverts the purity of the Application API somewhat, in that each method call is required to support an extra parameter just to allow the consolidation to occur. Finally, both forms of queuing provide support for correlation by including a CorrelationID slot within their version of the Queue Message. So after much agonizing, it was decided that the protocol would handle the correlation of messages (this decision may need to be revisited if more transports are added, and typically if the decision were reversed, the outcome would be the addition of custom headers into the Soap envelope to support the correlation approach). This correlation handling needs to be performed within the GetResponse() method of each implementation.

A "Response Uri"

Although the transportation infrastructure was developed explicitly to support non-standard protocols, it predominantly revolves around a target endpoint and assumes a single point of contact around the delivery of the request and response. Actually, when it comes to queuing we'd like to be able to give a client the ability to describe both a request Uri (endpoint to hit) and a response Uri, which the endpoint service could effectively call back on with the results of the call. What this means in our parlance is that we want be able to describe both a request queue and a response queue for the target of the request and target of the response respectively.

Figure 4: Object Model for MySoapClientProtocol

The best approach appears to be to derive from SoapHttpClientProtocol, and introduce a strict interface (ISoapClientProtocol) to police the management of a new data member called ResponseUrl (so named in order to bring a symmetry to the request - response Url pair).

The resulting class is called MySoapHttpClientProtocol. Introduction of this class requires that each generated proxy derive from it in order to be able to access the new property. This will require a manual re-pointing of the associated reference.cs file.

A point worth bearing in mind is that any changes made to this proxy are lost during regeneration of the proxy using the VS.NET menu item "Update Web Reference" or when the Web Reference is removed. It may be worth backing up the reference.cs file once the changes have been hand-made to the proxy so that they may be subsequently re-applied if, for example, the underlying Web Service API definition is required to be extended and hence the proxy needs re-generating.

In terms of how the response queue information might be used, one could imagine an implementation within the handlers GetResponse() method, which waited on the relevant response queue for a message corresponding to the request it had sent. This typically involves using Queue-specific API such as ReceiveByCorrelationID().

"Uri representation" of a Queue

As was suggested in the section entitled "Defining Internet and other Network Transports for Web Services" that the Uri class is the means by which we identify our endpoint service using URI syntax. The URI syntax is dependent upon the protocol scheme. In general, absolute URI are written as follows:

<scheme>:<scheme-specific-part>

The URI syntax does not require that the scheme-specific-part have any general structure or set of semantics, which is common amongst all URI. However, a subset of URI do share a common syntax for representing hierarchical relationships within the namespace. This "generic URI" syntax consists of a sequence of four main components:

<scheme>://<authority><path>?<query>

Well know examples include http formats like:

http://www.microsoft.com/webservices/testservice.asmx?wsdl

In order to make use of the pluggable protocol infrastructure we are required to map MSMQ and MQ style queue names into this syntax. As can be seen in the next section, this can be achieved with varying degrees of ease depending on which queuing implementation we are talking about.

Handling Errors

As part of the general .NET Web Services pluggable protocol infrastructure, version of the WebRequest and WebResponse classes throw both system exceptions like ArgumentException and transport-specific exceptions (typically these take the form of WebExceptions thrown by the specifc overridden GetResponse() method). We'd like to map our queue-related failures into this exception structure as it nicely hides the differences between the worlds of MQ and MSMQ exception reporting. The WebException class derives from System.InvalidOperationException and contains some extensions for supporting detailed error information:

Figure 5: WebException Object Model

The two queuing protocols support this approach, using the Status property to reflect the following WebExceptionStatus enumerated returns:

Status	Description
ConnectFailure	The remote service could not be contacted at the transport level. In our case this means that the opening of the request queue failed for a reason other than inability to resolve the queue name to a resource.
NameResolutionFailure	The name service could not resolve the host name. In our case, this means that the request Queue (or Queue Manager in the case of MQ – see below) identified in the Uri was unable to be mapped to a resource.
ProtocolError	The response received from the server was complete but the indicated an error at the protocol level. In our case, this means that a valid WebResponse-derived object was returned but that it will contain information pertaining to a failure within the relevant queued transport. This object will be made available as part of the WebException.
ReceiveFailure	A complete response was not received from the remote Server. In our case, this means that the reading of the response queue failed for a reason other than a timeout or inability to resolve the queue name to a resource.
SendFailure	A complete request could not be sent to the remote Server. In our case, this means that the placing of the message on the request queue failed after the queue had been successfully opened (connected).
Timeout	No response was received within the timeout period set for the request. In our case, this means that the request was placed successfully on the request queue by the relevant transport but no corresponding response was identified on the response queue before the timeout period was reached.

These status codes are mapped from the specific MSMQ / MQ failure codes within the relevant protocol specific sections below.

MSMQ-specific implementation

All the work is done in the overridden GetResponse() method.

Accessing MSMQ functions

In order to make use of MSMQ, the System.Messaging assembly must be included in the references for the project. For the uninitiated, this provides an easy-to-use object model for interaction with MSMQ, which covers connection to and administration of local and remote message queues, as well as the sending and receiving of messages via those queues.

Figure 6: System.Messaging

Queue Name Format

There are several ways to identify an MSMQ queue – by Path, by Format Name or by Queue Label. The Format Name is the native, unique representation of the Queue. The Path is a friendly representation of the Queue, which must be resolved to a Format Name by the MSMQ Directory Services. Normally the format of a Queue Path specification as understood by MSMQ takes one of the following forms:

Queue Type	Path Syntax
Public queue	MachineName\QueueName
Private queue	MachineName\Private$\QueueName, .\Private$\QueueName
Journal queue	MachineName\QueueName\Journal$
Machine journal queue	MachineName\Journal$
Machine dead-letter queue	MachineName\Deadletter$
Machine transactional dead-letter queue	MachineName\XactDeadletter$

We need to be able to express this kind of Path level information in an Uri format to work with the pluggable protocol infrastructure. Unfortunately there does not appear to be a particularly nice fit with MSMQ Path Names in this case. The RFC 2396 entitled "Uniform Resource Identifiers (URI): Generic Syntax" suggests that several characters are disallowed from the URI specification and hence removed during parsing by implementations such as the .NET Uri class. One of these is the "\" character, which is a problem for us given the format of an MSMQ queue name. In fact, this ultimately forces us to describe our MSMQ queue resources using an alternative means – in our case using "/" to segment parts of the queue name hierarchy. This will be held at Uri level in a valid form:

msmq://./private$/QueueName

While this then works well with Uri, it requires two things:

• The developer to remember to perform this alteration before calling the Web Service function using this transport
• The WebRequest implementation must convert this Uri-friendly format back to one capable of being used by MSMQ

However, we'd like to describe the queue in the standard way – as it may well be being used elsewhere in the wider context of our application, may be read from a configuration file etc. So we need to provide some assistance. As we'd ideally like to not have to differentiate between caller code making requests against MSMQ and code doing so against MQ (or indeed HTTP etc.), we will add a new method called FormatCustomUri() to the ISoapClientProtocol interface and the MySoapHttpClientProtocol implementation class. This will allow callers of the proxy to continue to make use of MSMQ-friendly queue names, but wraps up a simple Uri-friendly alteration before the string is fed to the Uri class:

// Get Queue name (from configuration file perhaps)

string strMyQ = ".\\private$\\myq1";

// Create the Service proxy

objHelloWorldService = new localhost.Service1();

// Set up the URI

objHelloWorldService.Url = objHelloWorldService.FormatCustomUri("msmq://"
                           + strMyQ);

…

// Make call

string[] arrRes = objHelloWorldService.HelloWorldArr("Simon");

To be clear, the FormatCustomUri() call acts as a pass-through for HTTP and MQ specifications – only affecting MSMQ queue name formats.

MSMQ Processing

This section essentially describes the guts of the GetResponse() method.

First, we need to place the Soap message on the MSMQ queue described in the request Uri. As just explained, for MSMQ the Uri format of a queue name looks a little awkward and needs to be re- formatted to be MSMQ-friendly:

// Create a message

System.Messaging.Message objMsg = new System.Messaging.Message();

// Simply hook in the stream as the Body of the message

objMsg.BodyStream = m_RequestStream;

// Here we need to interpret the MSMQ Uri a little

string strQueueName = this.m_RequestUri.LocalPath.Replace("/",  "\\");
if (strQueueName.ToUpper().IndexOf("PRIVATE$") > = 0)
strQueueName = "." + strQueueName;

// Open the Queue for writing too
objQueue = GetQ(strQueueName);

// Send the message under a single MSMQ internal transaction
objQueue.Send(objMsg, MessageQueueTransactionType.Single);

// Free resources on main Queue
objQueue.Close();

Making use of the MSMQ "Single" transaction type ensures a delivery of one (and only one) message onto the queue. For more sophisticated heterogeneous transactions support (e.g. covering the audit of the sending of this message to, say, a database) we would need to look at integrating this core work with .NET Enterprise Services to make use of COM+ style transactions. Note that, since we established we would use transport-specific correlation, the MSMQ message ID of the outbound message is captured and remembered at this point. We will rely on the Back End process to provide this ID as a "Correlation ID" (both MSMQ and MQ support this feature) to allow us to look for a matching response more easily:

// Get the ID of the outbound message to use as the Correlation ID

// to look for on any inbound messages

string strCorrelationID = objMsg.Id;

Once the message is placed on the queue, we then wait for a response to the message on a queue defined by the response Uri with a Correlation ID match. We wait for the period defined by the caller who can influence the timeout when creating the proxy:

// Open the response MSMQ Queue and wait for a message

try
{
   …

   // Wait for a response with the Correct Correlation ID

   TimeSpan tWaitResp = new TimeSpan(0, 0, m_intTimeout / 1000);
   objQueue = GetQ(strQueueNameResp);        
   msg = objQueue.ReceiveByCorrelationId(strCorrelationID, tWaitResp);
}
catch (Exception e)
{
   …            
}

One of two things will happen as a result of waiting on the response Queue:

1. An error occurs (error accessing the outbound / inbound queue, timeout waiting for the message response from the Back End etc. In this case, the error received from the managed MSMQ provider is thrown direct to the caller. Referring to the general WebException status code table presented in the section named "Handling Errors", here we list the specific mapping of major MSMQ failure scenarios to corresponding WebException status codes:

WebException Status	Triggering event
ConnectFailure	Failed to open the request queue in MSMQWebRequest::GetQ()
NameResolutionFailure	Failed to locate the request or response queue in MSMQWebRequest::GetQ()
SendFailure	Failed to put the message on the request queue in MSMQWebRequest::GetResponse()
Timeout	Failed to receive the corresponding message from the response queue before the timeout period was reached in MSMQWebRequest::GetResponse()
ReceiveFailure	Failed to receive the corresponding message from the response queue for a non-timeout reason in MSMQWebRequest::GetResponse()
ProtocolError	Any other failure within MSMQWebRequest / MSMQWebResponse. In this case, an MSMQWebResponse is created which details the exact failure.

Note, in the latter case, an MSMQWebResponse is created and passed as a parameter to the constructor of the WebException class – this also wraps up the actual exception which will become the 'InnerException' of the WebException :

catch( … )
{
   // Have we already processed this exception in some way?

   if ((e is System.Net.WebException) == false)
   {
      // No - Create an MSMQWebResponse - this is a protocol specific error

      objMSMQWebResponse = new MSMQWebResponse(
            (into)QueueTransportErrors.UnexpectedFailure, 
            e.Message, 
            e.StackTrace);

       // And throw the exception

       throw new WebException(
            "MSMQ Pluggable Protocol failure.", 
            e,
            WebExceptionStatus.ProtocolError,
            objMSMQWebResponse);
   }
}

2. A Soap message is received as a stream and the MSMQWebResponse instance is created and primed directly with the returned stream, in preparation for the Web Services client-side infrastructure to process it:

// Create a return, and stream results into it...

objMSMQWebResponse = new MSMQWebResponse();
objMSMQWebResponse.SetDownloadStream(msg.BodyStream);            
…
return objMSMQWebResponse;

Websphere MQ-specific implementation

As with the MSMQ support, all the work is done in the overridden GetResponse() method.

Figure 7: IBM ActiveX classes for MQ

Accessing MQ functions

In order to make use of MQ, the only credible option available to us in Websphere MQ 5.3 is the COM wrapper that is provided with the MQ Client (MQ is optional. The demo application can run on any combination of HTTP, MSMQ and MQ transports. This is configurable within the demo). The COM wrapper must be included as a COM interop reference in our project. The additional notes section describes an alternative that improves the overall throughput when using MQ, for now the COM wrapper will suffice.

Note that the resulting interop assembly, called Interop.MQAX200.dll , is not signed when included this way and therefore any assemblies using it will not be capable of being placed in the GAC unless a signed version of the Interop assembly can be created.

This signing can be achieved by undertaking the following steps:

• locate the COM DLL mqax200.dll, which can be found for a default installation within the C:\Program Files\IBM\WebSphere MQ\bin folder
• if a strong name is not already used within the development team, using sn –k <KEYFILENAME>to generate a file containing a strong name key pair
• run the tlbimp utility on the COM DLL file, specifying the appropriate keyfile as an option /keyfile<KEYFILENAME> and supplying a name for the output "signed" resultant assembly using /out<OUTFILENAME>

Once this has been done, and the resultant assembly included as a reference to the project, the host project could then be signed and deployed to the GAC as required.

Queue Name Access and Format

The model for accessing an MQ queue is different to that for MSMQ. Principally, whereas MSMQ object hierarchy is fairly "flat" in nature, MQ employs a hierarchical structure that involves a 3-tier access consisting of:

• Acquiring an MQ Session
• Connecting to a named MQ Queue Manager via the Session
• Requesting access to a named MQ Queue via the Queue Manager

Further to this, the name of a queue is built from two parts:

<queue manager name>/<queue name> e.g. QM_yoda/queue32

A major thing to note here is that, unlike MSMQ, entity names are case sensitive. Indeed there are several 'gotchas' when working with a combination of the Uri class (which if you remember lower- cases host names). When using the framework, a tell tale sign of a naming issue is a WebException with a status code of NameResolutionFailure , indicating either the Queue Manager or Queue could not be resolved (see the detail below). Some examples of queue access follow.

If connecting to a queue called "fred" the default Queue Manager, one would use a Uri notation like:

objHelloWorldService.Url = "mq://fred"

If connecting to a queue called "bill" on a Queue Manager called "QM_john" , the Uri for this would be:

objHelloWorldService.Url = "mq://QM_john/bill"

Hopefully this gives a flavor of MQ queue nomenclature. In order to help unbundled a Uri into a Queue Manager / Queue name combination, a utility method called ResolveManagerAndQueueName() is provided, and the transport code uses this like so:

// What is the Queue Manager and Queue Name we need to direct the message at?

ResolveManagerAndQueueName(
    this.m_RequestUri,
    out strQueueManagerName,
    out strQueueName);

MQ Processing

Apart from the obvious semantic differences described in the last section, the main differences between this implementation and the MSMQ version are in the syntax when sending messages and waiting on a response Queue. Also, whereas MSMQ provides good support for direct use of streams when sending and receiving messages, there is no such support for MQ (at least when using the ActiveX wrapper) and we must rely predominantly on sending and receiving byte arrays instead.

So, since MQ does not support the automatic use of streams as the body of a message, we need to convert the stream to an array of bytes, which can then be used to prime the outbound message:

// Access the Soap serialised stream as an array of bytes

byte[] bytBody = new Byte[m_RequestStream.Length];
m_RequestStream.Read(bytBody, 0, bytBody.Length);

Given we have the Soap body, and have resolved the request and response Queue Manager / Queue Name combinations (shown above), the first thing to do is to get an MQ Session. This is required before any queuing work may be attempted:

// Access a Session

objMQSession = new MQAX200.MQSessionClass();

Next, we attempt a connect to the Queue Manager, and ask it for a Queue object given the name:

// Get the named Queue Manager

objQueueManager = GetQManager(objMQSession, strQueueManagerName, true);

// Get the named Queue from the Manager

objQueue = GetQ(objQueueManager, strQueueName, true);

Once this has succeeded, we construct a message setting up message options, and write the body of the message:

// Set up the message to send

objMsg = (MQAX200.MQMessage)objMQSession.AccessMessage();
objPutMsgOpts =
          (MQAX200.MQPutMessageOptions)objMQSession.AccessPutMessageOptions();
// We need to send a byte array to cover off binary attachments etc....

objMsg.Write(bytBody);

Putting the message on the queue uses the (default) message options we just set up :

// Finally, put the message to the queue

objQueue.Put(objMsg, objPutMsgOpts);

[NOTE: If conversing with MQ running on AS/400 which operates in EBCDIC, the message would explicitly need to indicate it was in ASCII format prior to sending - objMsg.Format = "MQSTR " ;]

Note that, since we established we would use transport-specific correlation, the MSMQ message ID of the outbound message is captured and remembered at this point. We will rely on the Back End process to provide this ID as a "Correlation ID" (both MSMQ and MQ support this feature) to allow us to look for a matching response more easily:

// Get the ID of the outbound message to use as the Correlation ID

// to look for on any inbound messages

string strCorrelationID = msg.MessageId;

Once the message is placed on the queue, we then wait for a response to the message on a queue defined by the response Uri with a Correlation ID match. We wait for the period defined by the caller who can influence the timeout when creating the proxy:

try
{
    // Open the response MQ Queue and wait for a message

    objQueueResponse = GetQ(objQueueManager, strQueueNameResp, false);
    objMsgResp = (MQAX200.MQMessage)objMQSession.AccessMessage();
    objMsgResp.CorrelationId = strCorrelationID;
    objGetMsgOpts = (MQAX200.MQGetMessageOptions)
    objMQSession.AccessGetMessageOptions();
    objGetMsgOpts.Options = (int)MQAX200.MQ.MQGMO_SYNCPOINT +
                            (int)MQAX200.MQ.MQGMO_WAIT;
    objGetMsgOpts.WaitInterval = this.Timeout;
    objQueueResponse.Get(
        objMsgResp,
        objGetMsgOpts,
        System.Reflection.Missing.Value);

    // Use the appropriate reader

    strMsgRecv = objMsgResp.MessageData.ToString();
}
catch (Exception e)
{
     ...            
}

One of two things will happen as a result of waiting on the response Queue:

1. An error occurs (error accessing the outbound / inbound queue, timeout waiting for the message response from the Back End etc. In this case, the error received from the managed MSMQ provider is thrown direct to the caller. Referring to the general WebException status code table presented in the section named "Handling Errors", here we list the specific mapping of major MSMQ failure scenarios to WebException status codes:

WebException Status	Triggering event
ConnectFailure	Failed to open the request queue manager in MQWebRequest::GetQManager() or failed to open the request queue in MQWebRequest::GetQ()
NameResolutionFailure	Failed to locate the request or response queue in MQWebRequest::GetQ()
SendFailure	Failed to put the message on the request queue in MQWebRequest::GetResponse()
Timeout	Failed to receive the corresponding message from the response queue before the timeout period was reached in MQWebRequest::GetResponse()
ReceiveFailure	Failed to receive the corresponding message from the response queue for a non-timeout reason in MQWebRequest::GetResponse()
ProtocolError	Any other failure within MQWebRequest / MQWebResponse. In this case, an MQWebResponse is created which details the exact failure.

2. A Soap message is received as a byte array, converted to a stream, and the MQWebResponse instance is created and primed directly with the converted stream, in preparation for the Web Services client-side infrastructure to process it:

// Convert the result into a byte array

byte[] buf = new System.Text.UTF8Encoding().GetBytes(strMsgRecv);
MemoryStream stResponse = new MemoryStream();
stResponse.Write(buf, 0, buf.Length);

// Create a return, and stream results into it...

objMQWebResponse = new MQWebResponse();
objMQWebResponse.SetDownloadStream(stResponse);            
…
return objMQWebResponse;

Tapping into the serialized Soap stream

As explained above, when implementing a WebRequest –derived class, we need to provide a stream for the Soap serialization to make use of. Then before we can send this Soap "request stream" to any network resource via one of our new transports, we must first get access to this serialized stream.

The WebRequest–derived class we introduced above from which we shall derive our MSMQ and MQ variants contains a data member called m_RequestStream of base type Stream, specifically to carry the outbound Soap stream. The infrastructure will call our overridden method GetRequestStream() at the appropriate time to create and provide this stream during the outbound delivery phase of a Web Service call. Our initial attempt will use a member variable of type MemoryStream .

However, it appears (from a degree of single stepping through the code flow) that there is no direct access in WebRequest derivations to the request stream until our GetResponse() override is called (at which point the request stream has already been disposed of – see the Length property of m_RequestStream as an illustration of this):

Figure 8: Access to the Soap request stream

Observation of the UML Sequence diagram presented earlier in the Case Study will confirm that the infrastructure has already closed the stream at the point we need it. We need to come up with an alternative approach to gain access to the stream before it is disposed of. It turns out that the Microsoft infrastructure has to Close() the stream you provide so that the data is pushed out onto the wire - this is the standard programming model for WebRequests .

The following two options are available to us:

• Tapping the stream just after serialization by creating and using a SoapExtension and an associated attribute. We could hook the SoapMessageStage.AfterSerialize processing stage to store the serialized Soap stream. This will add extra classes to the mix, cause more rework of the (re-)generated proxy and may cause us issues as we develop our framework onwards to include WSE.
• Provide a custom stream, and override Close() to modify the streams behavior - effectively delaying the actual close of the stream until we've had chance to access it. This is the preferred approach as it fits more neatly with the framework.

Figure 9: MySoapStream

So, we provide a class called MySoapStream, suppressing the actual closure of the stream and providing an alternative called InternalClose() , which actually closes the underlying Stream.

public override void Close() 
{
    // DON'T CLOSE! BUT REWIND!

    m_Stream.Position = 0;
}

internal void InternalClose() 
{
    m_Stream.Close();
}

We also change the implementation of the GetRequestStream() method to use this new type of stream:

public override Stream GetRequestStream()
{    
   if (m_RequestStream == null)
      m_RequestStream = new MySoapStream(new MemoryStream(), true, true, true);
   else
      throw new InvalidOperationException("Request stream already retrieved.");

   return m_RequestStream;        
}

Hence, revisiting the MSMQ WebRequest handler as an example, the GetResponse() override alters slightly (MSMQ version shown but both applicable) looks like:

public override WebResponse GetResponse() 
{
    …

    // Create a message using the Stream

    objMsg.BodyStream = m_RequestStream;
    objMsg.Recoverable = true;

    …

    // Open the Queue for writing

    objQueue = GetQ(strQueueName);

    // Send the message under a single MSMQ internal transaction

    objQueue.Send(objMsg, MessageQueueTransactionType.Single);

    …

    // And close Soap Stream

    m_RequestStream.InternalClose();

    …
    
    // Now wait on a response


    …
}

In this case, the stream may be used directly as the MSMQ message body using the BodyStream property of the latter. While there is no equivalent direct-stream linkage in MQ, only one more step is required to read the stream into an array of bytes, which can then be written directly to the MQ message.

Once the message has been sent, we can close the stream ourselves using our InternalClose() method.

Hooking the new protocols into the generated proxy classes

Now we have two new transport protocol handlers, let's make them useable from the proxy class. The resultant proxy now looks a little different to its original "freshly-generated" form:

public class Service1 : MySoapClientProtocol
{
    ...        
        
    // The individual methods of the API are unchanged from their generated form

    [System.Web.Services.Protocols.SoapDocumentMethodAttribute(...)]
    public string[] HelloWorldArr(string name)
    {
           object[] results = this.Invoke("HelloWorldArr", new object[] {name});
        return ((string[])(results[0]));
    }
    
    // The individual methods of the API are unchanged from their generated form


    ...
        
    // Overridden methods for directing Soap messages

    protected override WebRequest GetWebRequest(Uri uri)
    {
           // Delegate to a common generator for new protocols

        return ProxyCommon.GetWebRequest(uri, this);
    }

    protected override WebResponse GetWebResponse(WebRequest webReq)
    {
        // Delegate to a common generator for new protocols

        return ProxyCommon.GetWebResponse(webReq);
    }
}

The proxy now derives from MySoapClientProtocol, including overrides for GetWebRequest() and GetWebResponse() calls. This constitutes the only changes required. Note that these changes will be lost and will need reapplying if the web service proxy is subsequently regenerated for any reason.

Using the new proxy from a client

Now the hard work is out of the way, clients of the proxy can make use of it in the following way:

localhost.Service1 objHelloWorldService = null;
string strRequestUrl = "msmq://.\\private$\\myreq";
string strResponseUrl = "msmq://.\\private$\\myresp";

try
{
    // Create the Service proxy

    objHelloWorldService = new localhost.Service1();

    // Set up the request Queue via the Uri

    objHelloWorldService.Url
                  = objHelloWorldService.FormatCustomUri(strRequestUrl);

    // Set up an alternative response Queue other than "<QUEUENAME>_resp"

    objHelloWorldService.ResponseUrl
                  = objHelloWorldService.FormatCustomUri(strResponseUrl);

    // And the timeout

    objHelloWorldService.Timeout = vintTimeoutInSecs * 1000;

    // Run the method and print out the return

    Console.WriteLine(objHelloWorldService.HelloWorld("Simon"));
}
catch(Exception e)
{
    ...
}

Support for asynchronous message delivery

There are numerous articles describing how to take advantage of asynchronous support in Web Services (see foot of the article for details), so we'll not dwell on this subject too much. Suffice to say there are three basic options when implementing asynchronous calling:

• Polling for Completion
• Using WaitHandles
• Using Callbacks

Whichever method is chosen they all rely on access to a token returned to them once an asynchronous call has begun. The generated proxy classes have synchronous and asynchronous versions of each method call – the asynchronous ones include a pair of methods of the form:

public System.IAsyncResult BeginHelloWorldArr(
    string name, 
    System.AsyncCallback callback,
    object asyncState)
{
    return this.BeginInvoke("HelloWorldArr", new object[] {name}, callback,
                           asyncState);
}

public string[] EndHelloWorldArr(System.IAsyncResult asyncResult)
{
    object[] results = this.EndInvoke(asyncResult);
    return ((string[])(results[0]));
}

The BeginXXX method call will return immediately with a token of type IAsyncResult , which may be used to poll or wait on request completion.

While not particularly complicated, asynchronous request support is nicely encapsulated in an assembly like AsyncUIHelper as provided in the MSDN article [3]. Use of this library means that our queue transports don't need to implement the asynchronous features for themselves (well for now, anyway).

The key aspect of the support is a class called Asynchronizer :

private Asynchronizer m_ssiReturnImage = null;

The caller of a method would include the use of the Asynchronizer as follows, in our example first hooking up the delegate-based callback, and then invoking the standard ReturnImage() call:

// Create the Service proxy

objHelloWorldService = new localhost.Service1();

// Set up the URI

objHelloWorldService.Url = objHelloWorldService.FormatCustomUri(vstrURI);

// And the timeout

objHelloWorldService.Timeout = vintTimeoutInSecs * 1000;

// Create a new Async token

m_ssiReturnImage = new Asynchronizer(
                             new AsyncCallback(this.ReturnImageCallback),
   objHelloWorldService);

// Begin the method call

IAsyncResult ar = m_ssiReturnImage.BeginInvoke(
                new ReturnImageEventHandler(objHelloWorldService.ReturnImage),
                null);

Any asynchronous responses would be directed to an event handler based on a delegate, which is defined with the return type of the standard synchronous version of the method call we are "asynchronizing":

protected delegate byte[] ReturnImageEventHandler();            

In this case, the responses to the call (namely the image data in this case) are accessible through a data member in the asynchronous token:

protected void ReturnImageCallback(IAsyncResult ar)
{
    localhost.Service1 objHelloWorldService
                                        = (localhost.Service1)ar.AsyncState;

    ...

    AsynchronizerResult asr = (AsynchronizerResult) ar;
    byte[] bytImage = (byte[])asr.SynchronizeInvoke.EndInvoke(ar);

    ...
}

Faking it – the "Back-end Service"

So far, we've majored on what it takes to get the client side talking in the right way to the request and response queues. This is pretty meaningless unless we have some form of service to pick off messages and process them. In real life solutions, one would expect the use of Web Services to be targeted at heterogeneous environments where the client and server run on different platform and Web Services is a nice way to mesh the two worlds.

However, due to a desire to keep the skill set required to keep up with the article to a manageable amount, we'll use .NET application at the back to consume the messages and produce (dummy) responses in order to allow the clients message reception to be tested. It should be noted that the service exists purely to act as a counterpoint to the client framework we have developed, and so very little work has gone into making it robust or scalable – but there's enough there to demonstrate the client capability.

In the non-WSE arena we are currently describing, the Service really can be quite basic. Its job is to perform the following steps:

• Read a configuration file containing the MSMQ / MQ Queues to wait on
• Loop the queues awaiting messages for a short period of time
• For any messages positively identified, a response Soap stream is manually generated and placed on the response queue for picking up by the client

The logic is contained within one class, BEService.

This first reads the supplied app.config file for all application level settings (that is those within the <appsettings> node):

// Read configuration settings

NameValueCollection colSettings = ConfigurationSettings.AppSettings;

Each queue found in the configuration file is added to the relevant pot:

string[] arrVals = colSettings.GetValues(strKeyName);

if (arrVals[0].ToLower().IndexOf("msmq://") == 0)
    arrMSMQQueuesToMonitor[intNumMSMQQueues++]
                    = arrVals[0].Replace("msmq://", string.Empty);
else
if (arrVals[0].ToLower().IndexOf("mq://") == 0)
    arrMQQueuesToMonitor[intNumMQQueues++] = arrVals[0].Replace("mq://", 
                                                          string.Empty); 

The MSMQ and MQ queues are distilled from the collection, as is a polling delay (the back end service does not currently employ a thread pool). The following simple loop is then entered:

// Wait on some queues - some MSMQ, and some MQ...

for (;;)
{
    // Do MSMQ queues first

    foreach (string strName in arrMSMQQueuesToMonitor)
    {
        WaitOnMSMQQ(strName, intPollDelay);
    }

    // Now MQ queues

    foreach (string strName in arrMQQueuesToMonitor)
    {
        WaitOnMQQ(strName, intPollDelay);
    }
}

WaitOnMSMQ implementation

This method is straightforward. First we wrap all work in a try-catch block in order to trap all exceptions and filter out "healthy" ones such as a timeout waiting on request message:

try
{
    … // Do the work here

}
catch(Exception e)
{
    // Ignore timeouts as this simply means no queue message was there

    if (e.Message.IndexOf("Timeout for the requested operation has expired") < 0)
        TSLog("Exception caught in WSAltRouteBEFake[MSMQ]: " + e.Message + 
                                                    e.StackTrace);
}

What we'll do first is wait a while:

// We'll need to wait a little time

TimeSpan tWaitResp = new TimeSpan(0, 0, 0, 0, vintPollTimerDelay);
  
// Get hold of the request queue

MessageQueue objQueue = GetQ(vstrQueueName);        
  
// Wait for a message

Message objRequestMsg = objQueue.Receive(
    tWaitResp, 
    MessageQueueTransactionType.Single);
  
// Close the Queue

objQueue.Close();

Assuming we got a message, i.e. no exception has been thrown, we open up the message and get the data content into a byte array, while picking off some salient attributes such as a potential Correlation ID and response queue name:

// Get the message content direct from the BodyStream

string strResp = string.Empty;
byte[] bufIn = new Byte[objRequestMsg.BodyStream.Length];
objRequestMsg.BodyStream.Position = 0;
objRequestMsg.BodyStream.Read(bufIn, 0, (int)msg.BodyStream.Length);
string strCorrelationId = objRequestMsg.Id;

// Close stream

objRequestMsg.BodyStream.Close();

// Get the response queue - we shall assume a default but

// hope the client has identified a queue for us to call back on

string strResponseQueue = vstrQueueName + "_resp";
if (objResponseMsg.ResponseQueue != null)
    strResponseQueue = objRequestMsg.ResponseQueue.QueueName;

We then check to see if this is a known message and if so, build a response message for it. There is very little intelligence in the matching method or generation of the response message – it exists purely to serve the client response code.

// Fake out a response to the message strResp = BuildFakeResponseFromBytes("MSMQ" , bufIn, strResponseQueue);

If we identified a request and were able to generate a response for it, then send it back to the client via the response queue using the Correlation ID:

// If we have a valid response...use it

if (strResp.Length > 0)
{
    // Get the response queue

    objQueue = GetQ(strResponseQueue);

    // Send a message back

    MemoryStream stResp = new MemoryStream();
    stResp.Write(new UTF8Encoding().GetBytes(strResp), 0, strResp.Length);
    Message objMsg = new Message();
    objMsg.BodyStream = stResp;
    objMsg.Recoverable = true;
    objMsg.CorrelationId = strCorrelationId;
    
    // Send the message under a single MSMQ internal transaction

    objQueue.Send(objMsg, MessageQueueTransactionType.Single);

    // Free resources on response Queue

    objQueue.Close();
}

Here we are geared up to use the BodyStream property of the MSMQ message once again, so we write the contents of our response string into the stream. We send as a single MSMQ transaction once again, and then tidy the response queue.

That's really all that is required to effect basic end-end Web Service processing over the medium of queues. We could stop at this point or exploit a very interesting new technology that will help us secure our messages independent of the transport over which we are running them.

How Web Services Enhancements (WSE) alters the picture for Client and Server

The WSE is Microsoft's initial implementation of some of the first 2nd generation GXA Web Services specifications - WS-Security, WS-Attachments (via DIME) and WS-Routing / WS-Referral. It was released in Dec 2002, after 4 months in beta test. The whole standardization push continues the initiative to improve the capability for interaction / interoperability within the web service stack and implementations of these particular standards will really form the baseline for entry-level trading partner applications based around the Internet.

While it is anticipated that Soap over a queued transport will find use within the trusted networks of a single organization, this does not negate the possibility that the data will require protection in transit between enterprise applications (e.g. sensitive payroll data).

So, the main aspect of the WSE that is interesting to us in this article stems from the implementation of the WS-Security specification; namely the ability to:

• control the "valid lifetime" of a Soap message,
• generate a "username token" with an associated password which identifies a user for verification,
• sign a Soap payload with that token to allow for tamper checking,
• encrypt the payload of the Soap body using a symmetric key (known as a "shared secret" since the Back End Service will have access to the same key),
• secure the Soap requests and responses to make this an end-end-end solution.

It should be noted that this functionality represents a subset of what the WSE is capable of providing. Interesting features we are not using here include the use of binary security tokens (which could be capable of housing a Kerberos ticket for example) and asymmetric encryption using X509 certificates.

In addition, as we are using WSE and WSE implements the WS-Attachments specification, we can look at extending our demonstration application to explore the ability to add binary attachments via DIME support. It will become evident that the DIME support does not follow the standard Soap processing model – expect to see it's implementation alter over the next 12 months.

WSE overview

The WSE is a data processing engine, which essentially applies (under instruction) a series of transformations to outbound Soap messages once they have been serialized by the .NET Web Services infrastructure, and conversely applies further transformations on incoming Soap messages prior to their de-serialization into objects / parameters. These transformations involve the inclusion of additional Soap headers and in some cases (like encryption) alterations to the body of a Soap message as well.

The API provided by the WSE is relatively low level which makes it great for getting in amongst and influencing the operation of both client and server side. A great article covering the inner workings of the WSE can be found in [4]. We'll draw out the salient points in the sections that follow to give us some context for the changes we shall make to our client and server.

The physical manifestation of the WSE 1.0 is a single GAC-installed assembly called Microsoft.Web.Services.dll, which implements the majority of the three specifications named above.

The WSE Filter Pipeline Architecture

The single most major change between the beta version of the software (called the WSDK, released Aug 2002) and the WSE 1.0 is that the functionality effecting the transformations on Soap messages has been repackaged into a series of filters. These filters are one of two varieties, Output filters concentrate on manipulation of an outbound Soap message, whereas Input filters operate on incoming Soap messages.

Figure 10: Filter Pipeline Architecture

Input Filter	Output Filter	Purpose
TraceInputFilter	TraceOutputFilter	Write messages to log files to help with debugging
SecurityInputFilter	SecurityOutputFilter	Authentication, signature and encryption support
TimestampInputFilter	TimestampOutputFilter	Timestamp support
ReferralInputFilter	ReferralOutputFilter	Dynamic updates to routing paths
RoutingInputFilter	RoutingOutputFilter	Message routing

Each set of filters is organized into either an input filter or output filter pipeline, which out-of-the-box runs the filters in the following order (for Client-to-Server flow only):

Figure 11: Filter Pipeline Architecture in more detail (Client -> Server flow)

Amongst other things this ensures that the encryption and signing (Security) are applied to a Soap payload that all other modifications have been completed on and that securing happens just prior to sending of a request, whereas decryption and signature verification happen straight after reception of a request and before any other data sensitive operations can take place.

The main thing to remember here is that each pipeline operates on a Soap envelope to produce different content within that envelope.

The pipeline concept within the WSE is a very powerful one, as pipelines can be manipulated programmatically to remove or reorder stages, and new custom pipeline stages may be introduced based on the WSE-provided base classes SoapOutputFilter and / or SoapInputFilter.

The integration of the WSE pipeline execution of input and output filters with the standard Soap serialization and delivery mechanisms needs to be managed in two different ways:

• For ASP.NET Web Service clients, this is achieved by deriving the generated proxy class from a new proxy base class called Microsoft.Web.Services.WebServicesClientProtocol.
• For ASP.NET Web Services, a new server-side Soap extension has been provided in the form of Microsoft.Web.Services.WebServicesExtension.

The purpose of the new proxy base class and server side extension is to sit between the transport and Soap serialization / de- serialization modules, intercepting messages and building / analyzing Soap headers as appropriate. As a result it is highly possible that when using, say, signature verification within a Web Service, should this verification detect tampering and generate an appropriate Soap fault, not a single line of application business logic will ever have been called by the time the fault is generated. This nicely insulates application code from WSEs infrastructure code, which seems an attractive benefit.

The WSE SoapWebRequest and SoapWebResponse

The WSE itself employs the .NET Web Services pluggable protocol infrastructure we talked about earlier to provide two new communication classes called SoapWebRequest and SoapWebResponse. As with our MSMQ and MQ versions, these are derived from System.Net.WebRequest and System.Net.WebResponse respectively.

The SoapWebRequest class parses the request stream containing the standard serialized Soap message into an instance of another WSE-provided class called the SoapEnvelope, which is derived from System.Xml.XmlDocument . It then drives the standard output filter pipeline, which conditions the contents of the envelope as described in the section above.

Our interest within the framework will revolve around integrating our queue-based classes with these new Soap web request / response classes, and in particular picking up the resultant Soap stream for use by our protocol handlers.

However, in the case of our Back End Service, SoapWebRequest and SoapWebResponse are irrelevant to us as we are operating outside the client or ASP.NET server infrastructure and must "hand-craft" WSE processing ourselves.

The WSE SoapContext for communication

We've talked about the operation of the WSE but so far not about how this processing is influenced by any application code we may write. The SoapContext class is the vehicle by which application code and WSE formatting code indirectly communicate. This happens in one of two ways, depending on the point in the process:

• When building Soap headers during outbound processing, the application first primes attributes of the SoapContext, which become the instructions for the filters to operate against i.e. in this case it is the SoapContext attributes that shape the content of the outgoing Soap message.
• When analyzing headers during inbound processing, the WSE primes a SoapContext with the "results" of the inbound filtering, prior to handing control to the server side application i.e. in this case it is the content of the incoming Soap message that shapes the setup of the SoapContext.

The class contains several attributes of interest:

Figure 12: WSE Soap Context

The Envelope contains XML marking the "wor