Duplex Web Services

• Download Duplex_WS_Sample - 365.35 KB

Introduction

While doing a code review for a friend of mine I came across an interesting technique for getting a web service to “push” an event or message back to the client when something occurs. We all know that the HTTP is a request/response protocol. The client makes a request, a socket is opened, the server allocates a thread for the request, the request is processed, the results are returned, and the socket is closed. In the code that I saw, and reworked/improved for this example, web services are used to communicate events or messages between the request and the response.

Think of a chat room application entirely handled by web services. The various “events” or messages that would have to be passed between users include “user has logged in”, “user has logged out”, and the actual text messages sent.

Background

Normally this would be handled by writing some TCP socket based application that maintains sockets between the server and clients. The advantage to using a web service is that you don’t have to manage the sockets or the thread pool yourself. You also don’t have to worry about configuring every firewall on every client machine to open a port for you; HTTP runs on port 80 and is not blocked. This is known as HTTP Tunneling. You’re clients can communicate with each other through HTTP without firewalls and anti-viral applications treating them like some Trojan horse.

The major disadvantage of this technique, and we’ll see this further in the article, is that it requires holding on to a thread pool thread on the server. Although the tuning and performance aspects of this problem are beyond the scope of this article, suffice it to say that this technique may not scale very well. Still if the application is served by a dedicated server with a predetermined amount clients, and the thread pool configuration on the server is properly done, this solution can be compelling.

Besides practical usage this article showcases some of the challenges and techniques in writing multi-threading applications. Some of the .NET threading classes that I used in the example can be replaced with others. I believe the ones of chosen best fit the example for both functional and performance reasons, but feel free to experiment with various constructs. At the end of this article I’ve included links to various sites that deal with multi-threading.

Using the code

OK, so how do I get a web service to notify the client when something occurs on another client or on the server itself? As I mentioned earlier web services use a request/response protocol. How do we get it to push back a message? The answer is we don’t. We simulate a push back by stopping the request on the server before it returns to the client. This process simulates “listening”. When an event occurs, such as another client sending a message through another web method, our listening web method is released returning an “event” object that represents the type of event/message that occurred and containing any data that pertains to it.

Initial State – Everyone’s listening for an event from someone

An event occurs – Listen request is released and a response occurs

This is possible since a web service can be called asynchronously from a form without locking up the form’s main thread. A callback event handler is used to process the incoming event from the server.

Calling a web service asynchronously is made easy in .NET 2.0 since every web method you create generates two methods and an event on the client proxy.

For instance, if you were to create a web method called HelloWorld you’ll see that the client proxy for the web service has a HelloWorld method, a HelloWorldAsynch method, and a HelloWorldCompleted event. The first method, as you probably know, relays the execution request to the server and blocks the calling client thread until a response occurs. It is said to be synchronous. HelloWorldAsynch acts as at fire-and-forget method. You call it from your client code but it never blocks the thread. You’re form is free to continue processing without having to wait for the web method to return a value or a response. When the response arrives it fires off the HelloWorldCompleted event which can be handled by your client code.

The Listen method described above works in this manner. When the form loads it call for a Listen using the asynchronous version of the method. When an event occurs on the server that needs to be reported to the client the ListenCompleted event fires off. This event is handled with a method that processes the “event”. It then calls listen again to be ready for the next event.

The ListenCompleted event comes complete with a ListenCompletedEventArgs object, also generated by the IDE. It contains an object of the same type as returned by the Listen web method.

Since the Listen web method returns an “EventObject” array defined by me, the event argument’s Result property will contain an object of that type. EventObject is the ancestor of three event classes that my client application is interested in listening to and processing. They are as follows: LoginEvent, LoggedOutEvent, and MessageEvent.

Since my sample application is a simple chat room, I want to display all users that are currently logged in as well and being told in real time when a new user has logged in, when a logged in user has logged out and when a message has been sent by someone in the chat room.

I made the Listen web method return an array of events because various events can occur while we were not listening. IE while we were processing an event on the client, and before calling the ListenAsync web method, various other events can occur on the server that we may have missed (I’ll get into the caching mechanism of these events further down the line). When we listen again we want to receive all the missed event in one shot. Hence an array of EventObjects.

The code for the ListenCompleted event handler looks like this:

/// <summary>
/// When Listen ws request returns then an event was registered and needs to be processed.
/// Process the event and call the Listen webmethod again to listen for further events
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void _ws_ListenCompleted(object sender, ListenCompletedEventArgs e) {

    bool getAllUsers = false;

    ChatClient.TwoWayWS.EventObject[] events = e.Result;
    
    System.Diagnostics.Trace.WriteLine("_ws_ListenCompleted - Made it in - _formClosing=" + _formClosing);
    System.Diagnostics.Trace.WriteLine("_ws_ListenCompleted - " + events.Length);

    if (_formClosing)
        return;

    foreach (EventObject eventObj in events) {

        System.Diagnostics.Trace.WriteLine("_ws_ListenCompleted - event Object type " + eventObj.ToString());

        _myLastEventID = eventObj.EventID;
        _myLastEventsListResetId = eventObj.EventsListResetID;

        switch (eventObj.ToString()) {
            case "ChatClient.TwoWayWS.LoginEvent":
                // If list of users on UI is empty and we got a loggin event then call another ws to 
                // get a list of all the logged in users. This condition would be true first we log in
                // and we want a list of the current users. Otherwise just add the new user(someone else logging in)
                // to the bottom of our list.
                if (lvLoggedInUsers.Items.Count == 0)
                    getAllUsers = true;
                else {
                    LoginEvent le = eventObj as LoginEvent;
                    // Login user may exist in the list due to bulk load of all logged on users. And the cached event
                    // of the login can be now processed as well.
                    try {
                        if (!lvLoggedInUsers.Items.ContainsKey(le.UserName))
                            lvLoggedInUsers.Items.Add(le.UserName, le.UserName, 0);
                    }
                    catch { }
                }
                break;
            case "ChatClient.TwoWayWS.LoggedOutEvent":
                LoggedOutEvent lo = eventObj as LoggedOutEvent;
                lvLoggedInUsers.Items.RemoveByKey(lo.UserName);
                break;
            case "ChatClient.TwoWayWS.MessageEvent":
                MessageEvent msg = eventObj as MessageEvent;
                string fromPart = "From " + msg.From + ":";
                // Add the line - Note SelectedText acts as an append text function.
                rtChatRoom.SelectionBackColor = Color.LightGray;
                rtChatRoom.SelectionColor = Color.Crimson;
                rtChatRoom.SelectedText = fromPart; // Append the text
                rtChatRoom.SelectionBackColor = Color.White;
                rtChatRoom.SelectionColor = Color.Black;
                rtChatRoom.SelectedText = " " + msg.Message; // Append the text
                rtChatRoom.SelectedText = "\n";
                break;
            default:
                // Non-event processing - increment the non-event count
                int nonEventCnt = int.Parse(lblNonEventCnt.Text);
                nonEventCnt++;
                lblNonEventCnt.Text = nonEventCnt.ToString();
                break;
        }

        // Get a list of all the currently logged in users - see comments above.
        if (getAllUsers) {
            LoginEvent[] allLoginEvents = _ws.GetAllUsers();

            foreach (LoginEvent le in allLoginEvents) {
                try {
                    lvLoggedInUsers.Items.Add(le.UserName, le.UserName, 0);
                }
                catch { }
            }

            getAllUsers = false;
        }
        
    }

    // Sleep the current UI thread to simulate network latency 
    Application.DoEvents();
    int milliSecdelay = string.IsNullOrEmpty(txtListenDelay.Text) ? 0 : int.Parse(txtListenDelay.Text);
    System.Threading.Thread.Sleep(milliSecdelay * 1000);
    txtListenDelay.Text = "0"; // reset to no delay.
    
    // re-establish a listen request.
    System.Diagnostics.Trace.WriteLine("_ws_ListenCompleted - Calling _ws.ListenAsync() for " + txtUsername.Text);
    _ws.ListenAsync(_myLastEventID, _myLastEventsListResetId);

}

Server Pushback and Thread Management

Let’s examine how the web service acts as a server for the chat application by processing the above mentioned events and pushing them back to all listening clients.

Core Objective’s Mechanism

The primary mechanism in the Listen method is to use a WaitHandle of type AutoResetEvent to cause the Listen event to block, and to register that WaitHandle to global list.

.
.
.
    waithandle = new AutoResetEvent(false);
    Global.WaitingListenerList.Add(waithandle);
    Global.EventBeingDelivered = false;
}
// Block the response until an "event" occurs.
System.Diagnostics.Trace.WriteLine("Listen - Aquired waithandle and blocked IIS thread");
//****************************************************************************
// Wait for an event, but continue before the request times out.
listenTimedOut = !waithandle.WaitOne(Global.EventWaitTimeout, true);
 

The list will be traversed upon the arrival of an event and each WaitHandle will be released with a set method on the WaitHandle. So if we peek at a snippet of code from the SendObjectEvent method that occurs with each event we’ll see this:

 // Release the waiting response threads
 foreach (AutoResetEvent waitingListner in Global.WaitingListenerList) {
    waitingListner.Set();
 } 

SendObjectEvent then goes into its own wait state that’s released only when all the Listen threads have been released so that it can do some post-event-release housekeeping. This is achieved by the Listen method by decrementing a listener count right before returning the event object to the client. When the count is zero it signals the SendObjectEvent method through a global waithandle called AllListenersReleased.

That’s the concept in a nutshell. Of course no multi-threaded solution is ever that simple. But before I dig deeper into the issues and solution that cropped up while writing this sample application its important to understand the core objective’s mechanism:

1. Have the listen method block the thread its on so that it doesn’t return a response until some event occurs and data pertaining to that event is ready to be returned to the client.
2. Retain the blocking WaitHandle object for each client application running the Listen web method on a web server thread. The list of WaitHandles are retained in a Global list.
3. SendObjectEvent releases each listening thread, when an event occurs that needs to be pushed back to all the listening clients, by issuing a Set on each WaitHandle that we’ve accumulated in the global list. It then blocks/waits until all the Listeners have been released.
4. The Listen method signals the waiting SendObjectEvent that all listing threads have been released so that it can do some housekeeping such as un-registering the Listener WaitHandles (Client will re-register a WaitHandle on subsequent calls to Listen).

Digging Deeper

Ancillary Issues

The following issues and challenges must be addressed before the core objective is complete.

1. Synchronization - The Listen web method and the SendEventObject method, which all events call to release the Listeners and pass event specific data, must be synchronized and thread safe.
1. Can’t start to Listen while sending an event.
2. Can’t send an event while the Listen method registers its waithandle.
3. Can’t process more than one event at a time.
4. Don’t want to register two listener’s waithandles at the same time.
2. Event Caching - The web service must cache events until it’s certain that all clients received them. This becomes an issue when events are pushed back while one or more clients aren’t listening due to being busy processing a prior event.
1. If outstanding events exits for a client they should be returned immediately upon the subsequently Listen call.
2. The web service must have a reasonable way to determine if all events have been delivered to all clients. Once that has been determined, the events cache can be cleared. The list of waithandlers can be cleared at this time as well.
3. Request Timeouts - Request timeouts must be addressed. Before a timeout occurs we must release a dummy event back to the client. The client in turn recognizes that it received a “non-event” event and simply re-executes a Listen.

Synchronozation

There are various locks in the code to insure that multiple threads do not modify global variables simultaneously. I’ll focus on the locks that synchronize access between the Listen web method and the SendEventObject method that all the “application events” use to push back an event.

Conceptually speaking listening for, and sending, an event needs to be synchronized. I don’t want to send an event while a Listen method is trying to register its WaitHandle to the global list. This could cause a race condition were the listen method hasn’t registered itself for receiving events, but an event has already been released to all the listening threads. Since all registered listening threads are assumed to have received the event, the event cache can be cleared and a client can lose an event.

Furthermore I don’t want two events processed simultaneously either. Although I can push back more than one event to a client, I want to manage "one event/one release" to all listening clients at a time.

Below are the first few lines of the Listen method. A generic lock object (Global.LockObj) is used in a code block to protect various global variables that are being changed - that has to be done atomically. Within that code block I enter a loop that checks if an event is being delivered and blocks the current thread if it is.

[WebMethod]
[XmlInclude(typeof(EventObject))]
[XmlInclude(typeof(LoginEvent))]
[XmlInclude(typeof(LoggedOutEvent))]
[XmlInclude(typeof(MessageEvent))]
public EventObject[] Listen(int myLastEventID, int myLastEventsListResetId) {

    AutoResetEvent waithandle;
    bool listenTimedOut = false;
    EventObject[] returnedEvents;

    lock (Global.LockObj) {
        // Don't register a listener while event is being delivered
        while (Global.EventBeingDelivered)
            Monitor.Wait(Global.LockObj); //Release the lock and wait for a pulse to re-check message delivery state again

        Global.EventBeingDelivered = true;

        // Immediatly return any outstanding events that have accumulated while I wasn't listening.
        returnedEvents = EventUtils.PrepareEventsForSending(myLastEventID, myLastEventsListResetId);

        if(returnedEvents.Length > 0) {
            Global.EventBeingDelivered = false;
            return returnedEvents;
        }

        waithandle = new AutoResetEvent(false);
        Global.WaitingListenerList.Add(waithandle);
        Global.EventBeingDelivered = false;
    }  

I use a global state variable called EventBeingDelivered to determine if a client is in the processes of registering a Listen, or if an event is in the process of being sent to all the clients. Since I’m using a “blocking condition flag”, as mentioned in Joseph Albahar’s excellent e-book on Threading in C#, I’ve choose the Monitor class as the synchronization mechanism between the Listen and the SendEventObject methods.

This is essential since a Monitor.Pulse may have executed indicating that an application event has been processed successfully, but another thread running a SendEventObject may have received execution control prior to the Listen code executing. We therefore check the blocking condition flag for its state prior to continuing with the registration. If the flag is still true we loop into another wait.

The same code exists on the SendEventObject method to ensure that another thread is notsending out an event prior to processing the current thread’s event.

public class EventUtils {

    public static void SendEventObject(EventObject eo) {

        lock (Global.LockObj) {
            // Don't register a listener or process a new event/message while an event/message is being delivered
            while (Global.EventBeingDelivered)
                Monitor.Wait(Global.LockObj); //

            // Cause New Listeners and events to wait until message is delivered to all existing listeners
            Global.EventBeingDelivered = true;

            System.Diagnostics.Trace.WriteLine("Event processing commenced");

            Global.NewEventID++;
            // Add the current event to the list.
            Global.EventsList.Add(eo);
            // Register the amount of listeners waiting to get events
            Global.ListenerCount = Global.WaitingListenerList.Count;

            // Release the waiting response threads
            foreach (AutoResetEvent waitingListner in Global.WaitingListenerList) {
                waitingListner.Set();
            }

        }

        // Before clearing the event and the listener waithandles, wait for all the listeners to release.
        
        Global.AllListenersReleased.WaitOne();


        lock (Global.LockObj) {
            Global.LastEventID = Global.NewEventID;

            // If all logged-on users got the last event, clear the events list
            if (Global.loggedInUsers.Count == Global.WaitingListenerList.Count) {
                // TODO: Need to clear the Global.NewEventID, but also figure a way to notify the threads to reset their event ids
                Global.EventsList.Clear();
                Global.NewEventID = 0;
                Global.EventsListResetId++;
            }

            // Clear all the listeners
            Global.WaitingListenerList.Clear();

            // Once an event has been delivered to all waiting listeners, allow new listeners waiting to register to attempt registration,
            // or process a new event/messages.
            Global.EventBeingDelivered = false;
            Monitor.PulseAll(Global.LockObj);

        }

        System.Diagnostics.Trace.WriteLine("SendEventObject - Released listeners and cleared lists");

    }

If you’re wondering why the Global.LockObj isn’t enough to synch the two methods that’s because both methods exit the lock block and go into a wait state. The Listen method waits for an application event, and the SendEventObject waits for all Listen methods to complete before continuing to do some housekeeping.

Event Caching

As I mentioned above we need a mechanism to cache events until we can insure that all clients received them. This is achieved logically, not technologically.

All events are cached in a global list of application event objects call EventList. By retaining an incremental EventId for each event and what I call an EventsListResetID, both Globally and on the event object being pushed back, I can determine if events occurred since my last Listen.

The PrepareEventForSending method returns the events to the client from the point it last received an event.

// <summary>
/// Each waiting response thread will call this method to return the current and all outstanding events.
/// It's assummed that all events missing by the client are retained in the events list since the list
/// is not cleared until all logged on users a fully notified.
/// </summary>
/// <param name="fromEventIndex"></param>
/// <returns></returns>
internal static EventObject[] PrepareEventsForSending(int myLastEventID, int myLastEventsListResetId) {

    // If the eventlist hasnt been reset since the last time the client received an event,
    // return the next event in the list.
    if (myLastEventsListResetId == Global.EventsListResetId) {
        if (myLastEventID > Global.NewEventID)
            myLastEventID = 0;
    }
    else
        // List has been reset since client last got an event, so return all events from
        // the top of the list.
        myLastEventID = 0;

    // Determine how many events the client is missing.
    int numberOfEventsImMissing = Global.NewEventID - myLastEventID;
    // Determine where in the list the missing events start from.
    int startIndex = Global.EventsList.Count - numberOfEventsImMissing;
    int returnIndex = 0;

    // Assemble an array of missing events for this client.
    EventObject[] eventToReturn = new EventObject[numberOfEventsImMissing];


    for (int missingEventIndex = startIndex; missingEventIndex < Global.EventsList.Count; missingEventIndex++) {
        eventToReturn[returnIndex] = Global.EventsList[missingEventIndex];
        eventToReturn[returnIndex].EventID = missingEventIndex + 1;
        eventToReturn[returnIndex].EventsListResetID = Global.EventsListResetId;
        returnIndex++;
    }

    return eventToReturn;

The EventsListResetId gets incremented every time the EventList gets cleared. The EventList gets cleared after all registered clients receive an event.

You’ll notice that when the client’s last event list ID doesn’t match the global one on the server we return the entire list to the client. That’s because the mismatch indicates the client’s last EventID, which acts as an index on the list, is no longer valid because the list has been cleared since the last time the client was listening for events.

In other words a mismatch between the clients last EventsListResetId and the server’s EventsListResetId means that last time you got an event, everybody got the event, and we cleared the EventList. So your EventID is no longer valid and can now start from zero, meaning give me the events from the beginning of the new list.

Clearing the list is done at the end of the SendEventObject method by comparing the number of logged on users to the number of registered waithandles that are blocking a listener. If both are equal then it’s assumed all logged-on users received the last event, and any outstanding events, so we can clear the list and increment the EventsListResetID.

/// Before clearing the event and the listener waithandles, wait for all the listeners to release.
// ***************************************************************************************************
Global.AllListenersReleased.WaitOne();
// ***************************************************************************************************

lock (Global.LockObj) {
    Global.LastEventID = Global.NewEventID;

    // If all logged-on users got the last event, clear the events list
    if (Global.loggedInUsers.Count == Global.WaitingListenerList.Count) {
        Global.EventsList.Clear();
        Global.NewEventID = 0;
        Global.EventsListResetId++;
    }

    // Clear all the listeners
    Global.WaitingListenerList.Clear(); 

Request Timeouts

This part is easy. In the Global.asa I determine the request time and take 90% of it to make up my EventWaitTimeout.

protected void Application_Start(object sender, EventArgs e)
{
    // Set the event timeout to be less than the page timeout.
    Global.EventWaitTimeout = (int)(HttpContext.Current.Server.ScriptTimeout * .90) * 1000;

}  

In the Listen web method a block with this value as a time limit. When the limit is reached the method stops blocking, and a dummy event is set to everybody using the same mechanism we use to send real events (that’s why everyone will be notified). The event class used for dummy events is the parent class to all my event classes. The client knows to ignore these events, and re-requests a Listen.

//**********************************************************************
// Wait for an event, but continue before the request times out.
listenTimedOut = !waithandle.WaitOne(Global.EventWaitTimeout, true);
//**********************************************************************
// When the code continues from here it's either because an event was recieved
// or the wait for one has timed out.
//**********************************************************************
System.Diagnostics.Trace.WriteLine("Listen - Waithandle released");

if (listenTimedOut) {
    // No events to return? Return an empty non-event.
    if (Global.EventsList.Count == 0) {
        EventObject dummyEvent = new EventObject();
        lock (Global.LockObj) {
            dummyEvent.EventID = myLastEventID;
            dummyEvent.EventsListResetID = myLastEventsListResetId;
        }
        // Fire off the dummy event asynchrously and wait for it (short wait) so as to stay 
        // within the wait/event paradigm in this example.
        SendDummyEventDelegate sendEventDel = new SendDummyEventDelegate(EventUtils.SendEventObject);
        sendEventDel.BeginInvoke(dummyEvent, null, null);
        waithandle.WaitOne();
    }
} 

Conclusion

With a minimal amount of code you can create servers that serve up real-time events to your client applications. Although I haven’t tried it I’m sure this same technique can be used for web-clients using AJAX and page methods, or a least web service methods.

Note: If you’re using the ASP.NET development server to run the sample app you’ll need to stop it between runs.

References

http://www.albahari.com/threading/index.html

http://news.speeple.com/msdn.com/2007/07/23/asp-net-thread-usage-on-iis-7-0-and-6-0.htm

http://www.eggheadcafe.com/articles/20050613.asp