|
|||||||||||||||||||||||
|
|||||||||||||||||||||||
|
Announcements
Chapters
Services
Feature Zones
|
DisclaimerThis code is not production ready; it is designed to demonstrate a theoretical solution to using COMET in ASP.NET. This article covers the server-side implementation of COMET and how to combat the scalability problems. To demonstrate the client code, I will release a smaller article soon which demonstrates a small tic-tac-toe game using the COMET thread pooling mechanism I mention below, which should give you some idea about using it in a real world application. IntroductionOver the past six months, I have been putting together an online chess application where players can register, login, and play chess in real-time. One of the obstacles I had to overcome was how to implement a real-time communication between the client and the server. For this to be successful, a number of factors need to be addressed:
So, I evaluated up all of my options. My first prototype used a standard AJAX mechanism which polled the server; this created too much latency and too much traffic, so I quickly moved on from this. I researched other mechanisms of transport, such as using socket communication via a hidden Flash applet; this required a browser plug-in, and so I moved on. I then found the COMET idea, and thought, bingo, this is what I want to use, so I did some more research and built a prototype. The idea behind COMETCOMET uses a persistent connection between the client (Web Browser, using This mechanism solves the Performance requirement; it means that whenever a message is needed to be sent to the client, and if a persistent connection is open, the client should receive it with very little latency, almost instantly. A second connection is used to send messages to the server; this connection is not persistent, and typically returns immediately after it is processed. From the point of view of a chess game, the persistent connection would be waiting for my opponent’s move, while the non-persistent connection would send my move. Real world use implementation of COMETSo far, everything looks great on paper; we have a mechanism that can deliver messages to a browser in real-time without a plug-in, but in practice, this is much more difficult. Many articles that describe the features of using a persistent connection comment about how much of a "hack" this is. I tend to disagree with these statements. It is true that COMET can run into issues when running on some browsers (mainly because HTTP imposes a two connection per browser, per host limitation). This limitation of the HTTP protocol was implemented to provide better performance for normal browsing over low bandwidth connections (e.g., dialup modems), and can cause performance issues when running COMET (there are ways around this!). This issue is only really noticeable in Internet Explorer (I think IE is strict with this standard up until IE8); Firefox 2 allows more connections, and manages them better, and Firefox 3 allows even more, meaning the future for COMET style applications is bright. The second issue comes from the scalability of the technology, and this is mainly what this article is trying to remedy. This issue is present because the platforms lack the support for a COMET style protocol, and presently do not scale well when using persistent connections. I would say this is not a failure of the overall idea of COMET, rather a failure of the implementation of a specific COMET server. Other developers have put together servers that sit in front of the platforms we develop on, these allow us to separate the COMET request mechanism from the web server, and allow the solution to scale by managing their own persistent connections. What I will demonstrate in this article is how you should not use COMET in ASP.NET, and the possible solution. Load testing COMETThe major drawback with persistent connections to ASP.NET is that each connection consumes an ASP.NET worker thread for the five seconds that the connection is open. Therefore, each client that connects will hold a thread from the ASP.NET thread pool and, eventually, under load, the server will stop responding. To demonstrate this, I put together a very simple application to simulate persistent connections to an ASP.NET application, with a handler that holds the requests open for 5 seconds before returning to the client: public class CometSyncHandler : IHttpHandler
{
#region IHttpHandler Members
public bool IsReusable
{
get { return true; }
}
public void ProcessRequest(HttpContext context)
{
int workerAvailable = 0;
int completionPortAvailable = 0;
ThreadPool.GetAvailableThreads(out workerAvailable,
out completionPortAvailable);
Debug.WriteLine("CometSyncHandler.ProcessRequest Start");
Debug.WriteLine(string.Format("Worker Threads Available: {0}",
workerAvailable));
Debug.WriteLine(string.Format("Completion Port Threads Available: {0}",
completionPortAvailable));
DateTime now = DateTime.Now;
while (true)
{
Thread.Sleep(50);
if (DateTime.Now.Subtract(now).TotalSeconds >= 5)
break;
}
Debug.WriteLine("CometSyncHandler.ProcessRequest End");
}
#endregion
}
This handler is very simple, it holds the execution of the request up to for 5 seconds, then returns. This simulates a COMET request that would eventually timeout and return the client. I also wrote a console application that ramps up The screenshot below shows this happening: I ramped up approximately 50 connections with two instances of the CometClientSimulator application, and as the website performance dropped, it started to drop connections from the simulator. To repeat this test, you can open and debug the CometAsync website, which would open default.aspx and set everything running, then open an instance of the CometClientSimulator console application and type addsyncclients; this will then ramp up 25 clients, adding one every two seconds into the ASP.NET application. Clearly, this is no good for any real world application, so I did some digging and devised a solution. IHttpAsyncHandlerThis is the first part of the solution. This little piece of magic allows us to run code asynchronously on the web server when we service a request to a handler. If you are not familiar with
The sequence diagram below shows how this would work:
CometThreadPoolThe sequence diagram above introduces a custom thread pool for servicing the COMET requests that sits around on the server. This is required because we don't want ASP.NET to use one of its threads while waiting around for a COMET request to timeout. The code for this pooling mechanism is located in the website in the CometAsync folder. It contains the following files:
This implementation works by first creating a set of background protected void Application_Start(object sender, EventArgs e)
{
//
// queue 5 threads to run
// the comet requests
CometThreadPool.CreateThreads(5);
}
This fires up five threads that idle along in the background, waiting for Our public IAsyncResult BeginProcessRequest(HttpContext context,
AsyncCallback cb, object extraData)
{
int workerAvailable = 0;
int completionPortAvailable = 0;
ThreadPool.GetAvailableThreads(out workerAvailable,
out completionPortAvailable);
Debug.WriteLine(
string.Format(
"BeginProcessRequest: {0} {1} out of {2}/{3} ({4} Requests Active)",
Thread.CurrentThread.IsThreadPoolThread,
Thread.CurrentThread.ManagedThreadId,
workerAvailable,
completionPortAvailable,
CometWaitRequest.RequestCount));
// get the result here
CometAsyncResult result =
new CometAsyncResult(context, cb, extraData);
result.BeginWaitRequest();
// ok, return it
return result;
}
The public void BeginWaitRequest()
{
CometThreadPool.QueueCometWaitRequest(new CometWaitRequest(this));
}
This code creates a new instance of the internal static void QueueCometWaitRequest(CometWaitRequest request)
{
CometWaitThread waitThread;
lock (state)
{
// else, get the next wait thread
waitThread = waitThreads[nextWaitThread];
// cycle the thread that we want
nextWaitThread++;
if (nextWaitThread == maxWaitThreads)
nextWaitThread = 0;
CometWaitRequest.RequestCount++;
}
// queue the wait request
waitThread.QueueCometWaitRequest(request);
}
This logic picks a The CometWaitThread classinternal void QueueCometWaitRequest(CometWaitRequest request)
{
lock (this.state)
{
waitRequests.Add(request);
}
}
The request is added into an internal list of At this point, the private void QueueCometWaitRequest_WaitCallback()
{
// here we are...
// in a loop
while (true)
{
CometWaitRequest[] processRequest;
lock (this.state)
{
processRequest = waitRequests.ToArray();
}
Thread.Sleep(100);
for (int i = 0; i < processRequest.Length; i++)
{
// timed out so remove from the queue
if (DateTime.Now.Subtract
(processRequest[i].DateTimeAdded).TotalSeconds >= 5)
{
//
// queue anotehr wait callback, so
// we tell close handler down
// the endRequest will exist on a
// different thread to this
// one and not tear down this thread
processRequest[i].Result.ResponseObject =
this.CheckForServerPushEvent(processRequest[i], true);
this.QueueCometWaitRequest_Finished(processRequest[i]);
}
else
{
object serverPushEvent =
this.CheckForServerPushEvent(processRequest[i], false);
if (serverPushEvent != null)
{
// we have our event, which is good
// it means we can serialize it back to the client
processRequest[i].Result.ResponseObject =
serverPushEvent;
// queue the response on another
// ASP.NET Worker thread
this.QueueCometWaitRequest_Finished
(processRequest[i]);
// dequeue the request
DequeueCometWaitRequest(processRequest[i]);
}
}
Thread.Sleep(100);
}
}
}
The It processes each item in the queue in a sequenced order in each loop iteration, e.g., if there were three requests pending, it would check request 1, 2, then 3, then continue the loop and process request 1, 2, and 3 again. This ensures that each request is processed as soon as possible without having to wait for a previous request to finish its 5 second timeout. The loop checks to see if the private void QueueCometWaitRequest_WaitCallback()
{
.
.
// queue the response on another ASP.NET Worker thread
this.QueueCometWaitRequest_Finished(processRequest[i]);
// dequeue the request
DequeueCometWaitRequest(processRequest[i]);
.
.
}
.
.
private void QueueCometWaitRequest_Finished(object target)
{
CometWaitRequest request = target as CometWaitRequest;
request.Result.SetCompleted();
}
.
.
The public void EndProcessRequest(IAsyncResult result)
{
int workerAvailable = 0;
int completionPortAvailable = 0;
ThreadPool.GetAvailableThreads(
out workerAvailable, out completionPortAvailable);
Debug.WriteLine(string.Format("EndProcessRequest: {0} {1}" +
" out of {2}/{3} ({4} Requests Active)",
Thread.CurrentThread.IsThreadPoolThread,
Thread.CurrentThread.ManagedThreadId,
workerAvailable,
completionPortAvailable,
CometWaitRequest.RequestCount));
CometAsyncResult cometAsyncResult = result as CometAsyncResult;
if (cometAsyncResult != null &&
cometAsyncResult.ResponseObject != null)
{
DataContractJsonSerializer serializer =
new DataContractJsonSerializer(
cometAsyncResult.ResponseObject.GetType());
serializer.WriteObject(
cometAsyncResult.HttpContext.Response.OutputStream,
cometAsyncResult.ResponseObject);
}
cometAsyncResult.HttpContext.Response.End();
}
This method responds to the client by serializing whatever response object we set in the completion into the response stream of the request's One thing to mention here is what threads are actually doing the processing of the request. When it arrives in We can see that in action in the following screenshot: At this point, it is also worth mentioning that the response from the website is very good, considering there are 200 persisted connections, and the CPU/memory usage for the box is good (it is also running the clients). To also double check everything is working smoothly, I log a counter for each request and response pair to ensure that each request is met with a response. The screenshot below shows this output from running the test with 200 clients for five minutes:
This shows that all the requests were complete successfully, (apart from 1, but I put that down to a small race condition in the shutdown code, you can see it completes after!). ConclusionBy implementing a custom thread pool, we can leverage a COMET mechanism in our ASP.NET server code without implementing a custom server, or even implementing any complicated messaging routines, just a simple thread pool to manage multiple requests (e.g., we had five threads managing all 200 COMET requests). Using the codeThe website project CometAsync will execute and provide a default.aspx page that lists the amount of available threads available to ASP.NET when that request executes. The CometClientSimulator application will simulate the client connections; it should be executed as follows: CometClientSimulator [websitehost]
websitehost = the host name of the website CometSync app is running e.g. http://localhost:2251
I am currently using a similar engine on my chess website, but it is undergoing vigorous testing right now. By the way, if anyone’s interested in playing chess on there and wants to help out with the testing, drop me an email at imebgo@gmail.com. Thanks! History
|
||||||||||||||||||||||