How to do asynchronous programming using ASP.NET, MSMQ and Windows Service, for long running processes

Introduction

We all have experiences of programming web programs. But very few of us actually have experience of long running Web programs that often times-out in the browser or kills the machine due to memory waste. We had this requirement of having to print about 20000 reports from the web directly to the printer. The idea was to create PDF at runtime and then send them to printer. However, with Crystal Reports runtime and ASP.NET worker processes, the entire process was going extremely slow; besides, the memory usage by aspnet worker process was tremendously high and no other processes could be run. At this point, the decision was taken to do asynchronous programming using MSMQ and Windows Service.

Background

The reader must have some basic understanding of MSMQ, Windows Services, and the .NET platform. Here are a few important web links for clearing the basic concepts.

• Leveraging MSMQ in ASP.NET Applications.
• Let's do the Message Queue! -- An Introduction in MSMQ.
• Messaging Patterns in Service-Oriented Architecture, Part 1.

Creating the Code

Before peeking at the code, we must have an idea of what we are supposed to do and how. Take a look at the above diagram. The diagram tries to illustrate the overall architecture and flow of the program to be discussed. This initially looks a little arcane, but as we go along the article, every line in this diagram will be as easy as anything you can think of.

Let's start with the blue ASP.NET Cluster. The user uses a web user control to submit message(s) into the messaging queue. Instead of posting a simple message, we post a custom RequestMessage object into the queue. This object is maintained in a separate assembly - Assembly 1. So the web user control submit button actually does two things:

1. Creates the RequestMessage object and hydrates it with appropriate data from the user inputs.
2. Submits the RequestMessage object to the Message queue using appropriate formatter. The web user thereby rests in peace.

//the web user control sample code
using System;
using System.Collections;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.Messaging;
using System.ServiceProcess;
//refer the assembly1 containing the requestmessage object
using Assembly1;
/*other functions and code goes here.
we will concentrate on our specific task */ 
........
.......
......
private void btnPrint_Click(object sender, System.EventArgs e)
{
    SendPrintRequestToMessageQueue();
}

private void SendPrintRequestToMessageQueue()
{
    try
    {
        //pick up the queuename from the web.config file
        string queueName = 
          ConfigurationSettings.AppSettings["PrintChecksMessageQueue"];
        MessageQueue destMsgQ = new MessageQueue(queueName);
             /*instantiate the message request object with correct parameter
        in this case we can assume that the parameter 
        comes from a drop downlist selection*/
        PrintCheckRequestMessage objMessage = new 
            PrintCheckRequestMessage(ddlBatchRun.SelectedItem.Text.Trim());
        //Create a new message object
        Message destMsg = new Message();
        /*In case of unsuccessful attempts to post messages 
        , send them to dead letter queue for audit traceability*/
        destMsg.UseDeadLetterQueue = true;
        /*Use binary formatter for posting the request message 
        object into the queue*/
        destMsg.Formatter = new System.Messaging.BinaryMessageFormatter();
        //Assign the request message object to the message body
        destMsg.Body = objMessage;
        //Post the message
        destMsgQ.Send(destMsg);
    }
    catch(System.Exception ex)
    {
        //display the error message somewhere (on a label may be)
    }
}

On the other hand, we have a pink Windows Service Cluster written in C# for communicating with the MSMQ. So there are two different programs communicating with the MSMQ to facilitate the asynchronous communication. The provider, i.e., the Web application posting messages to the queue, and the consumer, i.e., the Windows service which is receiving messages from the queue by polling it at regular intervals. Now, let's talk about the Windows service. I assume that you have atleast a basic familiarity of writing Windows services in .NET. Also assuming that the following code will not be difficult to understand. At a pre-specified time interval, the Windows Service calls the Message Queue and 'receives' one message at a time from the queue, unwraps the message using the formatter with which it was wrapped and posted by the ASP.NET program, rehydrates an instance of the RequestMessage object, and uses that object to do some meaningful database activity that will solve the client's business requirements. If that sounds fun, then follow on.. check out the PickupFromMSMQ function called from the timer elapsed event handler function.

using System;
using System.Collections;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.ServiceProcess;
using System.Messaging;
//refer the assembly1 containing the requestmessage object
using Assembly1;
//refer the assembly2 containing the sqlprovider object
using Assembly2;

namespace windowsMQPollService
{
    public class windowsMQPollService : System.ServiceProcess.ServiceBase
    {
        // a timer object for polling MSMQ at regular intervals
        private System.Timers.Timer timerPolling;
        private bool isTimerStarted;
        private System.Collections.Hashtable htConfigData;
        private string m_StrQueuePath=string.Empty;
        create the requestmessage object
        public  PrintCheckRequestMessage objCheckMsgRequest = 
            new PrintCheckRequestMessage(string.Empty);
        private System.ComponentModel.Container components = null;
        private System.Diagnostics.EventLog exceptionEventLog;
        //create the SqlProvider
        private SqlDataProvider checkProvider = new SqlDataProvider();

        public windowsMQPollService()
        {
            // This call is required by the Windows.Forms Component Designer.
            InitializeComponent();
            if (!System.Diagnostics.EventLog.SourceExists(this.ServiceName)) 
            {       
                System.Diagnostics.EventLog.CreateEventSource(this.ServiceName, 
                                                                "Application");
            }
            exceptionEventLog.Source = this.ServiceName;
            exceptionEventLog.Log = "Application";
        }

        private void InitializeComponent()
        {
            this.exceptionEventLog = new System.Diagnostics.EventLog();
            ((System.ComponentModel.ISupportInitialize)
              (this.exceptionEventLog)).BeginInit();
            this.ServiceName = "windowsMQPollService";
            ((System.ComponentModel.ISupportInitialize)
              (this.exceptionEventLog)).EndInit();
        }
    
        //Set things in motion so your service can do its work.
        protected override void OnStart(string[] args)
        {
            //Read config file entries for the Windows service 
            //into a hashtable for faster retrieval
            GetConfigurationData();
            /*check whether the config file had 
            correct entries or there were some 
            errors while populating the hash*/
            if (htConfigData["Service.MSMQPath"] == null)
            {
                exceptionEventLog.WriteEntry("Failed to read" + 
                    " the config file information.", 
                    System.Diagnostics.EventLogEntryType.Error);
            }
            int iPollingInterval = 60;
            if (htConfigData["Service.PollingInterval"] == null)
            {
                exceptionEventLog.WriteEntry("Polling Interval not specified." + 
                  " The service is starting assuming " + 
                  "polling interval to be 60 minutes.", 
                  System.Diagnostics.EventLogEntryType.Warning);
            }
            else
            {
                try
                {
                    //get the configured polling interval  
                    iPollingInterval = 
                        int.Parse((string)htConfigData["Service.PollingInterval"]);
                }
                catch
                {
                    exceptionEventLog.WriteEntry("Not a valid value" + 
                      " specified for Polling Interval. The service is starting" + 
                      " assuming polling interval to be 60 minutes.", 
                      System.Diagnostics.EventLogEntryType.Warning);
                }
            }

            //Create timer of interval iPollingInterval.
            timerPolling = new System.Timers.Timer();
            timerPolling.Elapsed += new 
                 System.Timers.ElapsedEventHandler(OnTimer);
            timerPolling.Interval = (double)(iPollingInterval * 60 * 10);
            timerPolling.Enabled = true;
            isTimerStarted=true;
            //start the polling activity of the Windows service

            timerPolling.Start();

        }

        private void OnTimer(object source, System.Timers.ElapsedEventArgs e)
        {
            if(isTimerStarted)    
            {
                timerPolling.Stop();                
                //At appropriate intervals get messages from the queue
                PickupFromMSMQ();
                timerPolling.Start();
            }
        }

        private bool PickupFromMSMQ()
        {    
            //read the queuepath from the hashtable 
            m_StrQueuePath = htConfigData["Service.MSMQPath"].ToString();

            string formattedMessage = string.Empty;
            try
            {
                System.Messaging.MessageQueue mqRecvQueue = 
                  new System.Messaging.MessageQueue(m_StrQueuePath);
                //use binary formater for receiving messages from the queue 
                mqRecvQueue.Formatter = new 
                            System.Messaging.BinaryMessageFormatter();
                DataSet allChecksDS;        
                //receive the message from the queue        
                System.Messaging.Message msgSrcMessage = mqRecvQueue.Receive();
                //recreate the requestmessage object
                objCheckMsgRequest = 
                      (PrintCheckRequestMessage)msgSrcMessage.Body;
                string strBatchNumber = objCheckMsgRequest.AuditId;
                //use the dataprovider to call functions to get DataSet
                allChecksDS = 
                      checkProvider.GetDataSetByBatchNumber(strBatchNumber);
                /*do your own stuff with the DataSet.
                For us it was calling another assembly 
                to print uncountable crytsal reports*/
                ....................
                ....................
                return true;            
            }
            catch(Exception ex)
            {
                //write exception message into the eventlog
                exceptionEventLog.WriteEntry("Error while reading from Queue :" + 
                    ex.Message +"--"+ 
                    ex.StackTrace,System.Diagnostics.EventLogEntryType.Error );
                return false;
            }
        }

        private void GetConfigurationData()
        {
            try
            {
                htConfigData = new System.Collections.Hashtable();
                System.Collections.Specialized.NameValueCollection colNameVal; 
                colNameVal = 
                  System.Configuration.ConfigurationSettings.AppSettings; 
                foreach(string sKey in colNameVal.AllKeys) 
                { 
                    htConfigData.Add(sKey,colNameVal[sKey]);
                }
            }
            catch(System.Configuration.ConfigurationException e)
            {
                exceptionEventLog.WriteEntry("The configuration file is missing." + 
                    " Could not start the service",
                    System.Diagnostics.EventLogEntryType.Error);
                throw(new Exception("Service Startup Error : " +e.Message));
            }
        }
    }
}
//some amount of code have been intentionally not mentioned

Note:

• Both the Windows service and the ASP.NET application must understand the MSMQ and the RequestMessage object. So if you look at the diagram, we have placed the Assembly 1 containing the RequestMessage class and the MSMQ instance at the center of both the clusters (Blue ASP.NET Cluster and Pink Windows Service Cluster) to give you an idea of how the plausible deployment scenario can be. Assembly 1 is shared by both the ASP.NET application and the Windows Service; System.Messaging namespace is used by both the ASP.NET application and the Windows Service for accessing MSMSQ.
• The SqlProvider assembly, in our case, is only used by the Windows Service and not by the Web application, so we have the Assembly 2 only in the Pink cluster.
• If the SqlDataProvider component makes use of any configuration file (say for storing DB connection information), then that file must be a available in the \bin folder of the Windows service along with its own app.config file, or else the service could not be run successfully.
• The Windows Service could make use of other assemblies to do other stuff. Like in our programs, we called another assembly with the DataSet values retrieved that contained code for printing out uncountable number of Crystal reports to a pre-specified printer.
• The queue name must be parameterized in both the Web application and the Windows service, and we should use config files for that. In case of the provider web application, use web.config file, and for the Windows Service, use app.config file. Below are examples of sample entries in the config files.

//Web.config entry
<appSettings>
    <add key="PrintChecksMessageQueue"
            value="name_of_the_machine_running_msmq\private$\Queue_Name"/>
</appSettings>

//Windows Service App.config entry
<appSettings>
    <add key="Service.MSMQPath"
            value="name_of_the_machine_running_msmq\private$\Queue_Name"/>
    <add key="Service.PollingInterval" value="10"/>
</appSettings>

Points of Interest

Programming with ASP.NET, MSMQ, Windows Service using different assemblies can be fun and seemingly easy at a glance, but depending on the business requirements and the type of deployment scenario, things can be quite complicated. Gaining an idea of the various messaging models possible is a healthy way to start off asynchronous messaging programming. Do check those links given above for detailed understanding.