Facade Design Pattern - Invoke Informatica Workflow through Web Service

• Download source - 100.41 KB

Introduction

This article will introduce how we use Facade Design Pattern to create a .NET client to invoke informatica workflows through the web service call.

Background

Nowadays, many companies use informatica as their ETL tool to transfer their data format from one to another. The informatica workflow drives the rules to convert the data from the source file to the target file. In a general case, if we want to communicate from outside to informatica workflow, an empty text file will be sent as a trigger file from client to kick off the workflows. Once the informatica receives the trigger file as the command, the transfer process starts and the trigger file will be deleted.

Problems Defined 

This mechanism has a couple of drawbacks as described below: 

1. The client will lose the communication back from informatica after client sends out the trigger, client will never know the status about how that workflow is doing after it starts. 
2. Multiple trigger files need to be sent if client needs to invoke different workflows. Most likely, it uses a unique file name for each workflow to allow the informatica recognize it. 
3. It requires the .NET developer to have some informatica knowledge to configure the informatica workflows.

Solution

As .NET developers, ideally we prefer just to have a client component that packages all the informatica work for us. The only thing we need to do is to start workflow and receive the status back. It seems that informatica web service hub perfectly matches what we need. Informatica web service hub has provided many useful web methods in their Integration Web service. From the .NET client, we could just talk to the Integration Web Services to utilize them.

Web Service Hub

Facade Design Pattern

Since we have over thousands of informatica workflows created in house, probably it's not a good idea to let our client to directly communicate to an informatica web service. We do want to keep our client's work as less as possible, therefore we introduce an Facade Agent to package all the necessary work here, and enforce our client with a standard way to invoke the web service.

See the difference between Non-Facade design & Facade Design:

Implementation

Web Service Proxy Class

A web service proxy class has to be created in the .NET client side before making any communication calls. It's really straight forward if we use the VS.NET IDE. Go to Add Service Reference to auto generate the proxy class, point the current address to generate it.

Agent Class Public Methods

Agent Class will be the core component that is responsible to communicate from client to the Web service hub. Agent creates the corresponding request object and passes it to the web service hub to initialize a web service call. Before every single service call, the agent needs to login to the server to be authenticated. Once the authentication process is successful, an assigned ticket will be returned to the agent. Agent holds the ticket and attaches it along with the rest of the web service calls.

In Agent class, it inherits the IDisposable that helps us to dispose the object, logoff the server before we leave.

Login() method will send all the login information and get an assigned ticket after it's authenticated.

StartWorkflow() method will start a workflow by passing the workfolder name and workflow name.

GetWorkflowDetails() method can retrieve a workflow current status. The response object (WorkflowDetails) is very informative. It includes all the information that relates to the requested workflow.

GetTaskDetails() method allows us to receive a specific task in a running workflow.

LoginOff() method sends the request to logoff, and web service hub will destroy the authenticated ticket for this particular session.

Agent Class

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using www.askbargains.com.ETLAgent.DataIntegration;
using System.Security.Cryptography.X509Certificates;
using System.Net.Security;

namespace www.askbargains.com.ETLAgent
{
    class ServiceAgent: IDisposable 
    {                
       #region Fields

        LoginRequest _loginReq;
        DataIntegrationInterfaceClient _diClient;
        string _sessionID;
        SessionHeader _headerContext;
        bool _isLoggedIn;
        DIServiceInfo _disInfo;

        #endregion

        #region Constructor

        public ServiceAgent()
        {
            _loginReq = new LoginRequest();
            _diClient = new DataIntegrationInterfaceClient();
            _headerContext = new SessionHeader();
            _disInfo = new DIServiceInfo();
            _isLoggedIn = false;
        }

        #endregion

        #region Service Caller Methods
  
        // Login Web Service Hub to be authenticated.      
        public void Login(string repositoryDomainName,
                          string repositoryName,
                          string username,
                          string password,
                          string userNameSpace,
                          string serviceName)
        {
            if (!_isLoggedIn)
            {
                _loginReq.RepositoryDomainName = repositoryDomainName;

                _loginReq.RepositoryName = repositoryName;

                _loginReq.UserName = username;

                _loginReq.Password = password;

                _loginReq.UserNameSpace = userNameSpace;

                //uncommon the following statement if there is no 
                //certificated install for https:// etc...
                //ServicePointManager.ServerCertificateValidationCallback = 
                //				TrustAllCertificatesCallback;

                _diClient.login(_loginReq, out _sessionID);

                if (!string.IsNullOrEmpty(_sessionID))
                {
                    _headerContext.SessionId = _sessionID;
                }
                else
                {
                    return;
                }

                _disInfo.ServiceName = serviceName;
                _disInfo.DomainName = repositoryDomainName;

                _isLoggedIn = true;
            }
        }

       // Use this operation to start a workflow. 
       // You must log in to the repository before you call this operation
        public void StartWorkFlow(string folderName,
                                  string workflowName)
        {
            if (_isLoggedIn)
            {
                WorkflowRequest wfRequst = new WorkflowRequest();
                wfRequst.FolderName = folderName;
                wfRequst.WorkflowName = workflowName;
                wfRequst.RequestMode = ETaskRunMode.NORMAL;

                wfRequst.DIServiceInfo = _disInfo;

                var response = _diClient.startWorkflow(this._headerContext, wfRequst);
            }
            else
            {
                throw new InvalidProgramException
		("Please Login Web Service Hub to be authenticated ");
            }
        }

       // Use this operation to get the details of a given workflow. 
       // If the workflow is running, 
       // the operation returns the details of the running workflow. 
       // If the workflow is not running, the operation 
       // returns the details of the last run of this workflow. 
        public WorkflowDetails GetWorkflowDetails(string folderName,
                                                  string workflowName)
        {
            if (_isLoggedIn)
            {
                WorkflowRequest wfRequst = new WorkflowRequest();
                wfRequst.FolderName = folderName;
                wfRequst.WorkflowName = workflowName;
                wfRequst.RequestMode = ETaskRunMode.NORMAL;

                wfRequst.DIServiceInfo = _disInfo;

                return _diClient.getWorkflowDetails(_headerContext, wfRequst);
            }
            else
            {
                throw new InvalidProgramException
		("Please Login Web Service Hub to be authenticated ");
            }
        }
  
       // Use this operation to retrieve the details of a task 
       // from the Integration Service. 
       // If the parent workflow is running and the task has already run, 
       // the operation returns the details of the current task in the running workflow.
       // If the parent workflow is not running, the operation 
       // returns the task details of the last workflow run. 
        public TaskDetails GetTaskDetails(string folderName,
                                          string workflowName,
                                          string taskName)
        {
            if (_isLoggedIn)
            {
                TaskRequest tkRequest = new TaskRequest();

                tkRequest.FolderName = folderName;
                tkRequest.WorkflowName = workflowName;
                tkRequest.TaskInstancePath = taskName;
                tkRequest.RequestMode = ETaskRunMode.NORMAL;

                tkRequest.DIServiceInfo = _disInfo;

                return _diClient.getTaskDetails(_headerContext, tkRequest);
            }
            else
            {
                throw new InvalidProgramException
			("Please Login Web Service Hub to be authenticated ");
            }
        }

       //
       // Use this operation to stop a running workflow. When you stop a workflow, 
       // the Integration Service tries to stop all the tasks 
       // that are currently running in the workflow. 
       // If the workflow contains a worklet, the Integration Service also tries 
       // to stop all the tasks that are currently running in the worklet
        public void StopWorkflow(string folderName,
                                  string workflowName)
        {
            if (_isLoggedIn)
            {
                WorkflowRequest wfRequst = new WorkflowRequest();
                wfRequst.FolderName = folderName;
                wfRequst.WorkflowName = workflowName;
                wfRequst.RequestMode = ETaskRunMode.NORMAL;

                wfRequst.DIServiceInfo = _disInfo;

                var response = _diClient.stopWorkflow(_headerContext, wfRequst);
            }
            else
            {
                throw new InvalidProgramException
		("Please Login Web Service Hub to be authenticated ");
            }
        }

       // Use this operation to log out of a repository. 
        public void LoginOut()
        {
            if (_isLoggedIn)
            {

                VoidRequest a = new VoidRequest();
                _diClient.logout(_headerContext, a);
                _isLoggedIn = false;
            }
        }

        #endregion

        #region helper method

        public static bool TrustAllCertificatesCallback
			(object sender, X509Certificate cert,
       			X509Chain chain, SslPolicyErrors errors)
        {
            return true;
        }
        #endregion

        #region IDisposable Members

        public void Dispose()
        {

            if (_isLoggedIn)
            {
                LoginOut();
            }

            if (_diClient != null)
            {
                _diClient.Close();
            }
        }

        #endregion
    }
}

ClientFacade Class Public Method

In my ClientFacade, I package the Agent's public method to make an other public method call. I named it InvokeWorkflow(). In this way, I enforce all the clients to follow the predefined public method to invoke the workflow. I also use a standard approach for error handling, logging or reporting behind the ClientFacade.

Case Study

One client has 3 workflows needed to run. We store the workflow names into the workflowCollection object. It's just a simple List<string> generic type to save the workflow names. 

After client logs into the server, it will  loop through the workflowCollection object to start each workflow. Since the response object comes back from informatica web service which only tells us if the workflow has started or not, we need to go back to the web service to call the GetWorkflowDetails for each workflow until the workflow status is not in the running status. That means the workflow has been completed (either SUCCEEDED/FAILED)  

We can use the loop to achieve a polling approach. Once the workflow status is not RUNNING, then we assume it has been completed, and it needs to be removed/deleted from our workflowCollection object. The loop will keep polling the workflow status until our workflowCollection object is empty.

Now we know all the workflows have finished, and the workflow details for each workflow have been saved into the workflowStatus object. It contains all the WorkflowDetails object. What we need to do is use the LINQ to Object to select all the Non-SUCCEEDED workflows to write to a log file and throw the error to our client.           

ClientFacade Class 

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using www.askbargains.com.ETLAgent.DataIntegration;

namespace www.askbargains.com.ETLAgent
{
    public class ClientFacade
    {
        public void InvokeWorkflows()
        {
           try
            {
                //initial ServiceAgnet object
                using (ServiceAgent agt = new ServiceAgent())
                {
                    //start login. we could use an configuration file to 
                    //store those login parameters
                    agt.Login("PrepositoryDomainName",
                              "RepositoryName",
                              "Username" ,
                              "Password",
                              "UserNameSpace" ,
                              "ServiceName");
                    //create workflow collection to store the running workflow 
                    //info when we have multiple workflow to invoke
                    List<string> workflowCollection = new List<string>();
                    
                    //fill the workflowCollection, could use configuration file 
                    //to store all workflow and add them all here from the loop. 
                    workflowCollection.Add("WorkflowName1");
                    workflowCollection.Add("WorkflowName2");
                    workflowCollection.Add("WorkflowName3");

                    //loop through the Workflow collection to start each of them
                    foreach (string workflow in workflowCollection)
                    {
                        agt.StartWorkFlow("WorkflowFolderName", workflow);
                        workflowCollection.Add(workflow);
                    }

                    //create workflowDetails colleciton to store the 
                    //response objects back from the web service hub
                    List<workflowdetails> workflowStatus = new List<workflowdetails>();
                    
                    //check the workflow status
                    while (workflowCollection.Count != 0)
                    {
                        //loop the workflowCollection to check each workflow's status
                        foreach (string workflow in workflowCollection)
                        {
                            WorkflowDetails wfDetails = new WorkflowDetails();

                            //get the workflowDetails for a specific workflow 
                            //by its workflowfoldername & workflowname. 
                            //The information could be stored in the configuration file,
                            //instead of hardcode here.
                            wfDetails = agt.GetWorkflowDetails
				("WorkflowFolderName" , workflow);

                            //if workflow is not runing. that means either it SUCCEEDED/
		          //FAILED. we need to stop check this workflow anymore
                            if (wfDetails.WorkflowRunStatus != EWorkflowRunStatus.RUNNING)
                            {
                                //we push the response workflowdetail object to 
                                //the workflowdetail collection for later use.
                                workflowStatus.Add(wfDetails);
                                //remove the current workflow, so we won't check 
                                //the status for it anymore
                                workflowCollection.Remove(workflow);
                                //continue looping the rest workflows in our 
                                //workflowCollection
                                break;
                            }
                        } 
                    }
                    
                    //for each succeeded workflow, I log it to a log file.
                    var workflows = workflowStatus.Where
			(w => w.WorkflowRunStatus != EWorkflowRunStatus.SUCCEEDED);
                    if (workflows.Count() >= 0)
                    {
                        foreach (var workflow in workflows)
                        {
                            //log the unsuccessful workflow's information.
                            //Log4net.WriteLog("Workflow Error --> " + 
		          //workflow.WorkflowName + ", Error Msg:" + 
			 //workflow.RunErrorMessage, LogMessageType.Error);
                        }
                        //for the error handling, I prefer to just throw 
		      //the error to the upper class.
                        throw new Exception ("Workflow running failed");
                    }
                }
            }
            catch (Exception  ex)
            {
                throw ex;             
            }
        }  
    }
}

Conclusion

In this article, we use the Facade Design Pattern to implement a .NET component that allows the client to use a simple method to start the Informatica Workflows.

History

• 14^th August, 2009: Initial post