Organizations have been using numerous applications that were built over many years. These applications probably are of different era, were written by using different languages and technologies, reside on different hardware platforms, use different operating systems, and provide very different functionality. In fact, many of the applications often have very little in common at all, resulting in isolated functionality and multiple instances of the same data. For your organization, these conditions can result in redundant activities, higher costs, and inefficient response to your customers.
Types of Application Integration
Application integration can be broadly categorized into three types:
· Manual application integration
· Semi-automated application integration
· Fully automated application integration
Application Integration type
Higher labor costs that scale badly. Subject to human error.
Little change required from existing low-technology environment.
Higher technology costs to implement. Subject to design-time and run-time errors.
Lower labor costs. Scales better. Less subject to human error at run time. Faster processing.
Highest technology costs to implement. Subject to design errors.
Lowest labor costs. Not subject to human error at run time. Lose human decision making on business processes, but faster processing.
Making Application Integration Scalable
An important part of making application integration scalable is to increase the number of automated steps and reduce the number of human steps. This generally involves creating interfaces between applications along with predefined logic that replaces human involvement. For automated application integration, you have two choices:
● Point-to-point model
● Integration hub model
Requirements for Application Integration
Typically, an environment that supports application integration meets at least the following requirements:
· Connectivity between different platforms
· Processing of complex business rules, including complex data transformation logic
· Support for business processes, from the very short to the very long, including processes that last weeks or months as data is passed and processed through different parts of the organization
· The ability to modify existing business processes or create new ones as business goals change
· The ability to adapt to changes in hardware, software, and business goals
To help meet these requirements, your application integration environment should:
· Expose a common interface through which applications can communicate, by using business semantics to request Web services.
· Allow service requests at the functional or data level for applications that do not support using business semantics.
· Use a common set of process and service rules to ensure consistency and reuse of integration services.
· Be capable of reusing the existing transport protocols that already exist in the enterprise.
· Insulate it from existing technologies by using interfaces.
Challenges of Application Integration
When building an application integration environment, you are creating an environment that allows components to communicate with each other even though they were never designed to do so. In many cases, these components are not even aware of each other. They may include a variety of preexisting systems (some of which are essential), yet your environment needs to be noninvasive to existing applications.
In these conditions, complex architectural issues such as the following can arise:
· Control and coupling
· Data exchange and data formats
· Distribution and concurrency
· Scalability, reliability, and availability
You do not want to completely rebuild the application integration environment itself every few months. The aim, instead, is to have a relatively stable application integration environment, which is flexible enough to allow the addition of new applications or the alteration of relationships between applications.
Organizational issues in application integration arise mainly from the fact that the application integration environment will span multiple departments in the organization. Staff in different departments may choose to deploy applications that will then need to integrate with the rest of organization environment. To design a truly effective application integration environment, you need to isolate application integration as a separate management function, and determine who is responsible for making it happen. In larger organizations, you may have a group of people specifically responsible for this function, including application integration architects, application architects, and application owners, whereas in other cases, application integration may be one of many management functions that you are responsible for. Either way, by defining the application integration management function in this way, you help to ensure that it is properly considered whenever changes are made to the IT infrastructure.
Levels of Application Integration
Application integration can occur at many different levels, including:
· Presentation level
· Business process level
· Data level
· Communications level
Business Process – Level Integration
Business process – level integration often forms the starting point for defining application integration environment, because it is most closely related to the business requirements of the organization. Business process – level integration starts with defining your business processes and then specifying the logical integration that corresponds to those processes.
Defining Your Business Processes
Before you can design your application integration environment, it is important to understand the organization in logical terms, removed from the technology that underlies it. Business processes consist of a group of logical activities that model the activities in your organization. These activities typically represent the smallest units of logical work that can be done in the organization.
Mapping Business Requirements to Application Integration Requirements
After you have determined the business processes that the organization uses, you can begin to examine how they affect your application integration requirements. It is important to realize that a simple business process does not always correspond to simple requirements for application integration.
By modeling each of your business processes, you can define which applications must integrate with one another, and in what way. This enables you to make intelligent decisions about the specifics of application integration environment.
You can define how applications communicate with each other at a business process level, but if they cannot understand the data they exchange, they cannot integrate successfully
There are two general ways to enable data-level integration:
· Add logic to enable each application to understand incoming data from other applications.
· Add logic to enable each application to interpret outgoing data to an intermediate data format and interpret incoming data from that format into a form the application understands.
To support the various complex data integration requirements, your application integration solution must often contain a considerable amount of logic that supports the access, interpretation, and transformation of data. You also need schematic models of enterprise data to describe data attributes. The definition and recognition of schemas enables the validation of data. Descriptions of how data elements may be related or mapped to each other allow for the transformation of data.
Enabling Communication between Applications
Because many applications are not designed to communicate directly with each other, it is likely that you will need to do some development work to ensure that your applications can be called by other applications.
There are two possible approaches to this problem:
· Rewrite your application to provide it with an API that other applications can call.
· Create a communications adaptor that acts as an intermediary between the application and other applications.
Important Considerations for Application Integration
There are few important considerations regarding Application integration and they are as follows
· Using Synchronous and Asynchronous Communication
· Increasing Automation
· Straight-Through Processing (STP)
· Ensuring Data Integrity
· Managing Latency
· Data Aggregation
· Combining Application Functionality
· Application Composition
· Managing Transactions
o Providing ACID Transactions
o Transactional Messaging
o Two-Phase Commit
o Long-Running Transactions
o Timed Transactions
Security and Operational Issue
Because different applications have different security requirements and features, it can be quite a challenge to ensure that your application integration environment functions properly without compromising your security requirements. Security is particularly important in application integration because a breach in an integration service may result in security breaches in other integrated systems.
The first step toward effective security in any environment is creating a written security policy. Many factors can affect the security policy, including the value of the assets you are protecting, the threats that your environment faces, and the vulnerabilities that are currently present
From an integration standpoint, your security policy should define:
- A mechanism for evaluating and classifying threats
- A mechanism for acting on threats
- A boundary for information security
- A plan for communication and enforcement.
- General security guidelines
- Reference to other documents
- A mechanism for modifying security policies
Defining Your Security Requirements
The precise security requirements of application integration environment depend on a number of factors, including the following:
· Security requirements of the organization
· Business requirements for application integration
· Technical requirements for application integration
· Capabilities of the applications which will be integrated
· Platforms of the applications
· Budgetary constraints
As a starting point for defining your security requirements, you should perform a risk analysis. Doing so allows you to identify the threats and vulnerabilities that you face and identify the countermeasures you can deploy to keep risk at an appropriate level.
Operational Management Considerations
Even after you build an application integration environment and it is running successfully, your work does not stop there. The environment will need to be monitored and maintained over time. This section discusses the various operational management considerations involved. Implementing the capabilities listed in this section should help to ensure that your architecture continues to function as smoothly as possible.
Defining an Operational Management Policy
Your operational management policy, at the minimum, should provide the following information:
· Clearly defined terminologies and target metrics
· Methodology and/or formulas for measuring the metrics to ensure that results are consistent
· Service-level prioritization to ensure that the most important policies are followed first
Defining Operational Requirements
The operational requirements of application integration environment depend on a number of factors, including the following:
· System Monitoring
· System and application health
· System and Application Performance Monitoring
· Security monitoring
· Service-level monitoring
· Business Activity Management
· Business transaction exception handling
· Contextual monitoring
· Rules-based alerting
· Historical data mining
· Change and Configuration Management
Microsoft Technology/Tools Mapping on EAI Capabilities
Capabilities Required for EAI Environment
The capabilities are defined as follows:
· Business process integration capabilities
· Data integration capabilities
· Communications capabilities
· Security capabilities
· Operational management capabilities
Business Process Integration Capabilities
Custom .NET Framework components, SQL Server stored procedures, and BizTalk Server Business Rules Engine
Transaction Management and Compensation
Microsoft Distributed Transaction Coordinator (DTC) and BizTalk Server long-running transactions
Windows Workflow Foundation and BizTalk Server Human Workflow Services
BizTalk Orchestration service
Common language runtime, IIS session management, and BizTalk Server orchestration state management
SQL Server triggers, MSMQ Events, and BizTalk Server messaging subscriptions
Windows Server Schedule service and SQL Server Job Scheduler
Windows Communication Foundation and BizTalk Orchestration service
Data Integration Capabilities
ADO.NET and SQLXML
Visual studio 2003/2005
SQL Server DTS, XSLT and BizTalk Mapped
XML schemas and BizTalk Server parsers
Windows Communication Foundation and BizTalk Server Message Box and subscriptions
BizTalk Server message ports
Microsoft Message Queuing and BizTalk Message Queuing
Windows Communication Foundation and BizTalk Server Message Box and subscriptions
BizTalk Orchestration correlation services
BizTalk Server message ports
Microsoft Message Queuing and BizTalk Server reliable messaging
Windows Card Space and Active Directory
Windows Card Space and Active Directory and Share Point Portal Server Enterprise single sign-on
Windows Card Space and <city w:st="on"><place w:st="on">Enterprise single sign-on
<city w:st="on"><place w:st="on">Enterprise single sign-on
Security Context Management
Active Directory and <city w:st="on"><place w:st="on">Enterprise single sign-on
Operational Management Capabilities
Business Activity Management
BizTalk Server Business Activity Monitoring (BAM)
Windows Management Instrumentation (WMI) events, Event Viewer, Performance Monitor, and MOM
System Management Server
BizTalk Server Business Rules Engine and schema versioning
Windows Server and MOM