The Application Automation Layer - Design And Implementation of The Bootstrap Loader And The Component Manager

• Download demo project - 75.4 Kb

Introduction

This is the second installment of a continuing development effort. Collaboration is welcome, and a SourceForge site is in process.

Prior articles:

• Introduction And Design[^]

Table Of Contents

Introduction

What Is A Bootstrap Loader

What Is The Component Manager?

What Is A Component

What Microsoft Considers A Component

What Is A Workflow And What Is The Workflow Manager?

Design Considerations

Centralized Reflection

Decentralized Reflection

Proxy Delegates

Interface Classes

Design Decision

Component Load And Initialization Sequence

Component Load Sequence

Startup Process Sequence

The Initialization Sequence

Phase 1

Phase 2

Component List

Implementation

The Bootstrap Loader

The Component Loader

A Test Component

Output Results

Closing Comments

Introduction

The fundamental concepts of the Application Automation Layer are:

1. decouple object dependencies
2. promote abstraction in object designs
3. promote component-based development
4. eliminate data format specialization
5. provide automatic instrumentation of process and data pathways
6. ensure a consistent design and implementation style
7. provide a framework that is responsive to requirement changes during and after the development cycle
8. provide an enhanced level of automation for known and stable technologies
9. interfaces to the plug-in remain stable (or are at most extended) during the lifetime of the plug-in

In order to achieve these goals, a framework must exist on which to build the component based technology. This requires several basic building blocks:

1. A Bootstrap Loader
2. A Component Manager
3. A Data Hub
4. A Workflow Manager

This article describes in detail the design considerations for the Bootstrap Loader and Component Manager. Initial implementations for both are provided.

What Is A Bootstrap Loader

The Bootstrap Loader is an EXE common to all applications using the AAL framework. It initiates the loading process of all technologies associated with the application. It also initializes internal application specific data, such as storing the command line parameters into the Data Hub component.

In the C++/MFC implementation of the AAL, the bootstrap loader also included implementations of several methods that could not exist in a DLL—basically, CWinApp overridden methods: InitInstance, ExitInstance, OnIdle, etc.

What Is The Component Manager?

The Component Manager (CM) is an essential building block to the Application Automation Layer. Its purpose in life is to load the components necessary to run the application and to provide registration/deregistration services for public interfaces.

What Is A Component

A component consists of:

1. publicly available interface methods;
2. private data instances;
3. private classes, methods, etc.

A component:

1. Registers publicly available interface points implemented as static methods;
2. Implements a technology solution that typically consists of:
   • Translating data to and from the common data representation (CDR) used by the Data Hub;
   • Methods that manipulate data—storage, retrieval and transformation;
   • Abstracting technology specific events by interfacing with the Workflow Manager to invoke workflow processes based on these events;
   • Additional script processing for application specific definitions that apply to the particular technology.

What Microsoft Considers A Component

The above definition of a component diverges from what Microsoft terms a component. (If someone has a better name for what I’m calling a component, let me know.) In Microsoft’s implementation:

1. A component is closely associated with its container;
2. Non-remotable component development usually requires an implementation for the IContainer, IComponent and ISite interfaces;
3. Remotable component development requires deriving from the base implementations for System.ComponentModel.Component;
4. The Component framework in .NET has design-time support for:
   • Displaying a component on a design surface;
   • Showing properties and events in a property browser;
   • Handling attributes;
   • Making components look and behave like components in Visual Studio .NET
   • Remoting
   • Marshalling by value or by reference

There is a webcast and PowerPoint presentation provided by Microsoft that covers the basics of developing components and controls (excluding remoting and marshalling) with Visual Studio .NET (in fact, that’s the title) here.

In the AAL framework, this implementation puts too much functionality into a single abstraction by combining loosely related concepts. In brief, components should not:

1. be associated with design-time processes;
2. be coupled with the development of GUI controls;
3. implement a remoting framework;
4. be associated with a marshalling framework.

Q: Why isn’t the component manager part of the bootstrap loader?

A: Several reasons. The bootstrap loader should be as minimal of an EXE as possible. The Bootstrap Loader represents the most generic element of an application. The Component Manager should be considered a component itself. In terms of design and code changes, the Bootstrap Loader should be the most stable piece of code and the Component Manager should be the second most stable piece of code. The Bootstrap Loader performs certain initialization functions such as command line parameter extraction that is inappropriate for the Component Manager to perform.

Q: Why are the interface points implemented as static methods?

A: Because the interface points comprise a façade design pattern. Instance, if necessary, is determined by a dictionary referencing instances by some token (usually a text string), and is resolved by the component. This is a useful layer of abstraction that is leveraged by the Workflow Manager (which will be illustrated in the future). While it creates a certain amount of overhead, it also provides the ability to easily enumerate all instances in a dictionary—a useful diagnostic feature.

What Is A Workflow And What Is The Workflow Manager?

A workflow is a collection of processes implemented as public interfaces (as in, a site) registered via the component manager. The Workflow Manager interprets a meta-language that specifies the individual processes to execute. A workflow is not necessarily a linear set of processes. It can include variables extracted from the Data Hub, rules, loops, sub-processes and state information. The Workflow Manager will be fully designed and implemented in a later article.

Design Considerations

The following discusses four different design approaches and their pros and cons.

Centralized Reflection

The entire “glue” between plug-ins could be implemented using Reflection. It could even be done in such a manner as to load the required plug-in on first usage by specifying the exposed interface points in a dictionary of methods and the associated component DLL. An unknown method would be looked up in the dictionary.

By using a common management facility like the Component Manager, instrumentation would still be achieved at the point where the reflected methods are invoked. Plug-ins such as the Workflow Manager would communicate with the CM to invoke the desired function.

Comments/Issues

Ideally, the Component Manager would load a plug-in on first usage and call an initialization function similar to the two phase approach described above. This also eliminates the requirement for a common “go-between” assembly (see Delegates and Interface below). Issues may exist in load ordering, and there is a performance hit with determining whether the component has already been loaded.

Decentralized Reflection

In another variation, the plug-in could specify its dependencies on other plug-ins and loads them on its own. Reflection can then be used to invoke methods, which would be managed by the plug-in itself.

Comments/Issues

This method decentralizes the plug-in dependency dictionary concept and the plug-in management. Unfortunately, this means that advantages gained by instrumentation are bypassed unless the component implements instrumentation on its own. Having each component manage the issues involved in reflection results in redundant code. This could be placed in a common assembly, but then you get back to the same problem—if a change is made to the reflection management code, then all dependant assemblies need to be recompiled.

Proxy Delegates

The plug-in registers its interface points with the Component Manager and provides a delegate for the callback. This essentially implements a proxy design pattern.

Comments/Issues

Somewhat simpler than using reflection and the delegate is type checked. The problem is that it:

1. Requires a common assembly referenced by the plug-in and the Component Manager that specifies the delegate;
2. When a component is extended, the common assembly needs to be updated, forcing a recompilation of all the dependant assemblies.
3. Each component would probably publish its own delegate assembly resulting in complicated file management issues.

Interface Classes

Alternately, the interface capability of C# can be used. By obtaining an instance of the object, you can communicate with the object directly.

Comments/Issues

1. It defeats one of the fundamental features of the AAL--instrumentation.
2. While it successfully decouples objects, it promotes application-centric coding. This eliminates the benefits gained with a workflow and data centric approach because objects are now communicating with each other instead of through a central workflow and data hub.
3. It makes it too easy for the programmer to break other plug-ins by changing the interface specification, especially regarding parameters to methods.
4. One of the fundamental concepts of the AAL is to get programmers to think more in terms of abstraction, modularization and components. Interfaces are too close to their derived class’ implementation.
5. Each plug-in provides an interface class specification. This specification must be placed in an assembly shared by the provider and the requestor. As with the delegate approach, the resulting file set would involve complicated management issues.
6. If a plug-in is extended, the interface specification would need to be extended, requiring recompilation of all dependent components.

Design Decision

Interfaces, except for performance considerations, are not suitable because they violate too many of the tenets of the AAL. Delegates (like interfaces) require a common assembly and have the additional problem of requiring a declaration for each unique set of parameters. Then, the caller has to know which delegate to use based on the parameter set. Reflection has an overhead associated with it but has the advantage of not needing a common assembly.

Therefore, reflection is chosen as the primary means of accessing public interface points within different plug-ins. To reduce overhead, a master file will be used which specifies all the plug-ins required by the application. The CM will preload all information required to invoke the method via reflection, so that only:

1. a dictionary lookup
2. parameter marshalling (to be discussed in the future)
3. the reflection call

is required. The plug-in itself is responsible for registering its public interface points with the CM as opposed to using an external specification file (however, the CM can generate this file after all the components have been loaded). Finally, the Component Manager (CM), Data Hub (DH) and Workflow Manager (WM) will each provide interface classes using a façade design pattern (to enhance isolation from the actual implementation). These are core technology components which are highly abstract and stable. Using an interface class to directly communicate with these components improves performance and increases the readability of the plug-in code.

Experience with the C++/MFC implementation of the AAL has shown that plug-ins rarely need to communicate between each other, but communicate frequently with the three core components (CM, WM, DH). Cross-component communication indicates either a poor design or that a workflow is being implemented which should instead utilize the Workflow Manager. The Component Manager should be used in circumstances where cross-component communication is necessary.

Q: Aren’t you just re-inventing existing solutions, such as ActiveX or COM?

A: Yes, in a sense I am. However, these technologies lack instrumentation and fail to support other tenets of the AAL (see the next question below). They require a managed code interface to work with C#. Finally, they violate the principles of data-centric and workflow-centric processing.

Q: Why not use .NET’s Component and Container classes?

A: Because they violate the tenets of the AAL. The built-in ability to use a .NET component at design time is not utilized in the AAL because again, it violates the tenets of what I consider to be true component-based development. In other words, you may think Microsoft’s component architecture gives your team an advantage, but this is an illusion because you end up tightly coupling your code with the component’s implementation. As an example of this, I had to rewrite significant aspects of an interface to Visio’s “component” with the new release of Visio. Variables' types had changed, method names had changed, data structures had changed and the invocation ordering of some processes had changed. Fortunately, because I was using the AAL, all of these changes were restricted to the AAL’s component wrapper of the Visio component. Once I made the necessary changes, all my applications that interface with Visio via the AAL (and there are several) worked again without any further changes! If I had used Microsoft’s methodology, I would have had to make all those changes to each application, repeatedly. This illustrates an important tenet of an AAL component—it’s interface MUST remain stable, even though the implementation might vary.

Q: What About Remoting?

A: The AAL architecture doesn’t implement remoting in the typical way. Instead, remoting is achieved by interconnecting Data Hubs, Component Managers and Workflow Managers. This issue is left for future implementation.

Component Load And Initialization Sequence

Component Load Sequence

The following diagram illustrates the load sequence of components:

Note that the Bootstrap Loader is responsible for loading only the Component Manager. After this, it is the Component Manager which is responsible for loading the required and optional technology components.

Startup Process Sequence

The specific load process is detailed in this diagram:

After the Component manager has loaded all the technology modules, it specifically initializes the Data Hub and then returns control to the Bootstrap Loader. This gives the Bootstrap Loader the opportunity to place startup parameters (for example, command line parameters) into the Data Hub before the remaining components are initialized. The Bootstrap Loader then returns control to the Component Manager which performs the three phases of component initialization, as illustrated in this diagram:

After the initialization sequence is complete, control is returned to the Bootstrap Loader which instructs the Workflow Manager to begin executing the startup workflow. The startup workflow is almost always an application specific workflow implemented in the AAL meta-language.

The Initialization Sequence

As illustrated in the following diagram, the initialization sequence consists of two phases:

Phase 1

During internal initialization, the component has the opportunity to instantiate internal (private) objects that are independent of other technologies. Secondly, the component registers with the Component Manager its own publicly available interface points. These interfaces represent processes that are part of a workflow (see below for further discussion of workflows).

Phase 2

At this point, all public interfaces have been registered. The component now has the opportunity to perform any final initialization that might require communication with another component.

Component List

While each application using the AAL shares the same EXE, the components that it loads are usually different. For now, we’ll load a component list from a simple text file that enumerates the components by name. For example, datahub.dll. This file must exist in the same directory as the Bootstrap Loader EXE.

Implementation

The following sections illustrate the code behind the design issues discussed above. Please be aware that the code is in the prototyping phase.

The Bootstrap Loader

This is a quite simple process in which the Component Manager's initialization method is invoked using reflection. Since this happens a lot in the Component Manager, the Bootstrap Loader draws upon a method in a "library" (actually an assembly) to load the Component Manager assembly and return a MethodInfo with the desired function.

using System;
using System.Reflection;
using AAL;

namespace AAL
{
    public class BootstrapLoader
    {
        static void Main(string[] args) 
        {
            Dbg.Initialize();
            Dbg.Problems.Add(
                new DbgKey("BadOrMissingDLL"),
                "",
                new string[] {"The necessary DLL is missing or corrupt."});
            Dbg.Problems.Add(
                new DbgKey("BadReflectionName"),
                "",
                new string[] {"The assembly/namespace.class" +
                " or function name cannot be found."});
            Dbg.Problems.Add(
                new DbgKey("BadInvoke"),
                "",
                new string[] {"The requested component " + 
                "and function name could not be found."});

            MethodInfo mi=
              Lib.Methods.GetMethodInfo
              ("ComponentManager.dll/AAL.ComponentManager/Init1");
            ICore.IComponentManager icm=mi.Invoke(null, 
               BindingFlags.Public | BindingFlags.NonPublic |
               BindingFlags.InvokeMethod | BindingFlags.Static, 
               null, null, null) as ICore.IComponentManager;
            icm.InitializeComponents();
        }
    }
}

The Component Loader

The CM reads the file components.txt, which in this example includes the two lines:

TestComponent.dll/Testbed.TestComponent
WorkflowManager.dll/AAL.WorkflowManager

The constructor for the CM adds itself to the component dictionary, loads the Data Hub component and initializes it, and loads the other components specified in the text file. Note that the Data Hub is only a stub at this point. The CM initialization function that the Bootstrap Loader calls returns an interface object that can be subsequently used to invoke the CM directly.

public ComponentManager()
{
    components=new Hashtable();
    icm=new ICM(this);
    components.Add("ComponentManager", 
      new Component("ComponentManager", icm));
    LoadComponent("DataHub.dll/AAL.DataHub");
    LoadOptionalTechnologies();

    // initialize data hub
    InitializeComponentPhase1((components["DataHub"] as Component).init1);
    InitializeComponentPhase2((components["DataHub"] as Component).init2);

    // designate pre-initialized components
    (components["ComponentManager"] as Component).initialized=true;
    (components["DataHub"] as Component).initialized=true;
}

When a component method is invoked through the CM, the following method in the CM is used:

public object[] Invoke(string fnc, object[] parms)
{
    string compName=AAL.Lib.StringHelpers.LeftOf(fnc, '.');
    string fncName=AAL.Lib.StringHelpers.RightOf(fnc, '.');
    Component comp=components[compName] as Component;
    Dbg.Fail(comp != null, new DbgKey("BadInvoke"), fnc);
    MethodInfo mi=comp.ipoints[fncName] as MethodInfo;
    Dbg.Fail(mi != null, new DbgKey("BadInvoke"), fnc);
    Dbg.WriteLine("AAL: Invoking "+fnc);
    object[] wrapperParms=new Object[] {parms};
    object[] retVals=(object[])mi.Invoke(null, BindingFlags.Public |
         BindingFlags.NonPublic |
         BindingFlags.InvokeMethod |
         BindingFlags.Static, null, wrapperParms, null);
    return retVals;
}

A Test Component

The test component is very simple. It merely outputs a string to the debug output when the interface point is invoked. Note that the Init1 function returns a null. The Component Manager records this value as an interface object that the component can optionally provide for direct access. As described in the design section, only the three core AAL components should be interfaced to directly, however this option is provided for additional core components and for possible performance issues.

using System;
using AAL;
using AAL.ICore;

namespace Testbed
{
    public sealed class TestComponent
    {
        private static IComponentManager icm;

        public static object Init1(IComponentManager icm)
        {
            TestComponent.icm=icm;
            icm.RegisterInterfacePoint("Test", 
              "TestComponent.dll", 
              "Testbed.TestComponent", "Test");
            return null;
        }

        public static void Init2()
        {
        }

        public static object[] Test(object[] parms)
        {
            Dbg.WriteLine("***Test***");
            return null;
        }
    }
}

Output Results

In order to test the capability of the system so far, the following line is added in the Bootstrap Loader (for test purposes only):

icm.Invoke("TestComponent.Test", null);

It illustrates the elegance (hopefully) of this system, in its ability to invoke a method in a component that is not referenced by the application or by the Component Manager. This demonstrates the concept of a true plug-in capability--all dependencies have been removed, as illustrated in the debug trace:

...
AAL: Loading component: DataHub.dll/AAL.DataHub
'BootstrapLoader.exe': 
  Loaded 'c:\projects.net\csaal\bin\debug\datahub.dll', 
  Symbols loaded.
AAL: Loading component: TestComponent.dll/Testbed.TestComponent
'BootstrapLoader.exe': 
  Loaded 'c:\projects.net\csaal\bin\debug\testcomponent.dll', 
  Symbols loaded.
AAL: Loading component: WorkflowManager.dll/AAL.WorkflowManager
'BootstrapLoader.exe': 
  Loaded 'c:\projects.net\csaal\bin\debug\workflowmanager.dll', 
  Symbols loaded.
AAL: Phase 1 - WorkflowManager
AAL: Phase 1 - TestComponent
AAL: Registering Test
AAL: Phase 2 - WorkflowManager
AAL: Phase 2 - TestComponent
AAL: Invoking TestComponent.Test
***Test***
The program '[1384] BootstrapLoader.exe' has exited with code 0 (0x0).

Closing Comments

This is a lot of concept and code to do very little right now, and to do it in a really complicated way. However, it is a crucial element to the entire issue of automation and the AAL framework, so hang in there!

Beware, this is a preliminary design and implementation! Note that the issue of parameters is nicely ignored. This, and the design/implementation of the Data Hub, is the topic of the next article. In the MFC implementation, I utilized the concept of a message envelope to implement the marshalling/unmarshalling of parameters. A similar system will be developed for the C# implementation.

In the previous article, I discussed a component called the Process Manager. This component has been renamed to be the Workflow Manager.