Click here to Skip to main content
14,355,411 members

Using Managed Extensibility Framework to Build a Modular Console Application

Rate this:
5.00 (6 votes)
Please Sign up or sign in to vote.
5.00 (6 votes)
24 Jul 2019CPOL
This article takes you through a step by step journey of building a fairly large console application which is modular and also extensible by using the principles of Managed Extensibility Framework.

Sample Image - maximum width is 600 pixels

Table of Contents

Problem Description

You are developing a command line utility. This could be a suite of custom batch jobs to support the management of an enterprise application. Your command line utility is expected to carry out several back end tasks and each of these tasks is parameterized through arguments that are specific to the task. This would not appear to be a challenge at all if our requirements are very well bounded and restricted. My experience has taught me that any software is simple when it begins but grows exponentially with the passage of time. Your end users are going to ask for more features and very soon, you are facing a daunting task of managing the development and delivery of a very complex piece of software.

If you are building an enterprise application managed tool, then your users would be the staff supporting the IT operations in your organization, or if you are a company like Github or Aws/Azure, then millions of developers. In this article, I will present a solution to the above problem by leveraging Microsoft's Managed Extensibility Framework.

Example - A Custom Backend Management Tool

Util.exe --task Backups --from 01/01/2019  --to 30/06/3019 --destination c:\Back\
Util.exe --task IndexDocuments
Util.exe --task ExtractImages -destination c:\dump\    

Example - Github

git config –global user.name "[name]"
git commit -m "[ Type in the commit message]"
git diff –staged
git rm [file]
git checkout -b [branch name]    

Example - Azure CLI

az group create --name myResourceGroup --location westeurope
az vm create --resource-group myResourceGroup --name myVM --image UbuntuLTS --generate-ssh-keys
az group delete --name myResourceGroup

Background

Knowledge about .NET, C# is essential. Some ideas about Managed Extensibility Framework is good to have.

Possible Approaches

Approach 1 - Custom PowerShell cmdlets

PowerShell is a beautiful framework which can be easily extended by writing custom .NET modules. Both Azure and AWS provide PowerShell interfaces to interact with their respective infrastructure on the Cloud. PowerShell cmdlets are simple C# classes that inherit from CmdletBase. PowerShell cmdlets give you the best of both worlds - a strongly typed development environment in the form of Visual Studio.NET and a superb scripting platform which becomes the client of your cmdlet (class library).

Image 2

Approach 2 - Multiple Command Line Applications on a Per Task Basis

This is a very simple approach. It works well if your requirements are small and there is a sense of urgency.

Approach 3 - Single Command Line Application Where Every Task is a Plugin

In this article, I will be focusing on the third approach. One single command line application that can perform various tasks and each task being encapsulated in its own class and then in later stages, the task implementations are physically isolated out into assemblies.

Util.exe --task Backups --from 01/01/2019  --to 30/06/3019 --destination c:\Back\
Util.exe --task IndexDocuments
Util.exe --task ExtractImages -destination c:\dump\

Brief Introduction to Managed Extensibility Framework (MEF)

MEF is a library built on top of Microsoft .NET Framework/Core and simplifies the development of plugin based applications. MEF can be considered to be a dependency injection framework with the capability to discover dependencies across assembly partitions. MEF opens up the possibility of decoupling your main application from the implementations. Microsoft's documentation on MEF can be found here. MEF addresses some very pertinent questions that frequently arise in the software development lifecycle:

  • Can your application be extended after it has been shipped without having to recompile the entire codebase?
  • Can your application be designed in such a way that so that the application can find its modules at runtime as opposed to compile time binding?
  • Can your application be easily extended by adding new modules/plugins?

Step 1 - Design the Contract

public interface IMyPlugin
{
    void DoSomeWork()
}

Step 2 - Implement Various Plugin Classes Which Implement Your Contract

///
///Class1 in Assembly1
///
[Export(typeof(IMyPlugin))]
public class Plugin1 : IMyPlugin
{
}

///
///Class2 in Assembly2
///
[Export(typeof(IMyPlugin))]
public class Plugin2 : IMyPlugin
{
}

Step 3 - Design Your Host Application to Accept the Discovered Implementations

///
///Host application
///
public class MyHost 
{
    [ImportMany(typeof(IMyPlugin))]
    IMyPlugin>[] Plugins {get;set;}
}

Step 4 - Discover Plugins Using Catalog Classes of MEF

///
///TO BE DONE  - Show snippets of catalog here
///

Lazy Loading of Classes in MEF

You can make MEF delay the instantiation of the plugin classes. MEF uses the class Lazy to discover implementations and hold a reference to the metadata of the plugins. The instantiation is done only when required. The class Lazy allows plugins to export meta-data. E.g., a unique name for a plugin.

///
///Plugin class with metadata
///
[Export(typeof(IMyPlugin))]
[ExportMetadata("name","someplugin2")]
[ExportMetadata("description","Description of someplugin2")]
public class Plugin2 : IMyPlugin
{
}

The attribute ExportMetadata plays a vital role here. When the class MyHost has been composed using MEF, the Dictionary object in the Lazy instance of every invokable plugin class is populated with the keys name, description and their values respectively. Remember - the plugin class is not yet instantiated.

///
///Host application - with lazy loading of plugins
///
public class MyHost 
{
    [ImportMany(typeof(IMyPlugin))]
    Lazy<IMyPlugin,Dictionary<string, object>>[] Plugins {get;set;}
}

Part 1 - A Simple Console EXE Which Uses Command Line Arguments and MEF to Identify a Task Handler

Overview

In this subsection, we will develop a simple EXE which is modularized into Task handler classes and the classes reside within the executable itself.

Agreeing on a Standard System of Command Line Arguments

For the purpose of this article, we will name our command line application as MefSkeletal.exe and the first argument will be the short name of a task. All arguments that follow would be simply arguments that are specific to the task.

Myutil.exe [nameoftask] [task argument1] [task argument 2]

MySkeletal.exe task1 arg0 arg1 arg3
MySkeletal.exe task2 arg5 arg6
MySkeletal.exe task3    

.NET Core EXE

Create a .NET Core EXE project MefSkeletal. For now, we will follow a simple approach where all Task handler classes are contained within the EXE project. In the later stages, we will refactor the solution so that every Task is contained in a separate class library project.

Contract Interface

Create a Contracts subfolder and create a class file ITaskHandler.cs:

///
///Every Task handler must implement this interface
///
public interface ITaskHandler
{
    void OnExecute(string[] args)
}

Nuget Packages

Add references to the following packages:

  • Install-Package System.ComponentModel.Composition -Version 4.5.0

Create the Task Handler Classes

We will add Task specific handler classes. Create a Tasks subfolder and add the following classes in this subfolder. Each of the classes implement the interface ITaskHandler. Add the MEF metadata name to make it discoverable.

///
///Task 1 - 
///
[Export(typeof(ITaskHandler))]
[ExportMetadata("name","task1")]
public class Task1 : ITaskHandler
{
    public void OnExecute(string[] args)
    {
        Console.WriteLine("This is Task 1");
    }
}
///
///Task 2 - TO BE DONE - Add MEF metadata
///
[Export(typeof(ITaskHandler))]
[ExportMetadata("name","task2")]
public class Task2 : ITaskHandler
{
    public void OnExecute(string[] args)
    {
        Console.WriteLine("This is Task 2");
    }
}
///
///Task 3 - TO BE DONE - Add MEF metadata
///
[Export(typeof(ITaskHandler))]
[ExportMetadata("name","task3")]
public class Task3 : ITaskHandler
{
    public void OnExecute(string[] args)
    {
        Console.WriteLine("This is Task 3");
    }
}

Create a Container Class to Import All Instances of Task Handlers

Create a new class Container.cs under the Contracts folder.

///
///Container class 
///
public class Container
{	    
[ImportMany(typeof(ITaskHandler))] 
    public Lazy<itaskhandler, dictionary="">>[] Tasks { get; set; }        
}

Discover Plugins Using the AssemblyCatalog Class

MEF provides different ways to resolve dependencies. For our example, we will use the AssemblyCatalog to discover various task classes:

///
///Container - Discover plugins
///
public class Container
{
    public Container() 
    { 
        var assem = System.Reflection.Assembly.GetExecutingAssembly(); 
        var cat = new AssemblyCatalog(assem); 
        var compose = new CompositionContainer(cat); 
        compose.ComposeParts(this); 
    } 
}

Instantiate the Lazy Instance and Invoke the Method OnExecute of ITaskHandler

MEF metadata is a useful way to de-couple implementations from their actual classes. We have provided a short name for each of our plugin ITaskHandler implementation classes. We will use the name metadata attribute to find and instantiate a concrete instance of ITaskHandler. The properties Value and IsValueCreated of the Lazy class are useful.

internal void ExecTask(string taskname,string[] args)
{
    var lazy = this.Tasks.FirstOrDefault(t => (string)t.Metadata["name"] == taskname);
    if (lazy == null)
    {
        throw new ArgumentException($"No task with name={taskname} was found" );
    }
    ITaskHandler task = lazy.Value;
    task.OnExecute(args);
}

Putting It All Together - Executing a Plugin from the Main Method

We are nearly done. The main method would bind all that we have just done.

static void Main(string[] args)
{
    try
    {
        Container container = new Container();
        string taskname = args[0];
        container.ExecTask(taskname, args);
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.ToString());
    }
}

Testing the EXE

Navigate to the output folder and fire the following commands:

Image 3

Part 2 - Extending the Console EXE by Implementing a Simple Help System

Overview

It would be nice if our simple .NET Console EXE could expose some usage documentation. Similar to the PowerShell help system, we want the documentation to be displayed on the command line. Ideally, we would want every Task handler to be responsible for publishing their own documentation. Why not make one single implementation of ITaskHandler solely dedicated for displaying Help (HelpTask.cs)? The HelpTask class should leverage the MEF meta-data attributes name and description.

Command Line Protocol for Displaying Help

///
///When the user types any of the following commands 
///     1)Preliminary information should be displayed
///     2)The list of available Tasks should be displayed
///
MyUtil help
MyUtil /?
MyUtil
///
///When the user types the following command, 
/// 1)Display help information specific to the task. 
///
MyUtil help task1

Using MEF Metadata to Make Every Task Emit Its Own Documentation

We will add the "help" meta-data to each of the ITaskHandler implementations. This attribute will store a meaningful usage information:

[Export(typeof(ITaskHandler))] 
[ExportMetadata("name", "task1")] 
[ExportMetadata("help", "This is Task1. Usage: --arg0 value0 --arg1 value1 --arg2 value2")] 
public class Task1 : ITaskHandler 
{ 
    public void OnExecute(string[] args) 
    { 
       Console.WriteLine("This is Task 1"); 
    } 
}

Create a New ITaskHanlder Implementation to Display the Help

In the implementation of HelpTask, we have two methods - DisplayAllTasks and DisplayTaskSpecificHelp. To discover information about other Tasks, we need to access an instance of the Container class. The MEF Attribute Import helps us in injecting dependencies when Lazy objects are instantiated:

[Export(typeof(ITaskHandler))]
[ExportMetadata("name", "help")]
public class HelpTask : ITaskHandler 
{ 
    public void OnExecute(string[] args) 
    { 
        if (args.Length  == 0)
        {
            DisplayAllTasks();
        }
        else
        {
            string taskname = args[0];
            DisplayTaskSpecificHelp(taskname);
        }
    }
    
    ///
    ///MEF will resolve this dependency at the time of instantiation
    ///
    [Import("parent")]
    public Container Parent { get; set; }        
}

How Does MEF Import Work?

To resolve a dependency marked by the Import attribute, MEF will look for a matching property annotated with the Export attribute:

public class Container
{
///
/// Used for dependency injection. E.g. HelpTask.cs 
/// would need this to discover all other Task objects
///
[Export("parent")]
public Container Parent { get; set; }
}

Method - DisplayAllTasks

/// 
/// Display a short list of all Task names. 
///         
private void DisplayAllTasks()
{
    Console.WriteLine("List of all Tasks");
    foreach(var lazy in this.Parent.Tasks)
    {
        string task = ((string)lazy.Metadata["name"]).ToLower();
        if (task == "help") continue;
        Console.WriteLine("-----------------------");
        string help = null;
        if (lazy.Metadata.ContainsKey("help"))
        {
            help = lazy.Metadata["help"] as string;
        }
        else
        {
            help = "";
        }
        Console.WriteLine($"{task}      {help}");
    }
}

Method - DisplayTaskSpecificHelp

/// 
/// Display the help description for the specified Task 
/// 
private void DisplayTaskSpecificHelp(string taskname)
{
    Console.WriteLine($"Displaying help on Task:{taskname}");
    var lazy = Parent.Tasks.FirstOrDefault
          (t => (string)t.Metadata["name"] == taskname.ToLower());
    if (lazy == null)
    {
        throw new ArgumentException($"No task with name={taskname} was found");
    }
    string help = (lazy.Metadata.ContainsKey("help") == false) ? 
    "No help documentation found" : (string)lazy.Metadata["help"];
    Console.WriteLine($"Task:{taskname}");
    Console.WriteLine($"{help}");
}

Putting It All Together - Parsing Command Line Arguments in the Main Method

Start
|
|
|
Analyze command line arguments
|
|
|
If zero arguments OR args[0] is 'help' then execute task 'help'
static void Main(string[] args)
{
    try
    {
        Container container = new Container();
        string taskname = null;
        if ((args.Length == 0) || (args[0].ToLower() == "help" ) || 
                                  (args[0].ToLower() =="/?"))
        {
            taskname = "help";//This is our custom ITaskHandler
                              //implementation responsible for displaying Help
        }
        else
        {
            taskname = args[0];
        }
        container.ExecTask(taskname, args.Skip(1).ToArray());
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.ToString());
    }
}

Testing - Displaying List of All Tasks

Image 4

Testing - Displaying Task Specific Help

Image 5

Part 3 - Implementing Tasks in Separate Assemblies

Overview

We now have the knowledge to refactor a complex executable into multiple classes where every class performs a specific Task. We know how to discover these classes through MEF lazy loading and ultimately invoking the methods via a contractual interface. There is one final step. We need to solve the problem of decoupling the various Task handler classes from the main executable. This will allow us to extend the system in a modular fashion without having the need to re-compile the complete executable.

Create a Class Library for the Contracts

Add interfaces ITaskHandler and IParent. The interface IParent will provide contextual information to every implementation of ITaskHandler:

/// 
/// Should be implemented by every custom Task implementation
/// 
public interface ITaskHandler
{
    void OnExecute(string[] args);
}
/// Allows a Task implementation to interact with the host
/// E.g. Task1 can get to know about other Task implementations 
/// that have been discovered through MEF
public interface IParent
{
	Lazy<ITaskHandler,Dictionary<string,Object>>[] Tasks {get;}
}

Create a .NET Core EXE

Add the following classes. Add reference to the Contracts class library project. For the sake of brevity, I have only displayed a portion of the source code.

MefHost.cs

Discover all subfolders in the Plugins subfolder and create a DirectoryCatalog for each subfolder. Combine AssemblyCatalog and DirectoryCatalog objects into a single instance of AggregateCatalog.

public class MefHost : MefDemoWithPluginsFolder.Contracts.IParent
{
    ///
    ///Responsible for discovering plugins by using a combination 
    ///of AggregateCatalog, AssemblyCatalog and DirectoryCatalog
    ///
    public MefHost(string folderPlugins)
    {
        List<DirectoryCatalog> lstPluginsDirCatalogs = new List<DirectoryCatalog>();
        ///
        ///Create a collection of DirectoryCatalog objects
        ///
        string[] subFolders = System.IO.Directory.GetDirectories(folderPlugins);
        foreach(var subFolder in subFolders)
        {
            var dirCat = new DirectoryCatalog(subFolder, "*plugin*.dll");
            lstPluginsDirCatalogs.Add(dirCat);
        }
        var assem = System.Reflection.Assembly.GetExecutingAssembly();
        var catThisAssembly = new AssemblyCatalog(assem);
        ///
        ///Combine all the DirectoryCatalog and 
        ///AssemblyCatalog using AggregrateCatalog
        ///
        var catAgg = new AggregateCatalog(lstPluginsDirCatalogs);
        catAgg.Catalogs.Add(catThisAssembly);
        var compose = new CompositionContainer(catAgg);
        this.Parent = this;
        compose.ComposeParts(this);
    }
}

Program.cs

We have used the Plugins subfolder under the EXE for all the plugin assemblies:

class Program
{
    static void Main()
    {
        string exeFile = System.Reflection.Assembly.GetExecutingAssembly().Location;
        string exeFolder = System.IO.Path.GetDirectoryName(exeFile);
        string folderPlugins = System.IO.Path.Combine(exeFolder, "Plugins");
        MefHost host = new MefHost(folderPlugins);
        string taskname = null;
        if ((args.Length == 0) || (args[0].ToLower() == "help") || 
                                  (args[0].ToLower() == "/?"))
        {
            taskname = "help";
        }
        else
        {
            taskname = args[0];
        }
        host.ExecTask(taskname, args.Skip(1).ToArray());
    }
}

HelpTask.cs

Similar to the implementation in previous sections.

Create Task1 and Task2 Plugin Class Libraries

///
///Task 1
///
[Export(typeof(MefDemoWithPluginsFolder.Contracts.ITaskHandler))]
[ExportMetadata("name", "task1")]
[ExportMetadata("help", "This is Task1. 
        Usage: --arg0 value0 --arg1 value1 --arg2 value2")]
public class Class1 : Contracts.ITaskHandler
{
    public void OnExecute(string[] args)
    {
        string sArgs = string.Join("|",args);
        Console.WriteLine($"This is Task 1. Arguments:{sArgs}");
    }
}

Add the Element CopyLocalLockFileAssemblies to the Task1 and Task2

The CSPROJ file of Task1 and Task2 needs a line of modification. We should set the element CopyLocalLockFileAssemblies to true. Why are we doing this? We want the class libraries to emit all the referenced assemblies. If this were .NET Framework, you would achieve the same by setting the Copy Local attribute. For .NET Standard and Core projects, dependent assemblies are not emitted right away.

<PropertyGroup>
    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
</PropertyGroup>

Add Post Build Step to Copy the Output of Task1 and Task2 to a Plugins Folder

We should remember that we are moving away from "hard" references. The EXE has no compile time knowledge of the existence of Task1 and Task2. In such a scenario, the output of Task1 and Task2 should be copied over the Plugins folder. For this project, we have selected the subfolder 'Plugins' directly under the EXE. To avoid repetition, we will script a BAT file to do the XCOPY. The BAT file reside in the root of the Solution. The physical layout of the solution would be as follows:

EXE----
        |
        |
       Bin--
            |
            |
            Release
                |
                |
                netcoreapp2.1
                    |
                    |
                   Plugins
                        |
                        |
                        Task 1
                        |
                        |   (all assemblies,PDB and other files from the Bin of Task1)
                        |
                        |
                        Task 2

                            (all assemblies,PDB and other files from the Bin of Task2)

Using the Code

Visual Studio 2017 would be a necessity.

Github

MefConsoleApplication.sln

Demonstrates a simple .NET Core console EXE and using MEF to discover ITaskHandler implementations within the same executable assembly.

MefConsoleApplicationWithPluginsFolder.sln

Demonstrates loading of plugin assemblies from an external folder.

References

History

  • 24th July, 2019: Initial version

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Sau002
Software Developer (Senior)
United Kingdom United Kingdom
With over 22 years experience in software development. My first job was to port C and Fortran code from UNIX to Windows NT. I have worn many hats since then. Windows forms, Web development, Windows Presentation Framework, Silverlight, ASP.NET, SQL tuning, jQuery, Web API, SharePoint and now machine learning.

My book on Neural Network: http://amzn.eu/8G4erDQ


Comments and Discussions

 
QuestionWindows Forms + ClickOnce Pin
kiquenet.com29-Jul-19 11:55
professionalkiquenet.com29-Jul-19 11:55 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Article
Posted 23 Jul 2019

Stats

4.7K views
13 bookmarked