Plug-in Manager

• Download source files - 24.7 Kb
• Download demo project - 16 Kb

Introduction

Frequently, developers discover that they have very specific requirements given by clients which would benefit only a very small group of people, and it becomes hard to justify the inclusion of features that meet only their requirements within the main codebase. Code bloat and feature creep can be dangerous for many reasons which we won't address here, but generally, the best solution to the problem is to allow external plug-ins to address those specific requirements without causing the code to become unmanageable.

.NET allows the dynamic loading of assemblies that is necessary for a good plug-in implementation. Runtime compilation can sometimes be a good alternative, especially when the end users are likely not to have development tools available to them, but there are some sacrifices in performance, and if the end users do happen to have access to Visual Studio, they will quite likely discover that they cannot use many of the context-sensitive tools that the IDE provides. Also, obviously, anyone can look at the code for a runtime compiled script, so deployment of uncompiled code containing trade secrets to a web server may be a security risk that some are not willing to take. Likewise, if plug-ins are to be sold by third parties, uncompiled scripts are really unsuitable.

If a plug-in system using dynamic assembly loading is the best solution, developers wishing to implement one have several hurdles to overcome first, not the least of which is poor documentation related to the classes and methods required to carry out the task. A fairly extensive knowledge of the inner workings of the .NET architecture is required to successfully build such a system. Unfortunately, there are precious few articles currently in existence on the subject, and the ones that do exist do not fully explain the subject. This article attempts to fill that void, as well as provide a solution for both dynamically loading plug-in assemblies and runtime compilation of plug-in scripts.

Major Obstacles

In .NET, assemblies that are loaded cannot be directly unloaded, and with good reason. There is no Assembly.Unload method. For plug-in systems, this means that without the aid of a secondary AppDomain, once a plug-in has been loaded, the entire application must be shut down and restarted in order for a new version of a plug-in to be reloaded. To address this problem, a secondary AppDomain must be created and all plug-ins have to be loaded into that AppDomain. Eric Gunnerson wrote a very excellent article on how to deal with dynamically loading assemblies into a secondary AppDomain, and quite a bit of the content of this article will be very similar in nature to his. However, his article doesn't quite accomplish what we require for a couple of reasons. First, his code (probably unintentionally) requires that plug-ins be placed into the same directory as the application itself. This is undesirable because most applications have other DLLs and class libraries that may be mistaken by end users as plug-ins if they reside at the same path. You wouldn't want to deal with support calls for someone who accidentally deleted a non-plug-in DLL. Beyond that, it's not really elegant. Second, his method for actually using plug-in code is extremely inflexible, and cannot really be easily reused and is useful only for demonstration purposes. Eric's article is an excellent (although ever so slightly inaccurate) starting point, but we need more.

So instead, we'll be looking at a method that extensively uses reflection to locate and make the plug-in types available to the main application code. Traditionally, plug-in managers have required plug-in authors to implement a specific interface (generally named something like IPlugin) and/or extend a specific class (generally MarshalByRefObject). This is all fine and good, and probably preferable for performance reasons, if your plug-ins only need to extend functionality in one area. However, if multiple sections of your code require plug-in extensibility, this is less than ideal because you would have to define multiple interfaces that the plug-in manager would have to be explicitly made aware of. Instead, by using reflection to retrieve class types that extend a specified type or implement a given interface, the plug-in manager does not need to know about the interface. Unfortunately, if one tries to access Type or Assembly objects directly from the primary AppDomain, .NET will make an unwanted attempt to load the plug-in into the main AppDomain, defeating the whole purpose for the secondary AppDomain -- the capability to unload the plug-ins. So instead, we need to write accessor methods on the RemoteLoader that take the fully qualified name of the class as an argument and perform the needed operation on the secondary AppDomain.

In order to allow loaded plug-ins to be overridden with newer versions, we must make use of a special feature called shadow copying. If you set up shadow copying for an AppDomain, instead of loading the assemblies within that AppDomain directly, .NET will automatically copy them to a cache folder first, and then load the assembly in the cache instead of the actual assembly. Then, the method for knowing when to reload the plug-ins after they've been changed involves using a FileSystemWatcher to trigger the correct event handlers. Unfortunately, the FileSystemWatcher has a tendency to trigger each event multiple times, which would result in the plug-ins being reloaded numerous times. In addition, if the assemblies are large enough and multiple assemblies are being copied, the reload might occur before all dependencies have been successfully delivered to the plug-in directory. In order to combat all of these problems, the reload will occur in a separate helper thread which triggers after the expiration of a 10 second timer which is initialized on an event from the FileSystemWatcher. If a new event is triggered during that period, the timer is reset to the beginning of the 10 seconds. This is probably overkill as most assemblies will not take a full 10 seconds to copy, but when dealing with something that's not theoretically guaranteed to work perfectly, it's better to err on the side of caution. Unfortunately, I know of no other possible method to deal with the copy delay problem.

The Code

The major issues mentioned above affect several places in the code, and we'll take a closer look at them now. First off, the construction of the secondary AppDomain turned out to be quite difficult, not because of the complexity of the code, but rather because the documentation on this functionality is quite thin. It's not immediately obvious, for instance, from the documentation that the PrivateBinPath property on an AppDomainSetup object requires a relative path, and that if you accidentally use an absolute path, you will encounter a variety of cryptic exceptions of type either FileNotFoundException or SerializationException. Once you realize this, though, it's not difficult to set up the secondary AppDomain and gain access to the RemoteLoader object.

/// <summary>

/// Creates the local loader class

/// </summary>

/// <param name="pluginDirectory">The plugin directory</param>

/// <param name="policyLevel">The security policy

///           level to set for the plugin AppDomain</param>

public LocalLoader(string pluginDirectory, PolicyLevel policyLevel)
{
    AppDomainSetup setup = new AppDomainSetup();
    setup.ApplicationName = "Plugins";
    setup.ApplicationBase = AppDomain.CurrentDomain.BaseDirectory;
    setup.PrivateBinPath = 
        Path.GetDirectoryName(pluginDirectory).Substring(
        Path.GetDirectoryName(pluginDirectory).LastIndexOf(
        Path.DirectorySeparatorChar) + 1);
    setup.CachePath = Path.Combine(pluginDirectory, 
        "cache" + Path.DirectorySeparatorChar);
    setup.ShadowCopyFiles = "true";
    setup.ShadowCopyDirectories = pluginDirectory;

    appDomain = AppDomain.CreateDomain(
        "Plugins", null, setup);
    if (policyLevel != null)
    {
        appDomain.SetAppDomainPolicy(policyLevel);
    }

    remoteLoader = (RemoteLoader)appDomain.CreateInstanceAndUnwrap(
        "PluginManager",
        "rapid.Plugins.RemoteLoader");
}

We set the ApplicationBase to the same value as the current AppDomain because we need access to the RemoteLoader type, which is defined in the same assembly as the PluginManager itself. The PrivateBinPath is relative to this directory, and cannot be set to a position lower than this directory through the use of paths like "..\where\ever". However, again, this is not obvious unless you go exploring through the fusion assembly binding logs.

/// <summary>

/// Initializes the plugin manager

/// </summary>

public void Start()
{
    started = true;
    if (autoReload)
    {
        fileSystemWatcher = new FileSystemWatcher(pluginDirectory);
        fileSystemWatcher.EnableRaisingEvents = true;
        fileSystemWatcher.Changed += new 
            FileSystemEventHandler(fileSystemWatcher_Changed);
        fileSystemWatcher.Deleted += new 
            FileSystemEventHandler(fileSystemWatcher_Changed);
        fileSystemWatcher.Created += new 
            FileSystemEventHandler(fileSystemWatcher_Changed);

        pluginReloadThread = new 
            Thread(new ThreadStart(this.ReloadThreadLoop));
        pluginReloadThread.Start();
    }
    ReloadPlugins();
}

We deal with the FileSystemWatcher as we set up the PluginManager here. We create and set up a looping thread that handles the delayed reloading of the plug-ins and then call ReloadPlugins() to initialize the PluginManager.

/// <summary>

/// Loads the assembly into the remote domain

/// </summary>

/// <param name="fullname">

/// The full filename of the assembly to load</param>

public void LoadAssembly(string fullname)
{
    string path = Path.GetDirectoryName(fullname);
    string filename = Path.GetFileNameWithoutExtension(fullname);

    Assembly assembly = Assembly.Load(filename);
    assemblyList.Add(assembly);
    foreach (Type loadedType in assembly.GetTypes())
    {
        typeList.Add(loadedType);
    }
}

This piece of code resides in the RemoteLoader class (which you'll note must extend MarshalByRefObject in order to cross the AppDomain boundary). Because the previous snippet of code created this instance of the RemoteLoader class within the secondary AppDomain, the call to Assembly.Load here loads the assembly into the second AppDomain instead of the first. We then load all of the types contained within this assembly into a list so as to speed up searching for any given type. Remember that the second AppDomain still contains many of the framework types. We're not dealing with a completely clean AppDomain here that we could efficiently iterate over using a doubly nested loop on AppDomain.CurrentDomain.GetAssemblies() and Assembly.GetTypes(). Also, you need to be aware that Type.GetType() will not work quite as expected. It will return types that have not been loaded via a dynamically loaded assembly. It's fine for pulling in types in a common library used by both AppDomains, but not for finding plug-in types. One more reason to keep a list of the loaded types.

/// <summary>

/// Returns a proxy to an instance of the specified plugin type

/// </summary>

/// <param name="typeName">The name of the type to create an instance of</param>

/// <param name="bindingFlags">The binding flags for the constructor</param>

/// <param name="constructorParams">The parameters to pass 

///                             to the constructor</param>

/// <returns>The constructed object</returns>

public MarshalByRefObject CreateInstance(string typeName, 
       BindingFlags bindingFlags, object[] constructorParams)
{
    Assembly owningAssembly = null;
    foreach (Assembly assembly in assemblyList)
    {
        if (assembly.GetType(typeName) != null)
        {
            owningAssembly = assembly;
        }
    }
    if (owningAssembly == null)
    {
        throw new InvalidOperationException("Could not find" + 
              " owning assembly for type " + typeName);
    }
    MarshalByRefObject createdInstance = 
        owningAssembly.CreateInstance(typeName, false, bindingFlags, null,
        constructorParams, null, null) as MarshalByRefObject;
    if (createdInstance == null)
    {
        throw new ArgumentException("typeName must specify" + 
             " a Type that derives from MarshalByRefObject");
    }
    return createdInstance;
}

Here, the RemoteLoader creates an instance of the specified plug-in type. First, we figure out which assembly owns the type. This is necessary for the same reason that Type.GetType() doesn't work as expected. Activator.CreateInstance can't find the plug-in types. It's fine for creating instances of the types within the common assemblies, but not dynamically loaded ones. So, we iterate over the plug-in assemblies and find the one containing the requested type. Then we simply make a call to Assembly.CreateInstance with the given BindingFlags and a set of parameters to pass to the constructor.

/// <summary>

/// Returns the value of a static property

/// </summary>

/// <param name="typeName">The type to retrieve

///      the static property value from</param>

/// <param name="propertyName">The name of the property to retrieve</param>

/// <returns>The value of the static property</returns>

public object GetStaticPropertyValue(string typeName, string propertyName)
{
    Type type = GetTypeByName(typeName);
    if (type == null)
    {
        throw new ArgumentException("Cannot find a type of name " + typeName +
            " within the plugins or the common library.");
    }
    return type.GetProperty(propertyName,
        BindingFlags.Public | BindingFlags.Static).GetValue(null, null);
}

Tragically, we have to write a custom method for accessing static properties and methods. It would have been nice if this could have been avoided. In any case, this code is pretty self-explanatory. It checks to make sure the type is indeed a plug-in type and then calls GetValue on the reflection PropertyInfo object. The CallStaticMethod implementation is also very similar.

/// <summary>

/// Generates an Assembly from a list of script filenames

/// </summary>

/// <param name="filenames">The filenames of the scripts</param>

/// <param name="references">Assembly references for the script</param>

/// <returns>The generated assembly</returns>

public Assembly CreateAssembly(IList filenames, IList references)
{
    string fileType = null;
    foreach (string filename in filenames)
    {
        string extension = Path.GetExtension(filename);
        if (fileType == null)
        {
            fileType = extension;
        }
        else if (fileType != extension)
        {
            throw new ArgumentException("All files" + 
                  " in the file list must be of the same type.");
        }
    }

    // ensure that compilerErrors is null

    compilerErrors = null;

    // Select the correct CodeDomProvider based on script file extension

    CodeDomProvider codeProvider = null;
    switch (fileType)
    {
        case ".cs":
            codeProvider = new Microsoft.CSharp.CSharpCodeProvider();
            break;
        case ".vb":
            codeProvider = new Microsoft.CSharp.CSharpCodeProvider();
            break;
        case ".js":
            codeProvider = new Microsoft.JScript.JScriptCodeProvider();
            break;
        default:
            throw new InvalidOperationException(
                "Script files must have a .cs, .vb," + 
                " or .js extension, for C#, Visual Basic.NET," + 
                " or JScript respectively.");
    }

    ICodeCompiler compiler = codeProvider.CreateCompiler();

    // Set compiler parameters

    CompilerParameters compilerParams = new CompilerParameters();
    compilerParams.CompilerOptions = "/target:library /optimize";
    compilerParams.GenerateExecutable = false;
    compilerParams.GenerateInMemory = true;
    compilerParams.IncludeDebugInformation = false;

    compilerParams.ReferencedAssemblies.Add("mscorlib.dll");
    compilerParams.ReferencedAssemblies.Add("System.dll");

    // Add custom references

    foreach (string reference in references)
    {
        if (!compilerParams.ReferencedAssemblies.Contains(reference))
        {
            compilerParams.ReferencedAssemblies.Add(reference);
        }
    }

    // Do the compilation

    CompilerResults results = compiler.CompileAssemblyFromFileBatch(
        compilerParams, 
        (string[])ArrayList.Adapter(filenames).ToArray(typeof(string)));

    // Do we have any compiler errors

    if (results.Errors.Count > 0)
    {
        compilerErrors = results.Errors;
        throw new Exception(
            "Compiler error(s) encountered" + 
            " and saved to AssemblyFactory.CompilerErrors");
    }

    Assembly createdAssembly = results.CompiledAssembly;
    return createdAssembly;
}

When batch compiling scripts, the first thing we need to do is ensure that all scripts are of the same type. Mixing and matching VB and C# in one Assembly is apparently impossible because you can only be using one CodeDomProvider at a time. So we check the extensions of each script and ensure that they are all the same.

Then we take the extension that is shared by all of the files and switch on it to choose the appropriate CodeDomProvider. Supported languages are C#, VB.NET, and JScript. We use that selected CodeDomProvider to generate our compiler. Then we set appropriate compiler options for loading a temporary assembly into memory. Obviously, this requires an output target of type "library". References are loaded from the list of DLLs that was passed in to the CreateAssembly method as a parameter. Actual compilation is done by the CompileAssemblyFromFileBatch method. Any compilation errors that have occurred will cause the errors to be saved and an exception to be thrown. This exception may either be caught or rethrown by the PluginManager depending on its settings. Assuming that compilation was successful, an Assembly is created and returned, and then processed and added to the list of managed assemblies by the RemoteLoader in much the same manner that the precompiled assemblies are.

Points of Interest

This was not as easy as it might look. This article was the culmination of something close to four weeks of rather frustrating research and fiddling. Lessons learned: wait until some other enterprising individual comes up with a solution when faced with a technical problem of this level of ridiculousness.

History

• 11/19/2004 - Version 1.0
• 11/24/2004 - Version 2.0 (Added runtime-compiled script support, polished code.)
• 1/14/2005 - Version 2.1 (Added some rudimentary support for security restrictions and fixed minor bugs.)