Virtual Extension Methods

• Download source files - 120 Kb

Introduction

How many times have you faced a problem that would most naturally be solved by adding a virtual method to a class, but you can't add that virtual method because the class in question belongs to another assembly that you cannot alter? Well, that problem has confronted me for the last time, as I've written a class library that simulates writing virtual methods on foreign classes.

Background

Suppose you are using a prepackaged library that includes classes like this:

class Foo {}
class FooSub : Foo {}
class FooSub2 : Foo {}
class FooSubSub : FooSub {}

If you want to extend those classes, you could subclass them, but there are occasions where subclassing isn't possible or it doesn't really convey what you're trying to do. It's particularly problematic if you're coding an operation that will behave different depending on exactly what sort of Foo is passed in.

It's tempting to write code like this:

static class ExtendFoo
{
    static void MyOperation( Foo _this, ... )
    {
        if ( _this is FooSubSub )
        { ... }
        else if ( _this is FooSub )
        { ... }
        else if ( _this is FooSub2 )
        { ... }
        else
        { ... }
    }
}

But code like that is dangerous and difficult to maintain and exactly the reason language designers came up with the virtual keyword in the first place. But again, if the classes are prepackaged you won't be able to add a virtual method and you're stuck doing something like this.

This is not a new problem.  Other languages (notably Eiffel) support a concept called the Mixin, which allow you to write a class that extends another class without altering it. While Mixins are well known to be a powerful concept, implementing them is quite tricky and there are still areas of disagreement among language experts on exactly how they should work. Consequently, Mixins are not part of mainstream languages yet and there are no immediate plans to implement them in any of the main .Net languages.

But the lack of support for language features has never fully deterred people from concocting ways to simulate them, and this library is just another in that tradition.  Language experts might well point out that this is really not a simulation of Mixins in total, rather just one little aspect of them.  So the usage of the term "Mixin" in this article is probably flawed, but we have to call it something.  Probably a more apt term would be Virtual Extension Methods.

Using the code

Basic Usage

This example shows how you can use the Virtual Mixin library to add a virtual method to some prepackaged classes:

using System;
using System.Collections.Generic;
using System.Text;
using TallTree.Mixins;

namespace Sample
{
    class Foo { }
    class FooSub : Foo { }
    class FooSub2 : Foo { }
    class FooSubSub : FooSub { }

    class Program
    {
        static void Main( string[] args )
        {
            Console.WriteLine( MyOperation.Hello( new Foo(), "lonely" ) );
            Console.WriteLine( MyOperation.Hello( new FooSubSub(), "ornery" ) );
            Console.WriteLine( MyOperation.Hello( new FooSub2(), "silly" ) );
        }
    }

    class MyOperation
    {
        [MixinDeclaration]
        public static string Hello( Foo _this, string adjective )
        {
            return (string)Mixins.Invoke( _this, adjective );
        }

        [MixinImplementation]
        static string _Hello( Foo _this, string adjective )
        {
            return "Just a " + adjective + " Foo";
        }

        [MixinImplementation]
        static string _Hello( FooSub _this, string adjective )
        {
            return "Just a " + adjective + " FooSub";
        }

        [MixinImplementation]
        static string _Hello( FooSub2 _this, string adjective )
        {
            return "Just a " + adjective + " FooSub2";
        }

        [MixinImplementation]
        static string _Hello( FooSubSub _this, string adjective )
        {
            return "Just a " + adjective + " FooSubSub";
        }
    }
}

The Output is:

Just a lonely Foo
Just a ornery FooSubSub
Just a silly FooSub2

In this sample, MyOperation.Hello is the virtual extension method. You give it an instance of Foo and it runs the implementation of MyOperation._Hello appropriate for the actual type of the instance

At this point, you're probably looking at the sample and saying, "Okay, I think I get how I would use this library, but it sure looks like the sample is broken! Shouldn't there be an argument or something in the implementation of MyOperation.Hello that would give Mixins.Invoke some sort of clue what mixin to run?"

No, the sample isn't broken. Mixins.Invoke accesses the stack frame to figure out what method called it, and from there, it can grab onto the [MixinDeclaration] attribute. It then scours all loaded assemblies for static methods with the [MixinImplementation] attribute. Once it's done this, it can simply look at the type of _this and figure out which one of the implementations to use.

Error Checking

With virtual methods, if you make a mistake like changing the method signatures of some, but not all of the implementations, you'll get an error from the compiler. With this system, you won't get an error from the compiler.  Instead, an exception will be thrown during the initialization process where all loaded assemblies are scanned. The error checking that's done during the initialization process is nearly the same as the compiler performs, as such, it's best to think of it as a post-compilation error check. The best practice is to call the Mixin initialization routine as early in the execution as you can. So our example would be more properly written as:

static void Main( string[] args )
{
    Mixins.Initialize();

    Console.WriteLine( MyOperation.Hello( new Foo(), "lonely" ) );
    Console.WriteLine( MyOperation.Hello( new FooSubSub(), "ornery" ) );
    Console.WriteLine( MyOperation.Hello( new FooSub2(), "silly" ) );
}

If you have unit tests that are run at the end of every build, Mixins.Initialize should certainly be called in at least one of the unit tests to validate the integrity of the Mixins.

Abstract Methods

You can make a mixin abstract by omitting the implementation for the base class(es). Just as with normal abstract methods, you will need to provide an implementation for every non-abstract subclass, otherwise the Mixin library will throw an IncompleteMixinSignature exception on initialization.

For example, if you alter the example above by deleting the base implementation of static string _Hello(Foo _this, string adjective) then you will get an error complaining that Hello has no implementation for Foo. But, if you then made Foo abstract, then the error would go away.

Mixins that Span Assemblies

In the example we have shown here, the entire Mixin signature is defined within one class. It's not necessary that this be so - implementations can be spread out over several classes, even spanning several assemblies. The example we gave above showed one of two best-practice techniques - in the above example, we grouped each operation into its own class. But if you have several methods that you want to add, you might do better to create "extension" classes for every class in the library. For example:

class ExtFoo
{
    [MixinDeclaration]
    public static string Hello( Foo _this, string adjective )
    {
        return (string)Mixins.Invoke( _this, adjective );
    }

    [MixinImplementation]
    static string _Hello( Foo _this, string adjective )
    {
        return "Just a " + adjective + " Foo";
    }
}

class ExtFooSub
{
    [MixinImplementation]
    static string _Hello( FooSub _this, string adjective )
    {
        return "Just a " + adjective + " FooSub";
    }
}

class ExtFooSub2
{
    [MixinImplementation]
    static string _Hello( FooSub2 _this, string adjective )
    {
        return "Just a " + adjective + " FooSub2";
    }
}

class ExtFooSubSub
{
    [MixinImplementation]
    static string _Hello( FooSubSub _this, string adjective )
    {
        return "Just a " + adjective + " FooSubSub";
    }
}

Mixins are related to each other based on their names and the type of their first argument.  Class assignment, and even assembly assignment don't matter.

Dynamically Loaded Assemblies

Assemblies can be loaded dynamically during the run and this poses a few challenges, particularly for abstract methods. What would happen if we kept Hello abstract and we dynamically load a new assembly that has a class that extends Foo but does not implement a Hello mixin? The answer is, we'd get an MixinNotImplemented exception the next time a Mixin is run.

The Mixin library can detect when new assemblies are loaded and analyzes these new assemblies the next time the Mixin library is used. You can call Mixins.Initialize() repeatedly to scan any newly-loaded assemblies. The sooner that errors are detected, the better, so it's best to call the initialization routine right after assemblies are loaded.

Calling Base-class Implementations

In C++, you could write virtual methods that called their base class' implementation like this:

class Super
{
    virtual void MyOperation()
    { ... }
};

class Sub : public Super
{
    virtual void MyOperation()
    {
        ...
        Super::MyOperation();
    }
};

While this does work, it's also unsafe - what if the classes are refactored and a class is introduced between Sub and Super that refines MyOperation? C# avoided that problem by introducing the base keyword:

class Super
{
    virtual void MyOperation()
    { ... }
}

class Sub : Super
{
    override void MyOperation()
    {
        ...
        base.MyOperation();
    }
}

With the Mixin library, it's tempting to directly call the base class in a manner similar to the C++, such as:

[MixinImplementation(MustOverride=true)]
static string _Hello( FooSubSub _this, string adjective )
{
    return "Just a " + adjective + " FooSubSub ("
         + _Hello( (FooSub)_this, adjective ) + ")";
}

But it's much better to use the Mixin Library's equivalent to the C# base syntax, like this:

[MixinImplementation(MustOverride=true)]
static string _Hello( FooSubSub _this, string adjective )
{
    return "Just a " + adjective + " FooSubSub (" + 
          (string)Mixins.InvokeBaseClassImplementation(_this,adjective) + ")";
}

Again, InvokeBaseClassImplementation is implemented by looking back up the stack frame to determine what implementation it was called from, and basing its decision on what implementation to call next accordingly.

If we call the base class, then you should inform the [MixinImplementation] attribute by setting its MustOverride flag to true. When you do this, the initialization code will verify that there is a base class implementation to call and thus give you better error detection.

Performance Tweaks

While run-time code like this can never perform as well as optimized compiler code, this library does run reasonably quickly and, in most cases, the overhead from the method calls will be far lower than the execution time of the method implementation.

There is one thing that you can do to improve performance; you can get the mixin declaration code to cache the method like this:

static MixinMethod _hello;
[MixinDeclaration]
public static string Hello( Foo _this, string adjective )
{
    if (_hello == null)
        _hello = Mixins.Find();
    return (string)_hello.Invoke( _this, adjective );
}

Writing the code this way increases your code size by a bit, but it removes a dictionary lookup and the stack-frame lookup that Mixins.Invoke was doing every time the method was called.

You can do the same sort of thing with base-class invocations:

static System.Reflection.MethodBase baseImpl;
[MixinImplementation(MustOverride=true)]
static string _Hello( FooSubSub _this, string adjective )
{
    if ( baseImpl == null )
        baseImpl = Mixins.FindBaseClassImplementation();
    return "Just a " + adjective + " FooSubSub "
         + "(" + baseImpl.Invoke( null, new Object[] { _this, adjective } ) + ")";
}

Mixins.Initialize() can take a measurable amount of time to run. You may be able to improve your application's apparent performance by causing initialization to be done in a separate thread. This would only work if it's likely that the user will not cause any mixins to be implemented for some time after the application starts. (The entire Mixin library is thread-safe, so there's no need for you to write any lock-out code to ensure that the initialization completes before a mixin is called.)

Advanced Language Features

Extension Methods

Visual Studio 3.0 is slated to support Extension Methods. Extension Methods, in short, allow you to write a static method like MyOperation.Hello and declare it in such a way that the compiler will make it appear to be a method on Foo. Unfortunately, extension methods are not slated to allow for polymorphism in 3.0. Fortunately, there is no reason why you couldn't use the Mixin library in conjunction with 3.0 extension methods and create the full appearance of polymorphic extension methods.

Generics

One thing this library does not support at the moment is generic arguments. I see no reason why it couldn't, I just haven't implemented it yet.

Points of Interest

There are a number of obscure techniques used to pull all this off.

• The source code for the Mixin library uses this attribute clause frequently:

  [MethodImpl( MethodImplOptions.NoInlining )]
  

  This tells the optimizer that it should never "in-line" the following method. That is, left to its own devices, the optimizer might improve the efficiency of a block of code by flattening out the code for a short function into the calling function. We need to suppress this behavior in a number of spots - most prominently, the methods that look back up the stack frame.

• Scanning the assemblies for errors is a potentially expensive operation and, one should hope, entirely pointless in release builds. Therefore, the Initialization routine does not do nearly as much error checking when it is called from an assembly that was not compiled for debug. The check is performed using a technique described here.

• This library detects that new domains have been loaded by registering with the AppDomain.AssemblyLoad event.

• The method that tells Mixins.Invoke what method called it is StackFrame.GetMethod.

History

• Initial release - 11/02/2006.