Dynamic interfaces in any .NET language

• Download source code - 106 KB

Introduction

In the Go programming language, you do not say explicitly that your type implements a given interface. Instead, a type is convertible to any interface, as long as it implements all the methods in the interface. This often reminds people of "duck typing" in dynamic languages such as Python or Ruby, but it is faster; in fact, Go interface calls are the same speed as virtual method calls in C++ and C#!

To put it in C# terms, if you have a class T...

public class T {
    public void Foo(int x);
}

...and an interface called "Interface"...

public interface Interface {
    void Foo(int x);
}

...then you can cast T to Interface even though T does not explicitly implement it.

Interface t = new T();

This cast can be implicit since the compiler can tell at compile time that T implements Interface. However, you can cast any object to Interface and, at run-time, Go will determine whether it implements Interface.

I've written a library modeled after this idea that I call GoInterfaces. The class T above can be "adapted" to Interface like so:

object t = new T();
Interface iface = GoInterface<Interface>.From(t);
iface.Foo(100);

Using GoInterfaces is just that simple, which is why I marked this article as being both "Beginner" and "Advanced". A beginner can use it, but understanding when to use it and how it works are more advanced topics.

Background

I recently wrote a wish list of features the .NET Framework should have, and I included Go interfaces on my list without actually knowing how they worked.

It bugged me, wondering how they worked. I searched the googlernet for awhile, and found nothing, so then I asked the "Go Nuts" Google group how Go method dispatch works, and was pointed to the article "Go Data Structures: Interfaces".

To summarize, the first time you convert a type T to an interface Interface, a vtable (virtual function table) is generated just like the kind used for virtual calls in .NET and C++. However, instead of storing the vtable in the object itself like C++ and .NET do, Go stores the vtable pointer alongside the interface pointer (i.e., an interface pointer is really two pointers). This simple but unique design allows a single object to implement an unlimited number of interfaces with an overall performance that is competitive with C# and Java.

Unfortunately, as far as I can tell, there is no way to efficiently implement this same technique in .NET without changing the CLR itself. A virtual method table is just a list of pointers to functions; importantly, function pointers in a virtual method table are not associated with a specific object, which makes them different from .NET delegates. By not associating the vtable with a specific object, it is possible to re-use the same vtable with any number of objects (as long as they are of the same class). However, .NET delegates are associated with specific objects, so we can't use them to form a reusable vtable.

Even if .NET allowed delegates that are not associated with a specific object, delegate invocation on .NET is slower than virtual method invocation; why this is so is not entirely clear to me, but part of the reason may be that Microsoft decided to make delegates reference types when they should have been a simpler 8-byte value type (just bundling a function pointer with a 'this' pointer).

Besides that, in order to get a reference to an interface, it is necessary to create an object on the heap, in addition to the object that you want to cast to the interface. The only way to support Go-style interfaces in .NET, while closely approximating the way Go itself works, is if the "interface type" is actually a value type (a 2-word structure). But if I took that approach, defining "interfaces" would be cumbersome, and they wouldn't work like normal .NET interfaces.

However, a week ago, I learned that Visual Basic 9 has a very similar feature to Go called "dynamic interfaces", which lets you do roughly the same thing as Go interfaces (albeit only in Visual Basic). So far, I've heard nothing about how VB's dynamic interfaces work, but I got to thinking: how hard would it be to bring Go-style interfaces to all .NET languages, and would it be possible to get good performance?

The technique I chose doesn't have performance as good as you would get from Go, but in exchange for a small performance hit (which I believe to be unavoidable anyway), the GoInterface classes provide automatic interface adaptations that you can't get in Go itself. Specifically, my GoInterface classes can automatically do small type-conversion tasks like enlarging int to long, boxing value types, and allowing return type covariance (for instance, if the wrapped method returns a string, the Interface can return an object). And since GoInterface returns heap objects that actually implement the interface you ask for (rather than a weird 2-word structure like I was just talking about), it's very easy to use.

Although GoInterface is slower than interfaces in the real Go language, my design does allow it to be more flexible, and I took the liberty of adding various features so that it can adapt to minor differences between the interface and the target type.

How it works

The GoInterface classes use .NET Reflection.Emit to generate wrapper classes in a "dynamic assembly", basically a DLL that exists only in memory. Each wrapper class implements a single interface of your choosing, and forwards calls on that interface to an object of your choosing.

As I mentioned, given the types from above...

public class T {
    public void Foo(int x);
}
public interface Interface {
    void Foo(int x);
}

...you can use GoInterface to cast T to Interface like this:

Interface t = GoInterface<Interface>.From(new T());

The first time you "cast" a T to Interface, GoInterface generates a wrapper class such as the following on-the-fly:

public class T_46F3E18_46102A0 : Interface
{
    T _obj;
    public T_46F3E18_46102A0(T obj) { _obj = obj; }

    public void Foo(int x) { _obj.Foo(x); }

    public override string ToString() { return _obj.ToString(); }
    public override int GetHashCode() { return _obj.GetHashCode(); }
    public override bool Equals(object o) { return _obj.Equals(o); }
}

The hex numbers in the name of the type are simply handles to the interface and type being wrapped, in order to guarantee no name collisions occur when you are wrapping a lot of different classes with GoInterface.

The first cast, I'm sorry to say, is very slow, because generating classes at runtime is very slow (as is the Reflection necessary to choose the code to produce). But after the first cast, all future casts are quite fast, especially if you call GoInterface<Interface,T>.From() instead of just GoInterface<Interface>.From(). That's because after GoInterface<Interface,T> is fully initialized, all its From() method does is invoke a delegate that contains the following code:

delegate(T obj) { return new T_46F3E18_46102A0(obj); }

I won't explain in detail how GoInterface works, because simply covering all the GoInterface features is quite enough for one CodeProject article. The source code has a lot of comments, hopefully enough to document itself.

How to use GoInterfaces

You can create wrappers with either GoInterface<Interface> or GoInterface<Interface, T> (note the extra type argument T).

• Go<code>Interface<Interface> is intended for creating wrappers when you do not know the type of the object at compile time. For example, if you have a list of objects of unknown type and you want to cast them to an interface, use this one.
• GoInterface<Interface, T> creates wrappers when you already know the type of the object at compile time. This version assumes that T itself (and not some derived class!) contains the methods you want to call. GoInterface<Interface, T> has the disadvantage that it is unable to call methods in a derived class of T. For example, you should not use GoInterface<Interface, object> because the object class does not contain a Foo method.

If you're not sure which one to use, use GoInterface<Interface>. If you need to adapt a large number of objects to a single interface, you should use GoInterface<Interface, T> where possible, because it is slightly faster. GoInterface<Interface>, in contrast, has to examine each object it is given to find out its most derived type. However, this process is optimized so that an expensive analysis is only done once per derived type, after which only a hashtable lookup is required.

GoInterface does a lot of work up-front in exchange for fast interface dispatch later. The first time you use any pair of types (an interface type and a target type), it's very slow, but once you have a pointer to an interface, actually calling methods on it is quite fast (almost as fast as calling through a normal interface).

Overhead compared to Go

Compared to interfaces in the Go programming language, which have a 1-word overhead for every interface pointer (the vtable pointer, which is 4 bytes in 32-bit code), GoInterface wrappers normally have 3 words of overhead (2 words for the wrapper's object header and 1 word for a reference to the wrapped object). Also, GoInterface generated classes are much more costly to produce (since they involve run-time code generation), and will increase your program's startup time significantly (a benchmark comes with the code, which demonstrates this). This generated code also has a fixed memory overhead that no doubt dwarfs Go's implementation. However, once you are up-and-running with GoInterface wrappers, their performance is pretty good.

Note: GoInterface can create wrappers for value types (structures), not just classes. Such wrappers have the same memory overhead as boxed structures, which is one word less than wrappers for reference types.

I would also like to compare GoInterface's performance to VB 9.0's dynamic interfaces feature, but I haven't got around to it (I don't normally use Visual Basic).

So what are they good for?

The natural question is, why would you want this feature?

The main reason I see is that sometimes you can see a pattern between different classes (classes not under your control) that the original authors did not see or, for some reason, did not make explicit. For instance, COM components sometimes offer collection types, but do not implement IList<T> even though they provide the most important methods of IList<T>. Heck, even Microsoft doesn't always implement IList<T> on its own collection classes! Mysteriously, Windows Forms classes such as TreeNodeCollection, ListViewItemCollection, and ObjectCollection do not implement IList<T> or even IEnumerable<T> (so LINQ doesn't even work on them unless you use the Cast<TResult> extension method).

Personally, I am working on a project in which I have written several read-only list classes. Some of them have indexers and some are merely enumerable, but with a Count property. I don't want to go to the trouble of implementing all 13 methods and properties of IList<T> for each class; instead, I define simplified interfaces:

public interface IEnumerableCount<T> : IEnumerable<T>
{
    int Count { get; }
}
public interface ISimpleList<T> : IEnumerableCount<T> 
{
    T this[int index] { get; }
}

The only problem is, if I want to use a normal .NET collection class, it doesn't implement these interfaces. While writing a wrapper that converts IList<T> to ISimpleList<T> isn't too hard, it is pretty nice to have it done automatically.

If you don't yet know what you can do with GoInterface, give it some thought. I often don't know how to use a new programming language feature until I come upon a problem that the feature solves. If you find a good use for GoInterface, please leave a comment!

Features of GoInterface

Struct, Class, and Interface wrapping

The target class T can be a struct, a class, or an interface.

The Interface does not have to be a .NET interface; it can be an abstract class instead. In that case, GoInterface will produce a wrapper for every abstract method in the class. Using an abstract class enables you to define extra (non-abstract) methods that do not exist in the target object.

Simple implicit conversions

The basic GoInterface.From() methods that I introduced above only work if the target class, value type, or interface (T) is compatible with the interface type (Interface). By "compatible", I basically mean that all the methods of Interface must be present in T, have the same number of arguments, and have implicitly convertible types. If the Interface has the method:

object Method(string x, short y);

and the target type T has a similar method:

bool Method(object x, int y);

then, GoInterface can successfully forward the call. The rule-of-thumb is that GoInterface.From can only adapt Interface to T if the wrapper doesn't require any explicit casts:

object Method(string x, short y) { return _obj.Method(x, y); }

Here, string x can be implicitly converted to object, short y can be implicitly converted to int, and the bool return value can be implicitly boxed and returned as an object. GoInterface implements the C# rules of implicit conversion; so, for example, it cannot implicitly convert int to float, short to uint, or object to string. Also, GoInterface does not have support for user-defined conversion operators.

Additionally, GoInterface allows covariance and implicit (widening) numeric conversions on out parameters. For instance:

// Successful match:
void Method(out byte a, out string b);      // method in target type T
void Method(out uint a, out IComparable b); // method in the Interface

If a good matching method of T is not found for every method of Interface, GoInterface.From throws InvalidCastException. However, you can force any cast to succeed using the ForceFrom() method instead.

Forcing conversion

Currently, GoInterface is structured in such a way that it always generates a wrapper class even if the target type isn't really compatible with the interface. Still, if you use the From() method, it won't create an instance of the wrapper when T and Interface are not completely compatible. The GoInterface.ForceFrom methods, on the other hand, always produce a wrapper instance even if some methods (or all methods) are missing from T or could not be matched for some reason.

Although the ForceFrom cast succeeds, when you call a method that could not be found in T or had incompatible arguments (or return value), the wrapper throws a MissingMethodException. This behavior of letting the cast succeed, but failing when you try to invoke a missing method, is inspired by VB 9's new "dynamic interfaces" which behave the same way. GoInterface lets you choose whether you want to fail during the cast (by calling From()) or when calling a missing method (by calling ForceFrom()).

GoInterface can actually match up some methods that have different numbers of arguments, even though the normal From() method throws an exception in such cases. If you use ForceFrom, then it is possible to call this target method...

void Foo(int x, out int y, out int z);

...via this interface method:

void Foo(int x, string y);

GoInterface allows the last argument(s) of the interface to be dropped (in this case, string y) if they do not exist on the target method (but they must be input arguments). It also allows out arguments on the target method to be dropped if they do not exist in the interface (y and z). It allows mismatches between ref and input arguments. For example, a ref int argument can be matched up with an int argument, and vice versa. Finally, an out argument on the target method can be matched up with a ref argument on the interface; the input value supplied by the caller is discarded, but the ref parameter receives the value of the out parameter.

Remember, the basic From() method will throw an exception if these kinds of mismatches occur; if you want to drop parameters like this or have ref mismatches, you must call ForceFrom or the second overload of From().

The second From() method takes a CastOptions value as a second parameter. The following CastOptions exist:

• CastOptions.As - if the cast fails, From returns null instead of throwing an exception.
• CastOptions.AllowUnmatchedMethods - allows the cast to succeed even if some methods were not found, ambiguous, or had irreconcilable arguments.
• CastOptions.AllowRefMismatch - allows the cast to succeed even if there are ref mismatches (e.g., ref float being matched up with float).
• CastOptions.AllowMissingParams - allows the cast to succeed even if certain methods could only be matched by dropping parameters from the end of their argument lists.
• CastOptions.NoUnwrap - normally, GoInterface<Interface> will detect if the object you are casting is already wrapped, and unwraps it if so, so that it does not produce wrappers around other wrappers. This option suppresses that behavior. Of course, this issue doesn't exist in the Go language, since Go does not need to use wrappers.

The CastOptions only control whether the cast succeeds, not whether method calls on the wrapper succeed. If there is a ref mismatch on a certain method, for instance, you can always call that method if the cast succeeded; the wrapper will not throw an exception. The same wrapper class is produced whether you call From() or ForceFrom(), and the wrapper can't keep track of whether you want calls to succeed or not, as it doesn't store a list of CastOptions anywhere.

Note: the From method that takes CastOptions is a bit slower than the other two.

Default arguments

GoInterface supports target methods with optional parameters. For example, if the target method is:

void Sleep(int milliseconds, bool nightmares = false);

but the interface contains only:

void Sleep(int milliseconds);

GoInterface will insert the missing default argument in the wrapper. The default value must be a simple primitive constant or a literal string.

Decorator assistance with the GoDecoratorField attribute

After writing the basic functionality of GoInterface, I realized it could also serve as a handy way to help implement the Decorator pattern. A decorator is a class that wraps around some target class (usually sharing the same interface or base class) while modifying the functionality of the target. For instance, you could write a decorator for TextWriter that filters out curse words, perhaps replacing them with uplifting terms about rainbows and butterflies.

Writing decorators is sometimes inconvenient because you only want to modify the behavior of some functions while leaving others alone. Without GoInterface, you must always write a wrapper for every method, manually forwarding calls from the decorator to the target.

GoInterface can help by generating forwarding functions automatically.

The following example shows how to use GoInterface to help you make a decorator:

// A view of an IList in which the order of the elements is reversed.
// The test suite offers this example in full; this partial version
// just explains the concepts.
public abstract class ReverseView<T> : IList<T> 
{
    // Use the GoDecoratorField attribute so that GoInterface will access
    // the list through this field instead of creating a new field.
    // Important: the field must be "protected" or "public" and have 
    // exactly the right data type; otherwise, GoInterface will ignore 
    // it and create its own field in the generated class.
    [GoDecoratorField]
    protected IList<T> _list;

    // The derived class will init _list for you if you have a default 
    // constructor. If your constructor instead takes an IList<T> 
    // argument, you are expected to initialize _list yourself.
    protected ReverseView() { Debug.Assert(_list != null); }

    // The downside of using GoInterface to help you make decorators is 
    // that GoInterface creates a derived class that overrides abstract
    // methods in your own class, which means your class must be abstract,
    // and users can't write "new ReverseView"--instead you must provide
    // a static method like this one to create the wrapper.
    public static ReverseView<T> From(IList<T> list)
    {
        return GoInterface<ReverseView<T>, IList<T>>.From(list);
    }

    // Here are two of several methods we need to rewrite in order to 
    // make a list appear reversed.
    public int IndexOf(T item)
    { 
        int i = _list.IndexOf(item); 
        return i == -1 ? -1 : Count - 1 - i;
    }
    public void Insert(int index, T item)
    {
        _list.Insert(Count - index, item);
    }

    // Here are the functions that we don't have to implement, which we
    // allow GoInterface to implement automatically. Unfortunately, when 
    // implementing an interface you can't simply leave out the functions 
    // you want to remain abstract. C#, at least, requires you to make a
    // list of the interface methods that you aren't implementing. This 
    // inconvenience is only when implementing an interface; if you are
    // just deriving from an abstract base class, you don't have to do 
    // this because the base class already did it.
    public abstract void Add(T item);
    public abstract void Clear();
    public abstract bool Contains(T item);
    public abstract void CopyTo(T[] array, int arrayIndex);
    public abstract int Count { get; }
    public abstract bool IsReadOnly { get; }
    public abstract bool Remove(T item);
    public abstract IEnumerator<T> GetEnumerator();

    // IEnumerable has two GetEnumerator functions so you must use an 
    // "explicit interface implementation" for the second one. In C#,
    // anyway, you must write the second one yourself, as it can't be
    // marked abstract.
    System.Collections.IEnumerator
    System.Collections.IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }
}

Overload resolution and ambiguity

When more than one method in the target matches an interface method, GoInterface can choose the "best" one. The rules for choosing the best match mostly mimic the C# standard, so it will generally work as you expect. Note that in some cases, the match is ambiguous - there is no best match. For example, if the interface is:

public interface IAmbig {
    void Ambig(string a, string b);
}

and you want to adapt the following class...

public class Ambig {
    void Ambig(string a, object b);
    void Ambig(object a, string b);
}

it doesn't work. Both methods match equally well, so neither is best and neither is chosen. You will get MissingMethodException when you try to call the method on IAmbig.

GoAlias for method renaming

Sometimes two parties pick different names for their methods. Just use the GoAlias attribute on your interface. For example, if a collection class uses silly method names like this:

public class MyCollection
{
     void Insert(object obj);
     int Size { get; }
     object GetAt(int i);
}

Stick GoAliases on the interface to switch those names to something more conventional:

public interface ISimpleList
{
    [GoAlias("Insert")] void Add(object item);
    
    int Count 
    {
        [GoAlias("get_Size")] get;
    }
    object this[int index]
    {
        [GoAlias("GetAt")] get;
    }
}

void Example()
{
    ISimpleList list = GoInterface<ISimpleList>.From(new MyCollection());
    list.Add(10); // calls MyCollection.Insert(10)
}

The GoAlias attribute is ignored if the target type has a matching method with the original method name. In this example, if the target type has an Add(object) method, then the alias Insert is ignored.

GoAlias supports multiple aliases (separate the names with commas).

GoInterface is case-sensitive, so you can also use GoAlias to work around differences in case.

If you do not control the interface, you can still use GoAlias. Make an abstract A class that implements the interface I, with an abstract method for each method of the interface. Then you can use GoAlias on the abstract methods. Be sure to use GoInterface<A> instead of GoInterface<I>.

System.Object method forwarding

GoInterface wrappers automatically forward calls to object.ToString(), object.GetHashCode(), and object.Equals(), even though these methods may not be part of the interface being wrapped.

.NET 2.0

Although written with some C# 3.0 code, GoInterface is still compatible with the .NET Framework 2.0.

Limitations of GoInterface

• Very important: both the interface and the target class must be public; GoInterface cannot handle internal types (note that internal is the default access level in C#). This problem arises because GoInterface produces a "dynamic assembly", a separate assembly from the one your code resides in. The CLR's security system prevents the dynamic assembly from using classes that are internal to another assembly. If anyone knows a way to bypass this restriction, please let me know!
• During call forwarding, GoInterface cannot use itself to convert arguments to other interfaces. For instance, suppose you make an ILength interface with a Length property. GoInterface cannot automatically convert a string argument in the interface to an ILength argument in the target.
• Currently, GoInterface has no specific generics support. Most importantly, GoInterface cannot handle interfaces that contain generic methods, and it will ignore generic methods on the target class. The class itself or the interface itself can have generic type parameters, but GoInterface does not produce generic code. Instead, GoInterface produces separate code for each specialization of a generic type. So, if you made wrappers involving List<string> and List<int>, GoInterface produces two separate wrapper classes.
• There is no code in GoInterface to support wrapping events.
• GoInterface cannot be used in the .NET Compact Framework or Silverlight, because those versions of .NET do not support Reflection.Emit.
• GoInterface cannot call user-defined implicit conversion operators. It can only convert smaller primitive types to strictly larger ones (like byte to int), and convert derived types to base types (like string to object).

A simple change that doubled the speed

After reading an article by Jon Skeet about beforefieldinit, I wondered if having a static constructor made a performance difference. It turns out that, heck yeah, it makes a big difference for GoInterface. By deleting this static constructor...

static GoInterface()
{
    _from = GenerateWrapperClassWhenUserCallsFrom;
    _forceFrom = GenerateWrapperClassWhenUserCallsForceFrom;
}

in favor of member-wise initialization...

private static GoWrapperCreator _forceFrom = GenerateWrapperClassWhenUserCallsForceFrom;
private static GoWrapperCreator _from = GenerateWrapperClassWhenUserCallsFrom;

the GoInterface<Interface> wrapper creation benchmark took 44% less time, while creating GoInterface<Interface,T> took 56% less time. I have therefore updated the screenshot and zip file.

One final note

Okay if I vent?

I learned a lot about Reflection and Reflection.Emit while writing this library, but the main thing I learned was that it sucks. The API smells really, really bad. The documentation is not well done, the design is clearly a mess, it's needlessly inefficient, and any code (including mine) that uses Reflection extensively is just plain ugly. I wish I was more familiar with the standard design patterns and principles, just so I could explain how badly they are being broken. Plus, I have some nitpicks about CIL. I hope Microsoft (or somebody) will someday write a much improved Reflection interface.

One thing that surprised me was that there are special Type objects for "ref" (and "out") parameters. At first, I assumed that ParameterInfo.IsIn and ParameterInfo.IsOut would indicate this, but it turns out that IsIn is always false, and IsOut is used only for "out" parameters and not "ref" parameters (even though ref implies both input and output). "ref" and "out" parameters get the same Type object (you can tell the difference based on ParameterInfo.IsOut or ParameterInfo.Attributes & ParameterAttributes.Out). Given a "ref" or "out" Type, you can get the corresponding non-ref type by calling Type.GetElementType().

Enjoy this library! Let us know what uses you find!

History

• August 13, 2010: Improved speed by removing static constructor (v1.01).
• June 16, 2010: Initial release (v1.0).