DynamicObjects – Duck-Typing in .NET

• Download source code - 232.66 KB

Update

4^th November, 2010: Added support for generic-method definitions and automatic casts for ref and out parameters of different types

Background

I think anyone who programs for a long time starts to create "code-generators". They are the first step to dynamic objects, as the code will be generated from some type of information you already have, like, for example, a database. If you could simply generate and compile such code at run-time, it is already dynamic.

I must admit that for a long time I used code generators, I generated classes to access my database in a typed manner (using C++), in my final project at university (to create a Delphi Server Pages) and, even in .NET, my first version of Remoting used a C# generated code that I compiled using CodeDOM, at run-time.

.NET and Today's Dynamic

Today, we have the dynamic keyword in C#, allowing for real dynamic code. But I consider that a completely different type of dynamic.

In my opinion, a dynamic object is an object to act as another object transparently. The dynamic keyword makes everything almost transparent. The compiler must see the object as "dynamic" to generate a completely different code. If you cast a dynamic to "object", to then cast it to another type, even if the dynamic object supports such a cast, an exception will be thrown, because if the compiler does not see the object as dynamic, it will not use its dynamic capabilities.

I am not saying the dynamic keyword is bad. I am only saying it is not the exact type of dynamic I expected.

In my view, I want to create an object with some properties or methods at run-time, or I know "something" about the objects, but the compiler doesn't. But, why not tell the compiler what I expect, beforehand, and then have a "typed" object, where I can use intellisense, even if the final validation is only done at run-time?

That's what I tried to do.

My Solution for Dynamic Objects

My solution for dynamic objects is the use of interfaces, together with the run-time emitting of objects that implement such interfaces.

I must say that I even asked Microsoft to add support to add interfaces to objects at run-time. This is not possible, but what was always possible to do was to create objects that implement some interface and redirect the calls to another (compatible) object, which does not implement such interface. And so, I created a code to do that, which started with CodeDOM, but now is emitting code directly.

Some Concepts

Before continuing with my solution, I want to talk about some concepts, which I used to refactor my library and update this article. What I wanted to do, in my initial idea, was to add support to the "duck-typing" concept, which I understood wrong.

The duck-typing concept I found in Wikipedia is funny, and the examples made me believe that if I had a class with methods A, B and C, which does not implement any interface, but I was able to cast it to an interface with the exactly same methods, that would be duck-typing, but that was more a structural-typing, so before continuing, I will try to explain my view on .NET Typing, Structural Typing, Duck-Typing and, finally, "dynamic" typing.

.NET Typing

.NET is a strongly typed language. Such strong typing, allied with non-virtual calls benefits performance a lot. Everytime you create a new method or property without making them virtual, you are creating a method that has no overhead in its call. A virtual method has an overhead, but a small one. But, .NET is also a hierarchical typed language. One class can only inherit from one parent class. But, that sometimes creates a problem. Let's try to think in a situation:

Class Button already exists, it is a Control, can be seen in screen, but it does not have a Print method.

Class TextBox is in the same situation.

Then, I create a class named PrintableButton with Print() and PrintableTextBox with Print(). But, that does not solve my problem. I want to put a Button and a TextBox into a list of... something printable.

So, how can I solve the problem?

Ah... simple. I use interfaces. I can create the interface IPrintable, change PrintableButton and PrintableTextBox to implement such interface, and then I can use a list of IPrintables, and everything will work fine.

But, wait a moment. Button and TextBox everyone knows, they are part of .NET. But PrintableTextBox and PrintableButton are not, but if they are not my classes and I only have the compiled version of them? And, to make everything worst, they are sealed. I can't inherit from them. What can I do?

Well, I still can use IPrintable. But, then, I will need to create a "wrapper" class that implements IPrintable and redirects the call to PrintableButton.Print or to PrintableTextBox.Print.

That works, but imagine doing that for every combination (Printable, Secureable or anything that starts to become common)... that's a lot!

Structural Typing

In structural typing, the Printable problem is solved. To make it more complete, let's imagine that IPrintable requires not only an object to have the Print method, but also to have a property to set the DPI, position and dimensions of the Printable. I know, Controls already have Positioning and Dimensioning properties, but the interface does not requires them to be Controls.

In fact, I can create a new class from scratch. If I add the properties Left, Top, Width and Height and the Print method, it will be a "Printable" in structure. But, I will not make it a Printable (or IPrintable) and I will create a DLL with it and send such DLL to my clients. There is no problem, they will be able to cast any of my controls that have such characteristics to IPrintable, as they have a compatible structure. That's why that's called Structural Typing. The structure is OK, then the "cast" is OK.

Duck-Typing

The most confusing part of Duck-Typing were the examples that always show that both classes had the same methods. But, Duck Typing goes further.

To try to use the "Duck" example, I will try to create a Duck class:

public class Duck
{
  public void Walk();
  public void Swim();
  public void Quack();
}

As I read, if it walks like a Duck, it Swims like a Duck and it Quacks like a Duck, it's a Duck.

But, then, I pass a Person as a Duck. In real life, we know that a person is not a Duck. But, these were the examples I got:

• When asked to walk, the person walks imitating a duck.
• When asked to swim, the person swims imitating a duck.
• When asked to quack, the person imitated a duck quacking.

That was the example I found, but that's not what really happens.

In fact, the Person only has two methods:

• Walk();
• Swim();

And it does not have a Quack method (or action... I don't know if I must talk like a programmer or like a person). So, in Structural-Typing, a Person will never be a duck, because a person does not "Quack".

But, in duck-typing, he "is", but what happens is:

var duck = person;
duck.Walk();
// The person walks like a PERSON, not a duck.
duck.Swim();
// The person swims like a PERSON.
duck.Quack();
// And you have a "WHAT A F***" or, in programming terms, a NotSupportedException.

What's the Real Difference?

In Structural-Typing, if the structure (all methods, properties and event) are compatible, that's ok. In duck-typing, every conversion (or cast) is ok, even if all actions will result into an exception because the action is not supported.

Which one is better?

In well done interfaces, structural-typing. I am sure. But Microsoft added some unused properties and methods in some base interfaces. If I am not wrong, IList has methods that IList<T> does not have. Why? Because they are too specific. But, think, which one you would prefer? Supporting IList and IList<T> and throwing an exception when an invalid method is call (duck-typing) or not allowing an IList<T> to be an IList (structural typing). To be honest, I prefer structural typing, but I decided to allow both. That's why I have StructuralCaster and DuckCaster classes.

dynamic Keyword

The dynamic keyword is, in fact, more "duck-typing" that duck-typing itself. In duck-typing, I will cast my object to some type, even if it is incompatible. But such cast can already map everything that is valid and invalid, so we can still have some performance benefits from it. The dynamic keyword never considers the object to be "of some type". At compile-time, everything is allowed. At run-time, every method, property-get or set must be validated. I know DLR tries to cache some used paths, but it still has a validation per method instead of "per-class". The advantage? If you are only calling 1 or 2 methods dynamically, you don't need to declare an interface for them before.

So, returning to my solution, what is it capable of doing?

• If you ask to get a static class as an interface, it will return an "instance object" that will redirect all the calls to the static methods (if possible). As already said, Duck-Typing (DuckCaster) will always succeed, while StructuralCaster will verify if the class has compatible methods, properties, and events.
• If you ask to cast an object to a given interface. If it already implements that interface, a normal cast will be used. If it does not implement that interface, then the rules for duck-typing or structural typing will be used, depending only on the class, so DuckCaster and StructuralCaster, both have the Cast methods. I also added extension methods DuckCast and StructuralCast.
• If you ask to proxy some interfaces, they will have all calls redirected to a proxy object. This is the slowest one, as the MemberInfo of each call will be passed as parameters to the ProxyObject methods, but this gives you the opportunity to do whatever you want with the calls. I use this in my Remoting framework, but you can use this to log the actions before executing them, to verify some attributes of a property or method, as a security layer, and so on.

In my opinion, these objects help complement the dynamic aspects of the language, as you can establish some rules for them into an interface that you use everywhere, even if they don't implement such an interface.

As I said in the first two items, if the methods and properties are compatible, they will be valid. With compatible, I mean: A read-only property can return a sub-type and it will be OK. A method can receive more "generic" parameters (like objects instead of strings) and it will be valid. (I don't know if structural typing allows this, but I do the casts necessary, and say that is OK, only generating errors on impossible casts).

Also, for the last one, you can give it a list of interfaces to implement. So, if you cast your object from one interface to another, it will still be able to be cast back, as it will be a single object that implements all of them.

And, to do something "more". By default, using DuckCast or StructuralCast, you can always recast to the original object. Doing another Duck or StructuralCast will do it on the real object (maybe capable of doing it), not on the fake object created, so you can use all the interfaces available. But, you can pass a parameter telling that such thing is NOT allowed. Why? Security. Before I used a ReadOnlyDictionary class that I implemented by hand. Today, it is an interface, which only the read-only methods, and the AsReadOnly (extension method added to the Dictionary class) will tell that a re-cast is invalid, so you can't get the original dictionary back to have write-access to it. That's a good thing, isn't it?

Using the Code

I will not try to explain every line of code. The concept is simple. An object will be emitted for the interface and redirects the calls, trying to do any necessary cast, box or unboxing operation. But emitting code in assembler is hard, and I am sure my actual code can be improved a lot to be made readable. I will only focus on how you can use the methods to create dynamic objects at run-time.

Case 1 - Making Types (static-methods) Implement an Interface

This is not exactly Duck-Typing or Structural-Typing, it is something that I think is missing. Delphi had the ability to declare static virtual methods. We don't have any direct way to do that, but if we analyze, all numeric types have the Parse method and MinValue and MaxValue constants. This is a rule and, so, I will show how you can threat the Type as a parseable type.

We simple declare the interface:

public interface IParseable
{
  object Parse(string str);
  object MinValue { get; }
  object MaxValue { get; }
}

The only thing this interface really says is: the class will need to have a Parse method, receiving a string and returning a value, and read-only MinValue and MaxValue properties or fields. In fact, we don't know the Type of the value returned, but there is no problem, as everything is compatible with object. Now, let's get int, short, and decimal types as IParseable interfaces.

IParseable parseableInt =
  StructuralCaster.GetStaticInterface<IParseable>(typeof(int));
IParseable parseableShort =
  StructuralCaster.GetStaticInterface<IParseable>(typeof(short));
IParseable parseableDecimal =
  StructuralCaster.GetStaticInterface<IParseable>(typeof(decimal));

Only to show some result, we have:

ShowParse(parseableInt, "57");
ShowParse(parseableShort, "58");
ShowParse(parseableDecimal, "57.58");
// note that this is affected by culture-information.

And ShowParse is implemented as:

private static void ShowParse(IParseable parseable, string value)
{
  object result = parseable.Parse(value);
  Console.WriteLine
  (
    "Parsed value " +
    result +
    " as type " +
    result.GetType().FullName +
    " MinValue: " +
    parseable.MinValue
  );
}

Case 2 - Getting Objects as an Interface They are Compatible With, But do not Implement

For this sample, I will create an IIndexed interface. I want to say that the object can be accessed by an index to return a value. The interface:

public interface IIndexed
{
  object this[int index] { get;}
}

And, to use it, I will create a Dictionary, a Hashtable, and a List. The Dictionary and the Hashtable have an indexer property of type object, but as an int is an object, there is no problem.

Dictionary<object, string> dictionary = new Dictionary<object, string>();
dictionary.Add(57, "Paulo");
dictionary.Add(200, "Zemek");
var indexed1 = StructuralCaster.Cast<IIndexed>(dictionary);

Hashtable hashtable = new Hashtable();
hashtable["FirstName"] = "Paulo";
hashtable[-57] = "Francisco";
hashtable["LastName"] = "Zemek";
var indexed2 = StructuralCaster.Cast<IIndexed>(hashtable);

List<string> list = new List<string>();
list.Add("Paulo");
list.Add("Francisco");
list.Add("Zemek");
var indexed3 = StructuralCaster.Cast<IIndexed>(list);

Console.WriteLine(indexed1[57]);
Console.WriteLine(indexed2[-57]);
Console.WriteLine(indexed3[2]);

I know the sample is very simple. It will in fact only show my name in the screen. But what I really want to show is that I am "casting" a List, a Dictionary, and a Hashset as an IIndexed interface, and this is supported because the types are compatible.

So, what is not valid?

Try casting a Dictionary<string, string> to IIndexed, and you will get an exception, because a string can't be assigned from an int, expected by the interface.

Case 3 - Redirecting to a Proxy Object

This is the most versatile solution, but also the slowest one. Instead of simple redirecting the method to compatible methods directly, all calls will be redirected to the Proxy object, passing the MemberInfo as a parameter, and the arguments as an array. But, even being slower, this allows you to see everything that happens, allowing you to block some methods, create pre and post processing, and so on. In this sample, I will only show the action requested and the parameters passed before doing the action.

I simple created a class named MyProxy, made it implement IProxyObject, and let Visual Studio implement the interface for me.

Then, I only filled the body of InvokeMethod.

public sealed class MyProxy: IProxyObject
{
  private object _realObject;
  public MyProxy(object realObject)
  {
    _realObject = realObject;
  }

  // In this sample I will only implement the InvokeMethod
  public object InvokeMethod(MethodInfo methodInfo,
                Type[] genericArguments, object[] parameters)
  {
    Console.WriteLine("Invoking method: {0} from {1}", methodInfo,
                      methodInfo.DeclaringType.FullName);

    var parameterInfos = methodInfo.GetParameters();
    int count = parameters.Length;
    for(int i=0; i<count; i++)
    {
      var info = parameterInfos[i];
      var value = parameters[i];

      Console.WriteLine("  {0} = {1}", info.Name, value);
    }

    object result = methodInfo.Invoke(_realObject, parameters);
    Console.WriteLine("Result: {0}", result);
    Console.WriteLine();
    return result;
  }

  // I will leave these unimplemented, as they are not needed for the example.
  public void InvokeEventAdd(EventInfo eventInfo, Delegate handler)
  {
    throw new NotImplementedException();
  }

  public void InvokeEventRemove(EventInfo eventInfo, Delegate handler)
  {
    throw new NotImplementedException();
  }

  public object InvokePropertyGet(PropertyInfo propertyInfo, object[] indexes)
  {
    throw new NotImplementedException();
  }

  public void InvokePropertySet(PropertyInfo propertyInfo,
              object[] indexes, object value)
  {
    throw new NotImplementedException();
  }
}

And, to test the object:

var myProxy = new MyProxy(dictionary);
var proxiedDictionary =
   InterfaceProxier.Proxy<IDictionary<object, string>>(myProxy);
proxiedDictionary.Add("My Full Name", "Paulo Francisco Zemek");

string result;
proxiedDictionary.TryGetValue("My Full Name", out result);

Those three situations were the ones I presented in my first version of the article, only showing how we can get Type static methods to implement an interface, an already instantiated object to implement an interface and how to redirect the calls to our proxy to decide what to do. But, that does not show us everything I tried to do when emitting code.

So, let's see the Duck-Typing and Structural-Typing in action.

// Case 4 - Differences between Duck-Typing and Structural-Typing.
Duck duck = new Duck();
UnknownAnimalThatQuacks unknown = new UnknownAnimalThatQuacks();
Person person = new Person();

DuckSample(duck);
DuckSample(unknown);
DuckSample(person);

StructuralSample(duck);
StructuralSample(unknown);
StructuralSample(person);

The DuckSample and StructuralSample are implemented as follows:

private static void DuckSample(object obj)
{
  Console.WriteLine("Starting duck-sample for " + obj.GetType().Name);
  IDuck duck = DuckCaster.Cast<IDuck>(obj);

  if (duck.GetType() == obj.GetType())
    Console.WriteLine("The cast did not create a wrapper for " + obj.GetType().Name);
  else
    Console.WriteLine("The cast created a wrapper for " + obj.GetType().Name);

  duck.Walk();
  duck.Swim();

  try
  {
    duck.Quack();
  }
  catch
  {
    Console.WriteLine("The Quack action was invalid for this object.");
  }

  Console.WriteLine();
}
private static void StructuralSample(object obj)
{
  Console.WriteLine("Starting structural-sample for " + obj.GetType().Name);

  IDuck duck;
  try
  {
    duck = StructuralCaster.Cast<IDuck>(obj);
  }
  catch
  {
    Console.WriteLine("The cast was invalid, so no action will be performed.");
    return;
  }

  if (duck.GetType() == obj.GetType())
    Console.WriteLine("The cast did not create a wrapper for " + obj.GetType().Name);
  else
    Console.WriteLine("The cast created a wrapper for " + obj.GetType().Name);

  duck.Walk();
  duck.Swim();
  duck.Quack();
  Console.WriteLine();
}

And, I will not lose time showing the animals. In fact, Duck and UnknownAnimalThatQuacks have the three methods, but only Duck is an IDuck (so if fact a normalcast will be used). Person does not has the Quack method, so it must not cast for Structural-Typing, but will cast for Duck-Typing, but will throw an exception if Quack is invoked.

Additional Features

The actual features shown are already very useable and can help anyone needing to cast objects created by someone else to a common interface created for a specific project. But, I started talking about dynamic and where do we need to use dynamic?

In my case, I created a project that could run using Firebird or SqlServerCe, and it is even capable of creating the database if it does not exist. The database connections use factories, so I don't need to have a direct reference for the SqlServerCe or Firebird DLL, but to create the database, I used direct references. My problem was that my clients needed to have the DLL for both databases, but it was obvious that they will only use one. I thought that dynamic could help me, but I still had nothing to help me with the fact I will need to use reflection to get to the class, will need to invoke the constructor using reflection to, only then, be able to use dynamic on the returned object.

In my solution, I can avoid one of those steps. After reaching the Type, I can use GetStaticInterface to get an object to the type. But, then, the problem is: SqlCeEngine has a constructor that receives a parameter. So, how do I tell an interface to call a constructor?

Well, I created the [CallConstructor] attribute exactly for that. In the interface that represent the static methods, marking a method with such attribute means that the call will be redirected to a call to the constructor. But, now, I have another problem, which type should such constructor return, if I don't have a reference to the DLL containing such type?

In this case, I could return an object, and then manually "Duck or Structure cast" the result to another interface, with the methods I need or receive such result as dynamic. But, if you want to use the interface, you can ask the result to be immediately be structure or duck-cast to such interface using [CastResult] attribute.

Finally, as it was the case for me, I also prepared the code to automatically convert enums to and from strings. That was needed in my project, because the FbDbType used an enum that was also only present in the library I didn't want to reference directly. This is something the dynamic keyword can't solve for me.

That's Almost All

I think now I have presented the concepts and at least explained a little the functionalities I tried to add. But, I said dynamic was slow, even the DLR trying to optimize it. So, I will now show that:

SpeedTestMySolutionAsObject();
SpeedTestMySolutionAsInt();
SpeedTestDynamic();

And the method are implemented as:

private const int RepeatCount = 10000000;
private static void SpeedTestMySolutionAsObject()
{
  Console.Write("e;Testing my solution, considering the data-type to be object: "e;);
  DateTime begin = DateTime.Now;
  var list = new List<int>().StructuralCast<IAddRemove<object>>();
  for (int i=0; i<RepeatCount; i++)
  {
    list.Add(i);
    list.Remove(i);
  }
  DateTime end = DateTime.Now;
  Console.WriteLine(end-begin);
}
private static void SpeedTestMySolutionAsInt()
{
  Console.Write("e;Testing my solution, considering the data-type to be int: "e;);
  DateTime begin = DateTime.Now;
  var list = new List<int>().StructuralCast<IAddRemove<int>>();
  for (int i=0; i<RepeatCount; i++)
  {
    list.Add(i);
    list.Remove(i);
  }
  DateTime end = DateTime.Now;
  Console.WriteLine(end-begin);
}
private static void SpeedTestDynamic()
{
  Console.Write("e;Testing dynamic solution: "e;);
  DateTime begin = DateTime.Now;
  dynamic list = new List<int>();
  for (int i=0; i<RepeatCount; i++)
  {
    list.Add(i);
    list.Remove(i);
  }
  DateTime end = DateTime.Now;
  Console.WriteLine(end-begin);
}

The results, in my computer, were:

Testing my solution, considering the data-type to be object: 00:00:01.5781250
Testing my solution, considering the data-type to be int: 00:00:00.8906250
Testing dynamic solution: 00:00:02.6718750

Why the Differences?

When using IAddRemove as object, the boxing and unboxing process must be done. When using it typed to int, such boxing and unboxing is avoided. But, even don't knowing exactly what dynamic did, I think it is not only boxing and unboxing values everytime, but it must also discover what path to follow everytime, while in my case this is decided during the cast operation.

A Final Note

In this article, I only wanted to show my vision of dynamic objects and present the utilization of my solution, which I consider to be a the nearest of "duck-typing" for .NET.

I don't consider this the "ultimate" solution, but I also think the ultimate solution will never be created, as making everything fully dynamic hurts the performance of the application.

Also, I consider that if you have access to the source code, you must make the classes implement all the interfaces they need, instead of asking the objects to be "converted" to some interface at run-time.

But, as we need to use components made from other persons, I consider this to be a valid addition, just as the dynamic keyword.

Demo

The demo project only includes the source-code of my personal library (which includes the source-code of the InterfaceImplementer and a lot of other things) and a project already compilable with all the samples presented in the article.