Fun with Dynamic Objects and MEF in C# 4.0 - A dynamic File System Wrapper

• Download source code - 661 KB

Introduction

This article is mainly about doing some fun stuff by scratching the surface of:

• The System.Dynamic namespace introduced in .NET 4.0
• The System.CompositionModel namespace (Managed Extensibility Framework or MEF - http://mef.codeplex.com/)

A brief note on C# 4.0 dynamic features

C# 4.0 introduced the dynamic keyword to support dynamic typing. If you assign an object to a dynamic type variable (like dynamic myvar=new MyObj()), all method calls, property invocations, and operator invocations on myvar will be delayed till runtime, and the compiler won't perform any type checks for myvar at compile time. So, if you do something like myvar.SomethingStupid();, it is valid at compile time, but invalid at runtime if the object you assigned to myvar doesn't have a SomethingStupid() method.

The System.Dynamic namespace has various classes for supporting dynamic programming, mainly the DynamicObject class from which you can derive your own classes to do runtime dispatching yourself.

You may also want to Read more on Duck Typing, and read a bit on the dynamic keyword and the DLR (Dynamic language runtime).

Back to this article.

The key objective of this article is to demonstrate the multi dimensional possibilities and exciting things we can experience with the dynamic capabilities of C# and .NET 4.0. My intention behind writing this article is to enable it as a learning material for implementing C# dynamic features - and the code attached is for reference regarding how the technique is implemented, rather than what it does (i.e., the functionality).

We’ll be having a couple of interesting objectives:

• Create a dynamic wrapper around the file system so that we can access files and directories as properties/members of a dynamic object.
• A way to attach custom methods and operators to our dynamic wrapper class and dispatch them to a plug-in sub system.

My intention behind writing this article is to enable it as a learning material for implementing C# dynamic features - and the code attached is for reference regarding how the technique is implemented. Hence, these objectives are from a learning perspective - and there may not be any significant advantage in accessing the file system via dynamic typing.

You may need to install VS2010 and .NET 4.0 beta to have a look into the code. Click here.

What We'll Achieve

Simply speaking, in the end of the day, here are a couple of things we’ll be able to do.

• Initialize a dynamically wrapped drive:

dynamic CDrive = new FileSystemStorageObject(@"c:\\");

• Create a sub directory named TestSub:

CDrive.CreateSubdirectory("TestSub");

• Magic - Create a file named File1.txt in the TestSub folder we just created:

using (var writer = CDrive.TestSub.File1.txt.CreateText())
{
    writer.WriteLine("some text in file1.txt");
}

• Magic – Wrapping properties with Get/Set methods. E.g., invoking the CreationTime property.

Console.WriteLine(CDrive.TestSub.File1.txt.GetCreationTime());

• More magic - Copy File1.txt to File2.txt using the >> operator:

var result = (CDrive.TestSub.File1.txt >> CDrive.TestSub.File2.txt);

• More magic - Another way of copying, but calling a method in FileInfo using our dynamic type as the parameter

CDrive.TestSub.File2.txt.CopyTo(CDrive.TestSub.File3.txt); 

• Delete the newly created folder:

CDrive.TestSub.Delete(true);

High Level View

Here is a quick high level view of the classes and interfaces involved:

• DynamicStorageObject – Inherited from thr System.Dynamic.DynamicObject class. Wraps the logic for loading plug-ins using MEF, and invokes a method or a property on a specific type.
• FileSystemStorageObject - A concrete implementation of DynamicStorageObject, for the File System.
• CommandLoader – Relies on MEF for loading methods and operator extensions for a specific type, by inspecting the metadata.
• IDynamicStorageCommand – The interface that all method and operator plug-ins should implement.

Also, these are the major methods in the System.Dynamic.DynamicObject class that we are overriding, in our inherited classes.

• TrySetMember- Provides the implementation of setting a member.
• TryGetMember-Provides the implementation of getting a member.
• TryInvokeMember- Provides the implementation of calling a member.
• GetDynamicMemberNames- Returns the enumeration of all dynamic member names.
• TryBinaryOperation - Provides the implementation of performing a binary operation.

Now, to the code.

Exposing Sub Folders and Files as Properties

In the above examples, you might have noticed that we are exposing the sub directories and files as properties of the dynamic object, like CDrive.TestSub.File2.txt. And we are supporting extensions too.

Let us have a quick look towards how this is handled. First, we return all dynamic member names, by overriding the GetDynamicMemberNames (look inside FileSystemStorageObject.cs).

public override IEnumerable<string> GetDynamicMemberNames()
{
    return Directory.GetFileSystemEntries(CurrentPath, Filter).AsEnumerable();
}

Now, as CDrive is a dynamic object of type FileSystemStorageObject, whenever a property is invoked, the TryGetMember method in the FileSystemStorageObject will be called by the Runtime.

Have a look at the TryGetMember method in the FileSystemStorageObject class. As mentioned earlier, this method will be called each time, whenever a property of a dynamic object is accessed. Our implementation of TryGetMember is a bit tricky, as we need to decide whether the returned object represents a valid file or folder (path), or just a dummy object (to deal with extensions like txt in File2.txt).

public override bool TryGetMember(GetMemberBinder binder, out object result)
{
    var path = Path.Combine(CurrentPath, binder.Name);

    if (TryGetQualifiedPath(binder.Name, path, out result))
        return true;
    else
    {
        result = new FileSystemStorageObject(CurrentPath, binder.Name, 
                                             Filter, this, true);
        return true;
    }
}

The method TryGetQualifiedPath is for returning a new FileSystemStorageObject instance with a qualified path; otherwise, if it is an extension, we’ll return a dummy FileSystemStorageObject with the dummy parameter as true. You might want to put a couple of break points here and debug a bit if you are more curious.

Dispatching Method Calls

Whenever a method call happens on a dynamic instance of FileSystemStorageObject, like CDrive.TestSub.File2.txt.CopyTo(..), behind the scenes, the runtime calls TryInvokeMember. Have a look at the TryInvokeMember method we have in the FileSystemStorageObject class.

public override bool TryInvokeMember(InvokeMemberBinder binder, 
                     object[] args, out object result)
{

    if (Commands.ContainsKey(binder.Name))
    {
        //Execute a custom command
        Commands[binder.Name].Execute(this,null, args, out result);
        return true;
    }          
    else if (Directory.Exists(CurrentPath) && StoreItemType!=ObjectType.Unspecified )
    {
        DirectoryInfo info = new DirectoryInfo(CurrentPath);
        if (TryInvokeMethodOrProperty(info, binder.Name, args, out result))
            return true;
    }
    else
    {
        //Treat as a file command
        FileInfo info = new FileInfo(ProjectedPath);
        if (TryInvokeMethodOrProperty(info, binder.Name, args, out result))
            return true;
    }

    throw new InvalidOperationException
        (string.Format(Properties.Settings.Default.ErrorInvalid, 
                       binder.Name, MemberName));
}

Though the code is self-explanatory, here is bit more explanation towards how we are handling the method calls on our dynamic object.

• If the method name is already present in any of the registered plug-ins, that plug-in will be invoked. You can see that the Commands property holds a collection of pre-loaded plug-ins.
• Else if the path is a directory, we try to invoke the method on the related DirectoryInfo object.
• Else (we’ll reach here if the current path is a file, or the current path doesn’t exist), we’ll treat the current path as a file, and try to invoke the method for the related FileInfo object.

The third step is what enables us to call methods on non-existing paths, like:

using (var writer = CDrive.TestSub.File1.txt.CreateText())
{
    writer.WriteLine("some text in file1.txt");
}

One more point of interest might be the TryInvokeMethodOrProperty method, defined by the base class DynamicStorageObject. If you’ve the driller (VS 2010), you might dig in the code to have a quick look.

As you might have thought, one obvious task of this method is to invoke the method given on a specified object – but more than that, it also enables you to call the properties of FileInfo and DirectoryInfo as Get/Set method calls, like CDrive.TestSub.GetCreationTime() for accessing the CreationTime property. This is because, based on our convention so far, CDrive.TestSub.CreationTime represents a folder or file name CreationTime in TestSub, instead of a property of TestSub.

Using MEF - Plugging-In Methods

Managed Extensibility Framework or MEF - http://mef.codeplex.com/, or the System.CompositionModel namespace as boring guys call it, is hot ‘n’ cool. Well, at the time of writing, it’s still under beta preview.

We are using MEF here to enable support for plugging in method calls and operators to our dynamic object. Here, I’m more or less explaining the usage of MEF with respect to the context of our application; you can learn more by going to the above URL.

Here is a minimal approach to create a plug in sub system using MEF.

Create a Contract for your Plug-ins to Implement

First of all, we are creating a contract for our plug-in subsystem. Have a look at the IDynamicStorageCommand interface, in the DynamicFun.Lib project.

Creating and ‘Exporting’ Plug-ins

Create a couple of plug-ins that implement this interface, and “export” them using the Export attribute, by providing the contract as the parameter, so that we can discover them later. For an example, have a look at the BackupOperation implementation in the DynamicFun.Commands project.

[Export(typeof(IDynamicStorageCommand))]
[ExportMetadata("Command", "Backup")]
[ExportMetadata("Type", typeof(FileSystemStorageObject))]
public class BackupOperation : IDynamicStorageCommand
{
    #region IDynamicStorageCommand Members

    public bool Execute(DynamicStorageObject caller, 
                object partner, object[] args, out object result)
    {
        result = null;
        var path = caller.CurrentPath;

        if (File.Exists(path) && !path.EndsWith(".backup", 
                              StringComparison.InvariantCultureIgnoreCase))
        {
            File.Copy(path, path + ".backup",true);
            return true;
        }
      
        return false;
    }

    #endregion
}

You may notice that we are also exporting some meta data. We’ll come to this later.

‘Importing’ Plug-ins

Obviously, we need to import these plug-ins so that we can use them later. We have the CommandLoader class for doing this job. Have a look at the constructor of the CommandLoader class.

[ImportMany]
private Lazy<IDynamicStorageCommand, IDictionary<string, 
                object>>[] loadedCommands { get; set; }

public CommandLoader(Type type,string filter) 
{                
    var catalog = new DirectoryCatalog(System.Environment.CurrentDirectory, filter);
    var container = new CompositionContainer(catalog);
    container.ComposeParts(this);
    _compatibleType = type;
}

The ImportMany attribute tells MEF that when this part is composed (resolved), MEF should import all the exported type instances to this collection, from the catalog (in this case, we are using a directory catalog for loading the plug-ins from assemblies). Behind the scenes, MEF will initialize the collection, and will create instances of exported types to add them to the collection.

Also, have a look at how we are initializing the CommandLoader. We are creating an instance of CommandLoader from the constructor of the DynamicStorageObject class.

if (CommandsCache == null)
    CommandsCache = new Dictionary<Type, Dictionary<object, IDynamicStorageCommand>>();

if (!CommandsCache.ContainsKey(this.GetType()))
{
    var loader = new CommandLoader(this.GetType(), "*.Commands.*");
    CommandsCache.Add(this.GetType(), loader.Commands);
}

The Commands property of the loader will return a dictionary of commands that are compatible with the current type. The key of the key value pair will be the “Command” parameter’s value (like “Backup”) we exported as our plug-in’s metadata (remember?). The value will be the actual plug-in of type IDynamicStorageCommand itself.

Type compatibility check to determine whether this command is compatible with the caller is done using the “Type” parameter’s value we exported as our plug-in’s metadata.

Dispatching a Method Call to a Plug-in

Now, when the user invokes a method like CDrive.TestSub.File1.Txt.Backup(), as discussed earlier, TryInvokeMember will get called by the runtime. Dispatching a method call, if it is available as a plug-in, is straightforward. Have a look at the TryInvokeMember method from where we are invoking the Execute method of the command, if the method name is there in the Commands dictionary.

if (Commands.ContainsKey(binder.Name))
{
    //Execute a custom command
    Commands[binder.Name].Execute(this,null, args, out result);
    return true;
}

Dispatching Operators

And finally, we’ll examine how we are dispatching operators, when applied on our dynamic object. You might have already seen how we are using the Right Shift operator to perform the copy operation, like:

var result = (CDrive.TestSub.File1.txt >> CDrive.TestSub.File2.txt);

Dispatching operator invocations to the plug in sub system is much like how we do the same for methods. However, the key point to note is, we are doing this from the TryBinaryOperation method in the DynamicStorageObject class. Also, at the time of exporting Metadata, we should specify the operator as the “Command” parameter’s value. For example, have a look at the CopyOperation.

[Export(typeof(IDynamicStorageCommand))]
[ExportMetadata("Command", System.Linq.Expressions.ExpressionType.RightShift)]
[ExportMetadata("Type", typeof(FileSystemStorageObject))]
public class CopyOperation : IDynamicStorageCommand
{
   //..
}

Conclusion

This article was just for introducing a couple of new framework features, and the objective is just to demonstrate them for the curious. So, in the implementation, you might have noticed that we have various functional limitations, like not supporting files or folders with spaces etc.

A couple of points to note:

1. The dynamic calls will be slower for the first call; the resolved call site will be JITed and cached if possible for all subsequent calls.
2. C#'s underlying type system has not changed in 4.0. As long as you are not using the dynamic keyword, you are still statically typed (i.e., the types are known for the compiler at compile time).
3. Error handling when you use dynamic features is a bit difficult and can't be very specific, as you don't know much about the foreign objects to which you dispatch the calls.

Also, in this post, I'm not discussing the scenarios where you can implement dynamic dispatching. However, a couple of interesting possibilities include using C# to manipulate the HTML DOM, having a fluent wrapper to access XML data islands etc :)

In future posts, I may write more on these topics. That is it for now.

• Follow me on Twitter, I tweet mostly about .NET stuff - http://twitter.com/amazedsaint
• Stay tuned to my blog http://amazedsaint.blogspot.com for more

Appendix - I

As I started getting lot of "Scared" comments (see below), I thought about adding this appendix :)

I want to re-emphasize that my intention behind writing this article is to enable it as a learning material to explain about the dynamic features available in C# - and it is certainly up to the discretion of a developer to decide when to use it.

Just listing down here a few points from the "New Features in C# 4.0" manual regarding scenarios where you may increasingly use it - [Get the complete doc here][^]:

"The major theme for C# 4.0 is dynamic programming. Increasingly, objects are “dynamic” in the sense that their structure and behavior is not captured by a static type, or at least not one that the compiler knows about when compiling your program. Some examples include:.

• Objects from dynamic programming languages, such as Python or Ruby
• COM objects accessed through IDispatch
• Ordinary .NET types accessed through Reflection
• Objects with changing structure, such as HTML DOM objects

Appendix - II

I've a second part for this, but as it is too short for a CodeProject article, I've decided to keep it as a blog post. You may love reading that as well. Read MyExpando Class - A minimal, implementation of System.Dynamic.ExpandoObject.

History

• Friday, Sep. 04, 2009 - Published.