BlockingFileObserver - an IEnumerable<string> implementation

• Download source files - 20.5 KB

Introduction

Has it ever been the case that you need to monitor a directory for new files? It certainly has for me, whether it was for monitoring the BadMail directory of my IIS SMTP service, or for incoming files from some other process, and the list goes on.

Now, normally, I would set up my FileSystemWatcher, register for events and such, and happily block and wait for incoming files. However, at some point I found that approach tedious, and frankly ... boring. What I would like to do, is use a friendlier interface to such "functionality".

What I would ideally like, is an IEnumerable<string> instance, that I can enumerate over, do LINQ queries on, and generally have such a more versatile interface to work with.

What I would like, is this:

foreach(string fileName in BlockingFileObserver.Create
    (@"D:\projects", "*.txt", FileOperation.None | FileOperation.Create))
    Console.WriteLine("File Notification: {0}", fileName);

In the code above, I'm making a very interesting assumption. "My thread should block until every next item comes out of the enumerator." Well ... tough cookie.

This is what the little toy project included with this article is all about. It contains a class, BlockingFileObserver, that is an IEnumerable<string> subclass, and will handle monitoring file changes in a directory, and report them to you as they arrive, blocking until the next one comes along.

The class provided in the project goes a bit further than the code above. A BlockingFileObserver is very nice, until you need to shut it down. The syntax above will just keep going forever. The only way to stop the console app included would be Ctrl-C. So, I included another footprint that includes an out Timer and a long millis parameter, to indicate that we want the enumeration to stop after a certain time interval has elapsed. This syntax is illustrated below:

foreach(string fileName in BlockingFileObserver.Create
    (@"D:\projects", "*.txt", FileOperation.None | FileOperation.Create, 
    out timer, 30000))
    Console.WriteLine("File Notification: {0}", fileName);

I did take some time to justify to myself why I needed the out Timer stopTimer parameter. I concluded that it was the only way to expose the Timer's event to the caller, in case it also required notification of the timeout. Whenever I would unleash some while(true) { // block execution ... } code out on the wild, I would like to be informed when and if it stops running.

There are in total four factory methods on the BlockingFileObserver, exposing most possible parameter combinations:

public static BlockingFileObserver Create(string folderPath);
public static BlockingFileObserver Create
    (string folderPath, string fileTypeFilter);
public static BlockingFileObserver Create
    (String folderPath, FileOperation opType);
public static BlockingFileObserver Create
    (String folderPath, string fileTypeFilter, FileOperation opType);
public static BlockingFileObserver Create
    (String folderPath, string fileTypeFilter, FileOperation opType, 
    out Timer stopTimer, long millis);   

FileOperation is an enum type that is included with the project, defined like this:

/// <summary>
/// Defines the three basic file type operations, and 
/// one more, None, meaning an unspecified value
/// </summary>
[Flags]
public enum FileOperation 
{ 
    Create = 1,
    Delete = 2,
    Change = 4,
    None = 8
}

Since it is decorated with the [Flags], its values can be '|'-ed together, like FileOperation.None | FileOperation.Delete and so on. This gives an extra touch of flexibility to the interface, allowing you to specify the types of notifications you're interested in using a very familiar syntax - if not just a very pretty one.

Background

First and foremost, we owe the ability to create such enumerations to the yield keyword. In simple terms, whenever used in a method or property getter, yield allows the flow of execution to return to the caller, and when it requests the next element in the enumerator, execution will continue right after your last yield return statement. Perhaps this is simpler to illustrate, than to describe. This is how yield is being used inside the BlockingFileObserver:

public IEnumerator<string> GetEnumerator() 
{
    // ok, do the existing files first, 
    // but only if we've .None in our FileOperation ... 
    if ((FileOperation.None & this.FileOperation) > 0) 
    {
        foreach (string fileName in Directory.GetFiles
            (this.FolderPath, this.FilterString))
        yield return fileName;
    }          
            ...
}
</string>

Each time we hit the yield return statement, execution continues in the caller's side. Immediately after that, execution will continue where it left off here. What this allows me to do is to now set up a FileSystemWatcher, register for the events I need, and block execution of my thread until I receive an event notification.

In order to block until I receive an event, I had to do a little digging and poking around the System.Threading namespace. In there, I found the ManualResetEvent class.

The ManualResetEvent has two states: Signalled, or Not Signalled. The useful thing about it is that if you call .WaitOne() on it, your next line of code after that will get executed whenever someone else sets the ManualResetEvent's state to Signalled. You're blocking, until signalled to continue. I use that to block until I receive a file notification event from my internal FileSystemWatcher. This looks something like this (these are the lines immediately after the previous code snippet, still inside the GetEnumerator() method of the BlockingFileObserver):

// ok, now I need to create a FileSystemWatcher here ... 
m_Watcher = new FileSystemWatcher(this.FolderPath, this.FilterString);
            
// Init the watcher, register for events etc. 
... 
            
// Do I need to keep listening ?
if (0 == keepListening)
    yield break;

// start the watcher ... 
m_Watcher.EnableRaisingEvents = true;

// Now I need to start waiting for events ... 
while (true)
{
    m_ResetEvent.WaitOne();
    m_ResetEvent.Reset();
    
    if (this.Active)
        yield return m_LastFile;
    else 
    {
        yield break;
        break;
    }
}

The 'magic' here is inside the while(true) loop. Immediately when we enter the loop, we block, until the ManualResetEvent gets signalled. We reset it, so it returns to the non-signalled state, and proceed to return the next available value, or a yield break to exit the loop.

The event gets Signalled, in the event handler code for the FileSystemWatcher:

#region FileSystemWatcher & eventing fields & methods
/// <summary>
/// Will observe our folder for new/deleted/updated files and notify us ... 
/// </summary>
private FileSystemWatcher m_Watcher = null;

/// <summary>
/// Will help us block execution until the FileSystemWatcher 
/// posts a new event ... 
/// </summary>
private ManualResetEvent m_ResetEvent = new ManualResetEvent(false);

/// <summary>
///  Little "buffer" variable, to hold the last returned file name
/// </summary>
private string m_LastFile = string.Empty;

private void Watcher_PostEvent(object sender, FileSystemEventArgs e) 
{ 
    // ok, store the file name ... 
    m_LastFile = e.Name;
    // post an event ... 
    m_ResetEvent.Set();
}

#endregion

The ManualResetEvent member also gets Signalled in the set part of the Active property. This is required, in conjunction with the 'particular' syntax of my while loop in the GetEnumerator() method, to ensure that I won't find myself in a case where my Active property is set to false, but I'm still waiting for one last FileSystemWatcher event before I exit.

private bool m_Active = true;

public bool Active
{
    get 
    { 
        return m_Active; 
    }
    set 
    { 
        m_Active = value;
        // In case I'm blocking right now ... notify
        if (!m_Active)
            m_ResetEvent.Set();
    }
}

Using the Code

The class is really so small, that I didn't think it was any use putting it on a separate ClassLibrary project, so it's included in the one and only .cs file in the project, Program.cs. I've carefully defined regions in there, so getting it out with a simple copy-paste shouldn't be too much trouble.

Other than that, using the class is pretty straightforward. Use one of the four available .Create(...) static methods, to get back an instance that'll behave just like any other IEnumerable<string>:

class Program
{
    static void Main(string[] args)
    {
        Timer timer;            
        // Enumerate ... 
        foreach(string fileName in BlockingFileObserver.Create
            (@"D:\projects", "*.txt", FileOperation.None | 
            FileOperation.Create, out timer, 30000))
        Console.WriteLine("File Notification: {0}", fileName);

        Console.WriteLine("Finished enumerating");
        Console.ReadLine();
    }
}

Points of Interest

One last interesting tid-bit is the .Create implementation that also registers a timeout for the enumeration. In there, I instantiate the out Timer stopTimer param I've received, and feed it an anonymous delegate to handle it's CallBack. Interestingly enough, since at the time of creation I have a reference to the BlockingFileObserver instance I want to deactivate, I can use that variable inside the body of my anonymous delegate! I think this is just beautiful ...

public static BlockingFileObserver Create
    (String folderPath, string fileTypeFilter, FileOperation opType, 
    out Timer stopTimer, long millis)
{
    BlockingFileObserver enumerator = 
        Create(folderPath, fileTypeFilter, opType);
    // Create the new timer ... 
    stopTimer = new Timer
                (new TimerCallback(
                delegate(object sender)
                {
                    enumerator.Active = false;
                }),
                null,
                millis,
                Timeout.Infinite);
    return enumerator;
}

I'd think that this is a powerful programming paradigm for service-like 'listeners'. Hiding all the blocking complexity behind an IEnumerable<T> interface is so very versatile, but most of all in my opinion, it is so pretty.