File Manager Component

• Download source files - 21.5 Kb
• Download demo project - 23.5 Kb

Introduction

These days most relational databases feature some sort of BLOB data type ('BLOB' stands for Binary Large OBject). BLOBs are designed to store large chunks (> 64K) of binary data. BLOBs can be used in a variety of ways, but they are most frequently used to store the contents of external files. That's unfortunate, in my opinion, since databases (generally speaking) aren't a very good substitute for a file-system. Let's face it, if they were, then most desktop computers would use SQL-Server for a file-system instead of NTFS. There are lots of very good reasons not to use BLOBs in this way, most of them are somewhat technical, and most are beyond the scope of this article (not to mention my writing ability). So I'll weasel out, and not defend my position, and instead just state that I don't think BLOBs should be used to store files in a relational database.

So what's the alternative? This article outlines my approach to file management. What I am about to present is a .NET component that includes design time support, is thread-safe, and is specifically designed to store and manage large numbers of files. You can easily integrate this component with your database, allowing your database server to concentrate on database related stuff, and your file server to concentrate on file related stuff.

Background

I actually designed this component years ago, and implemented it as an ANSI C++ class for use in Windows 3.x and XENIX (anyone remember XENIX?). Over time, I ported the original code to various languages and operating systems, so the design is fairly well tested. I wrote this C# version in 2001, with an eye towards increasing my understanding of the .NET framework. Since then, I have made various bug fixes and improvements (My! how the framework has changed since then!). Admittedly, the component I'm presenting here is a simplified version of the one that I routinely use, but it is nonetheless quite capable.

Using the code

The main player is a class named StorageContainer. StorageContainer derives from System.ComponentModel.Component and contains the following public properties and methods:

Properties

• public StorageFileAttributes StorageAttributes
• public string RootPath
• public int MaxFilesPerDirectory
• public bool Enabled
• public ISynchronizeInvoke Owner

Methods

• public string Add(string sourceFileName, string metaTag)
• public string Add(Stream inputStream, string extension, string metaTag)
• public bool Delete(string managedFileName, string metaTag)
• public void LevelStorageTree()

Design time support is included for most of these properties. Here is what things look like within Visual Studio.NET:

The StorageAttributes property determines what attributes will be applied to files, when they are stored within the StorageContainer. For instance, choosing the Hidden element causes everything to be stored as hidden files, choosing ReadOnly causes everything to be stored as read-only files, etc. This property is available at design time as well as runtime.

The RootPath attribute determines where the StorageContainer will store its files. At runtime this property should always contain a valid path to a storage location. At design time it is permissible for this property to remain empty, since it isn't always possible to know what the value should be at runtime. This property is available at design time as well as runtime. Here is what the editor for this property looks like:

The MaxFilesPerDirectory property determines, how many files will be placed into any given sub-directory within the storage tree. For instance, setting this value to 10 would cause the StorageContainer to create a new sub-directory for every 10 files. This property is available at design time as well as runtime.

The Enabled property acts like the Enabled property for a GUI control. StorageContainer instances are disabled by default, so you must set the property to true at runtime, in order to add or remove files. This property is only available at runtime.

The Owner property is used in situations where a StorageContainer has been placed on a Form that has been wired to receive StorageContainer related events AND also contains code that will drive the StorageContainer from multiple threads at runtime. This is required by the framework since .NET has a tendency to become unsettled whenever the main thread of an application (the GUI thread) receives events from some other thread. Most of the time I don't use this property, since I don't write many GUI applications. You may need to be concerned with this if you write multi-threaded GUI applications. In that case, simply assign the Owner property to your form, like this:

private void MainForm_Load(object sender, System.EventArgs e)
{
    m_storageContainer.Owner = this;
}

The Add method is overloaded, and is used to add stuff to the StorageContainer with files. You can use this method to add files directly, or add the contents of a stream instead. The metaData parameter isn't used by the StorageContainer, but is instead passed back out, along with some other information, whenever the StorageContainer fires the FileAdded event. (I'll cover events soon.) When this method is called, a managed file is created within the StorageContainer and the contents of either the source file, or the source stream, are then copied into that managed file. The original file is not modified in any way.

The Delete method is used to remove files from the StorageContainer. Once a file is removed from storage, it cannot be recovered - short of using a file recovery utility. Note that the managedFileName parameter is not the original name of the file (which was not retained), but is the name of the managed file instead. Remembering to always use the correct name will save you some head scratching down the road. The metaData parameter is used in the same way that is is in the Add methods.

The LevelStorageTree method is used to fill "holes" left in directories by the removal of managed files. Over time, if the number of files added and removed is large (really large), then some directories may contain something less than whatever the MaxFilesPerDirectory property is set to. This method will iterate over all the directories in the StorageContainer, and move files around as needed in order to ensure that all the directories are full. As you can imagine, this method can run for awhile (a few seconds anyway), so it is best to call it during slow periods. Play around with it, and use your own discretion.

Events

StorageContainer is capable of firing the following events:

• FileAdded
• FileDeleting
• FileDeleted
• FileMoved

The FileAdded event is fired whenever a new file (or stream) is added to a StorageContainer. The event is fired after the file is added. The event includes arguments that contain the metaData associated with the original file, the FileInfo for the original file, and the FileInfo for the managed file. Note that the StorageContainer does not manage any of this information internally. If you want to know the details of the original file, you must save it in response to this event!

The FileDeleting event is fired before any managed file is deleted from a StorageContainer. This event gives the receiver the ability to cancel the delete operation. The event includes arguments that contain the metaData associated with the original file and the FileInfo for the managed file. The event argument also includes a cancel flag.

The FileDeleted event is fired whenever a file is removed from a StorageContainer. The event is fired after the managed file is deleted. The event includes arguments that contain the metaData associated with the original file and the FileInfo for the managed file.

The FileMoved event is fired whenever a file is moved within a StorageContainer. The event is fired after the managed file is moved. The event includes arguments that contain the FileInfo for the managed file before and after it was moved. Note that this event is only fired during the LevelStorageTree method.

Great, but how do I use it?

Assuming you want to create a GUI application, start by adding the StorageContainer component to your Visual Studio.NET environment by right clicking over the design toolbox, and selecting the "Customize Toolbox" menu choice. This produces the "Customize Toolbox" dialog. Next, select the ".NET Framework Components" tab, and then click on the "Browse" button. Browse to the PrivateStorage.DLL file on your computer, and select OK. This causes Visual Studio.NET to add the StorageContainer component to your design toolbox.

Once the StorageContainer component is part of your toolbox, you can work with it in the same way that you do with any other component.

Points of interest

The neatest aspect of this component is the way that it manages the creation of the various sub-directories. Because there is no practical limit on the number of files that can be stored within a StorageContainer, the number of directories has the potential to grow quite large over time. So, the first thing I had to do was figure out how to generate directory names that would be unique - without creating a path the size of Texas. The solution I found was to increment the directory names from 'A' to 'Z', one level at a time, moving down a level as needed. In other words, the sub-directories in a StorageContainer get created in the following fashion:

'A' through 'Z' created, then 'A\A' through 'A\Z', then 'B\A' through 'B\Z', continuing on to 'Z\Z', then moving to 'A\A\A', 'A\A\B', 'A\A\C', etc.

The code to produce this never-ending path is shown here:

private string _IncrementDirectoryName(string curDirectory)
{

    // Ensure the path is terminated with a backslash.
    if (!curDirectory.EndsWith(Path.DirectorySeparatorChar.ToString()))
      curDirectory += Path.DirectorySeparatorChar;
    StringBuilder sb = new StringBuilder(curDirectory);
    bool af = true;

    // Loop and increment.
    for (int pos = sb.Length - 1; pos > 0; pos -= 2)
    {
        // Should we reset the current "digit"?
        if (sb[pos - 1] == 'Z')
        {
            sb[pos - 1] = 'A';
            continue;
        } // End if
        // Increment the current "digit".
        sb[pos - 1]++;
        af = false;
        break;
    } // End for.
    // Should we append a new "digit" to the path?
    if (af == true)
        sb.Append(@"A" + Path.DirectorySeparatorChar);
    // Return the next directory name.
    return sb.ToString();
}

Another interesting detail is that the StorageAttributes property has a type of StorageFileAttributes. This enumeration is nothing more than a subset of the FileAttributes enumeration, with some of the non-file related elements removed. Here is a listing:

[Flags]
[Serializable]
public enum StorageFileAttributes
{
    Archive = FileAttributes.Archive,
    Compressed = FileAttributes.Compressed,
    Encrypted = FileAttributes.Encrypted,
    Hidden = FileAttributes.Hidden,
    Normal = FileAttributes.Normal,
    NotContentIndexed = FileAttributes.NotContentIndexed,
    ReadOnly = FileAttributes.ReadOnly,
    System = FileAttributes.System
}

The reason I created a custom enumeration, rather than simply using System.IO.FileAttributes and masking out the unwanted parts, is that I wanted to display the property in the Visual Studio.NET designer without the other elements showing up. At first I thought that I could get by with using System.IO.FileAttributes by writing a custom System.Drawing.Design.UITypeEditor to filter the list of elements. That approach almost worked, except that I could still iterate through every element in the enumeration by double-clicking the property in the designer. That little missing detail is exactly the kind of thing that keeps me up at night, so to protect my sleep I created the StorageAttributes enumeration to solve the problem once and for all.

Next, I wanted the ability to select individual elements of the StorageAttributes property. Unfortunately, the default type editor applied to enumerated types in Visual Studio.NET doesn't do that, it only let's you select one value at a time! I poked around in the dark for a little while before stumbling into a fantastic solution by Stephen Toub on http://www.gotdotnet.com/. Inspired by Stephen, I created a custom type editor for my property, adding code to ensure that a developer could never unselect all the elements at once. In that scenario, the type editor selects the "Normal" element by default. I wont include a listing of this class here. If you are interested, you can either look up Stephen's original article, or look at my source code.

Finally, I wanted the ability the set the RootPath property visually, without forcing a developer to manually type in a path string. So I associated the System.Windows.Forms.Design.FolderNameEditor with the RootPath property by applying the following attribute to that property:

[Editor(typeof(FolderNameEditor), typeof(UITypeEditor))]

This causes Visual Studio.NET to display a Folder Browser dialog, whenever the user wants to edit the RootPath property. Much better than type long storage paths in, by hand!

About the demo

The demo illustrates a quick application of the StorageContainer. The buttons on the toolbar can be used to stuff a test file into the StorageContainer 1, 10, or 100 times. The Clear button clears the list box of entries, and deletes the associated storage tree on the hard drive. The FileAdd event is intercepted in the application as well, and is used to write file information to the listbox. In a real application, you could use the FileAdd event to store meta-data about your files in a database. Once again, I think that this approach is better than storing files internally using BLOBs.

So there you are. The code I have provided here is both functional as well as educational. Keep in mind that there are many other ways that this class can be used. For instance, I have derived from StorageContainer to create a file-based database component, a custom cache component, a source control component, a workflow component... Look things over and see what uses you can come up with!

Have Fun! :o)

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here