A Namespace Extension Toolkit

• Download source files - 210 KB
• Download Java demo project - 519 KB

Introduction

The Galaxy FilesSystem Toolkit (GFT) allows developers to create user-level file-system extensions that are viewable through Windows Explorer. While the toolkit itself is constructed using C++ COM/ATL code, extension developers can program in higher-level strongly-typed, garbage-collected languages such as C# and Java. Thus, the Toolkit eases the development of user-level file-systems on the Windows platform. The GFT has been used with a Java file-system extension to create a Windows Explorer interface to NFS version 2 fileservers. Other examples of possible extensions include:

1. Windows Explorer interface to Google’s Gmail, similar in functionality to Gmail drive. Gmail folders would be modeled as file-system folders, and mail messages would map to files. Alternatively, blocks within a file could be mapped to individual messages to allow for larger file storage.
2. A Windows Explorer interface to a SQL database. Tables could be modeled as folders, and rows could be modeled as files within folders.
3. A Windows Explorer interface to the PlanetLab. Folders would contain sites of machines, and files would contain attributes of individual machines, e.g., load averages.
4. A /proc-like interface using Windows Explorer. Files within the "/proc" folder would map to processes, and file contents would show process attributes.

1.1 What is it?

The Toolkit is a Windows Shell NameSpace Extension (NSE) which communicates to your custom code (the extension) via a proxy and stub pair. For a view of this, see Figure 1. The Toolkit manages a part of the Shell namespace, e.g., the Windows path "C:\Custom", and converts all Windows Explorer operations on this namespace into the appropriate method invocations on your file-system extension. For example, when a file is dragged-n-dropped onto the folder "C:\Custom", the Toolkit will first invoke CreateFile and then a series of Write calls on your extension.

The rest of this document describes the steps necessary to install the Toolkit and to create a simple extension. Section 2 covers the steps needed to install the Toolkit, while the bulk of the document, Section 3, will guide you through the creation of a sample extension in C#. Known issues and limitations are listed in Section 4, and Acknowledgements are given in Section 5.

2 Installation

To begin, download either the Toolkit source code or the binary installer package from the website. C# programmers will install from the source, since the source zip file contains the necessary interfaces and sample code. Java developers will typically install the GFT using the binary installer, since this procedure does not require Visual Studio.

2.1 Installing from the source

The GFT can be installed from a Visual Studio solution file and the source code, using the following steps:

1. Download: Download the latest GalaxySource.zip file from the CodeProject website.
2. Extract: Extract the source, and open the Galaxy.sln Visual Studio solution file.
3. Build and Start: Navigate to the Build menu in Visual Studio, and select Build Solution. Then, navigate to the Debug menu in Visual Studio, and select Start. This will cause the C# Mirror FileSystem to start (you should see a blank Command Prompt window). Now, go to Section 2.3.

2.2 Installing from the binary

Installing from the binary involves the following steps:

1. Download: Download the latest GalaxyBinary.zip binary installer from the website. This is a Zip file that can be extracted to a temporary location.
2. Run the setup.exe program. Open the setup.exe or GalaxyToolkit.msi install files. This will install the C++ namespace extension (NSE) and a C# mirroring file-system.
3. Start the Mirror FileSystem. Navigate to Start, Programs, Galaxy, and click on the "Start C# Mirror FileSystem" link. This step will not work on an older version of Windows, such as Win98, or any OS which does not have the .NET redistributable installed. Get the .NET 1.1 redistributable from the Microsoft website.

2.3 Exploring the FileSystem

At this point, you should have the Galaxy namespace extension (NSE) installed, and a sample C# Mirroring FileSystem extension running.

To explore this Mirror FileSystem, start a Windows Explorer process (Windows Key + E) and navigate to "My Computer". You should notice a new system folder called "Galaxy". Browsing to this folder should reveal two files: COPYING.txt and Subfolder. You can open the text file (the GNU GPL) and also open the Subfolder folder, as well as open the JPG file contained in the Subfolder folder. (These files are mirrors of the files contained in the folder, Test, which was distributed with the source or installed along with the binary, depending on your method of installation.)

You can try copying files to and from the Galaxy file-system, with the caveat that multiple selections are not yet supported. (In other words, you can copy entire folders, but you cannot highlight and copy two or more files at the same time.) Other caveats are listed in Section 4.

3 Tutorial

To create your own C# file-system extension, you will need to perform two steps:

1. implement a single abstract class, CSharpFileSystem.FileSystem and,
2. change line 50 of the file Driver.cs to specify your extension class name.

In this tutorial, we will create a dummy read-only file-system extension called "RandomFileSystem", which will display random text files which are each filled with random text.

First, startup Visual Studio (or your personal IDE) and open the CSharpFileSystem solution file. Now, create a new class in the CSharpFileSystem namespace, which inherits from CSharpFileSystem.FileSystem. We will call this new class "RandomFileSystem". Give the default constructor a single integer argument which specifies how many files to display per folder.

Your code should look like the following:

using System;
using System.Diagnostics;
using System.IO;
namespace CSharpFileSystem f /// <summary>

/// Summary description for RandomFileSystem.

/// </summary>


public class RandomFileSystem : FileSystem {

int myNumFiles;
public RandomFileSystem(int i) {
     myNumFiles=i;
        }
    }
}

Now, we need to implement the nine methods of the FileSystem abstract class. In each of the following sections, we will implement a method and discuss the implementation details. In the methods where we do not provide an implementation, we still return a success code of true; Galaxy’s error handling is not yet robust, and thus a false return value is likely to cause a failed assertion.

3.1 Command

This command is used for extensibility, and it allows the namespace extension to pass an arbitrary command string to your file-system. Currently, the only command that is passed is "trace", which gets passed when the user selects "Trace" from the context menu (by right clicking). We can safely ignore this, and so we give a dummy implementation:

public override bool Command(string command string) {
 if (command string.ToLower().Equals("trace")) {
    Debug.WriteLine("We should log our trace files to a log file here.");
    return true;
} else { 
    return false;
}

3.2 CopyFile

This method is used for a fast-path copy so that the Galaxy namespace extension can pass filenames (instead of file contents) when it wants to copy a file to your file system. Since this is an optimization, we will ignore this method and provide the following debugging implementation (Note: You must fully-qualify the FileInfo class since Galaxy defines a FileInfo class as well):

public override bool CopyFile(string windows src, string galaxy dst) {
    System.IO.FileInfo fi = new System.IO.FileInfo(windows src);
    Debug.WriteLine("The Galaxy NSE wants to copy a file of length"+
    fi.Length+" to our filesystem");
    return true;

}

3.3 CreateDirectory

Since we are implementing a random read-only file system, the CreateDirectory call is a null operation in our implementation.

Use the following code to implement this method:

public override bool CreateDirectory(string path)
{
  Debug.WriteLine("We would normally create a directory with the path" 
                  + path + " at this point");
  return true;
}

3.4 CreateFile

The same goes for the create file command, so use the following code here:

public override bool CreateFile(string path) 
{
    Debug.WriteLine("We should create a 0¡length file with the name " 
                    + path + " here");
    return true;
}

3.5 Delete

Again, since our RandomFileSystem is read only, use the following code for delete:

public override bool Delete(string path) { 
    Debug.WriteLine("We are being asked to delete the file "+path);
    return true;
}

3.6 ListFiles

Here, we need to create a random set of files and return them. We will make use of the myNumFiles field to help us choose how many files to return. We choose each odd numbered file to be a text file, while each even-numbered file returned is a folder (i.e., a directory). Each text file is given a random FileSize from 0 to 1023 bytes. This size will be displayed in Windows Explorer, and also used by Galaxy: whenever the file is opened or copied, only the first FileSize bytes will be read.

Folders are given a size of 0, and whenever a folder is opened in Windows Explorer, this ListFiles method will be called on that path name. (Note: While we fill in all of the fields, only the FileName, FileSize, and FolderFlag are currently used by Galaxy.)

public override FileInfo[] ListFiles(string path) {
    Random rnd = new Random();
    int num files to return = rnd.Next(myNumFiles);
    FileInfo[] ret = new FileInfo[num files to return];
    for (int i=0;i<num files to return;i++) f ret[i] = new FileInfo();
        ret[i].CreateTime = DateTime.Now; // not currently used

        ret[i].LastAccessTime = DateTime.Now; // not currently used

        ret[i].LastModifiedTime = DateTime.Now; // not currently used

        ret[i].FilePath = "Empty path field"; // not currently used



        ret[i].FolderFlag = (i%2 == 0); // even numbered files are folders


        if (ret[i].FolderFlag) f ret[i].FileName = ""+i;
            ret[i].Size = 0;
        } else {
            ret[i].Size = rnd.Next(1024); // files can be up to 1k in size

            ret[i].FileName = i+".txt"; // non¡folders are text files    

        }
    }
    return ret;
}

3.7 Read

The Read method is called whenever a file in Galaxy is copied to Windows, or when a Galaxy file is opened. Since we are implementing a simple read-only random file system, we will return a random string of bytes for the read call (taking care to return only the specified number of bytes). Note that we do not keep the file size consistent across calls, and simply create a new file size in this call. Also, we make sure not to return more bytes than Windows Explorer is asking for (count), and we only return data when the offset into the file is equal to 0 (i.e., normally, the first read call).

public override byte[] Read(string path, int offset, int count) {
    Random rnd = new Random(); // files can be up to 1k in size

    int max file size;
    if (offset > 0) { // only return data the first time we are asked

        max file size = 0;
    } else { max file size = rnd.Next(1024);
    }// only return up to ’count’ bytes

    int file size = Math.Min(max file size,count);
    byte[] ret = new byte[file size];
    for (int i=0;i<ret.Length;i++) { 
        ret[i] = (byte)(’a’+ rnd.Next(26));
    }
    return (ret);
}

3.8 Stat

The Stat method is called by Galaxy to get file information whenever files are copied from Galaxy to Windows. We will parse the path name to determine if we are being asked for a file with a name containing the string ".txt". If it does, then we know that the path refers to a file. Otherwise, the path refers to a folder. (Since we control all file names, we can be sure that folders do not contain the string ".txt")

Note that we do not take special care to preserve consistent file sizes: the file size returned from this method is random, and is probably different from the file size returned from the ListFiles method.

public override FileInfo Stat(string path)
{
    FileInfo fi = new FileInfo();
    fi.CreateTime = DateTime.Now; // not currently used
    fi.LastAccessTime = DateTime.Now; // not currently used
    fi.LastModifiedTime = DateTime.Now; // not currently used

    Random rnd = new Random();
    // parse the path into file and directory parts

    int index = path.LastIndexOf("/");
    string filename = path.Substring(index+1);
    string filepath = path.Substring(0,index);
    if (filename.IndexOf(".txt")!=(¡1))
    {
        // this must be a file
        fi.FolderFlag = false;
    }
    else
    {
        // this must be a folder
        fi.FolderFlag = true;
    }
    fi.FileName = filename; // keep the same filename
    fi.FilePath = filepath; // not currently used

    if (fi.FolderFlag)
    { 
        fi.Size = 0;
    }
    else
    { 
        fi.Size = rnd.Next(1024); // files can be up to 1k in size
    }
    return (fi);
}

3.9 Write

Since we are implementing a read-only file system, we ignore this method and provide a simple implementation:

public override int Write(string path, int offset, int count, byte[] buffer) {
    Debug.WriteLine("Explorer wants to write "+
        count+" bytes into a file called "+
        path+" at byte offset "+offset);
    return count; // pretend that we wrote all of the bytes

}

A complete listing of the random file system is available with the source Zip file.

3.10 Changing the Driver file

Now, we will modify the driver file Driver.cs to create our new file-system extension. Change the line (roughly line 50):

FileServer s = new FileServer(int.Parse(args[0]), 
               new FileSystems.Mirror.MirrorFileSystem(args[1]) );

to

FileServer s = new FileServer(int.Parse(args[0]), 
               new RandomFileSystem(int.Parse(args[1])));

Note that we are passing the second command line argument to the RandomFileSystem constructor as an integer. So, let’s now change this command line parameter to a proper value. Select the CSharpFileSystem project, then select Properties. Change the command line params under "Configuration Properties, Debugging, Start Options, Command Line Arguments" from "8052 ../../Test" to "8052 10". This should allow up to 10 files per folder in our new random file-system. Apply the change, and click "OK". Now, build the code and start our new random file-system by hitting "F5".

Navigate to the Galaxy file system using Windows Explorer, and verify that the random file system works. You may get what seems like strange behavior (zero files, the number of files keep changing, the content of each file changes) but this is how the RandomFileSystem works! You can copy files from the RandomFileSystem, but take care when copying folders: you have a very good chance of exceeding the maximum path length of 260 characters since the depth of any folder is a random variable with unbounded expectation.

3.11 Next steps

As a next step, you can read through the MirrorFileSystem file system contained in FileSystems.MirrorFileSystem.cs in the source code bundle. This is a more complete file system example which mirrors a specified folder of the local Windows file system. Be sure to change the command line arguments of the CSharpFileSystem project back to their original form.

Since, for the MirrorFileSystem, the second command line argument specifies which folder to mirror, you are free to change this to any folder that you choose. For example, you can change the command line arguments to "8052 C:\Windows" if you would like. (The first argument, 8052, specifies which port the File Server will listen to and must match the port used by the Galaxy namespace extension.)

4 Known issues

While basic file operations are implemented (e.g., cut-and-paste, drag-n-drop), there are some limitations in the NSE which should be fixed in future releases. These limitations for Galaxy v1.0 are listed below:

1. Explorer will hang if your file system is accessed before your extension starts up. This is fixable, but as long as you take care to start your file system extension before you navigate to it, you should be okay. If you run into this problem, simply kill the explorer.exe process and restart it using the Task Manager.
2. Icons for files are limited. Icons are restricted to either the folder icon or the icon for text files. This will be fixed in a future release.
3. Renaming of files is not currently supported. This is a priority for me to fix right now, and should change with the next release of the code.
4. Multiple selections are not allowed when copying from/to the Galaxy namespace. This is a restriction that should be relaxed in future releases.
5. Deletion of files occurs with out confirmation. When you delete a file in the Windows file system, typically a Yes/No dialog box is used to confirm the delete. This is not implemented (yet) in Galaxy.
6. Properties of files/folders in Galaxy is not yet supported.
7. The attributes of a file/folder is limited to filename and file size. Future releases will support the display of the file’s last modified and creation times, among other attributes.
8. The current maximum file and path length is 260 characters. If Unicode paths are used, then it is possible to extend this to 65,000 characters.

5 Acknowledgments

The task of learning COM and the NSE interfaces was made easier by the contribution of several whom we would like to acknowledge here. First, the articles published by Pascal Hurni, Nemanja Trifunovic, Henk Devos, and Michael Dunn on the CodeProject.com website were invaluable in bringing us up to speed on namespace extensions and COM programming in the Windows environment.

The GPL’d code from Pascal Hurni on the CodeProject website and the GPL’d Amiga Disk File namespace extension by Bjarke Viksoe formed the basis for our GFT toolkit. Finally, all of the users who frequent the newsgroup microsoft.public.platformsdk.shell and, in particular, Jim Barry were helpful in finding subtle bugs in our implementation.