Creating a Custom Tool to Generate Multiple Files in Visual Studio 2005

• Download demo project - 8.3 KB
• Download source - 19.5 KB

Introduction

Visual Studio is a great environment for Rapid Application Development. It also provides a rich extensibility API for customizing it for your specific needs.
A good example of this is the highly utilized "Custom Tool" facility.

When a file is selected in the Solution Explorer, the property grid displays several attributes, common across any file type. One of these is the "Custom Tool" attribute. When this attribute is set correctly, it refers to an Assembly/Type that can generate a new file (whenever the source file is saved), based upon the source file as input, and store it as a child node of the source file, in the Solution Explorer.
Examples of this are the MSDiscoGenerator, MSDataSetGenerator and ResXFileCodeGenerator.

The complexities behind creating and deploying your own Custom Tools are surprisingly few. The name that is placed in the "Custom Tool" attribute, is simply referring to a node in the registry, which in turn, points to an Assembly and a Type in that Assembly (in the case of Custom Tools written in Managed Code). This allows Visual Studio to instantiate the Custom Tool, and execute its interface methods to generate the required output.

One pre-requisite of a Custom Tool is that it implements a specific Managed Interface, to allow Visual Studio to call the transformation, for a given file. This interface, Microsoft.VisualStudio.TextTemplating.VSHost.IVsSingleFileGenerator, unfortunately, only contains hooks for generating a single destination file from each source file.

Background

This document does not explain the procedure for creating or deploying a Custom Tool, you are expected to already know this. This document explains how to create a reusable base class, which overcomes the 'Single File' limitation.
For information on creating Custom Tools, please read Jasmin Muharemovics article titled "VS.NET CodeDOM-Based Custom Tool for String Resource Management".

An Extensible Approach to Multiple File Generation

Obviously, generating multiple files from a single source is going to have some inherent application specific dependencies.
For example, if your input file is HTML, and you are going to generate a child file for each occurrence of the <IMG> tag in that HTML, you are going to need specific code to find each instance, and then iteratively perform the 'generation' routines.

I have used .NET 2.0, and generics to make the implementation highly flexible, allowing you to simply override the base class, named 'VsMultipleFileGenerator', and implement 3 simple methods.

To begin, I will explain the inner workings of the base class.

Under the Hood

The attached project implements an abstract base class (an implementation of IVsSingleFileGenerator) which is easily subclassed to implement multiple file generation.

Firstly, let's begin the class declaration:

Required References and Prerequisites

Before you begin, you will need to make sure that you have the Visual Studio SDK installed, to have access to the assemblies you will need.

You can download this from the Microsoft Visual Studio Extensibility website.

You will need the following references added to your project to use the required classes and interfaces.

1. EnvDTE
2. Microsoft.VisualStudio.OLE.Interop
3. Microsoft.VisualStudio.Shell
4. Microsoft.VisualStudio.Shell.Interop
5. Microsoft.VisualStudio.Shell.Interop.8.0
6. Microsoft.VisualStudio.TextTemplating.VSHost

Our Type Declaration

We define our class as abstract, so that we force the implementation to declare certain methods required for the iterative and generation process.

public abstract class VsMultipleFileGenerator<IterativeElementType><T> :
                                   IEnumerable<T><IterativeElementType>,
                                   IVsSingleFileGenerator,
                                   IObjectWithSite
{
    #region Visual Studio Specific Fields
    private object site;
    private ServiceProvider serviceProvider = null;
    #endregion

    #region Our Fields
    private string bstrInputFileContents;
    private string wszInputFilePath;
    private EnvDTE.Project project;

    private List<string><STRING> newFileNames;
    #endregion

    protected EnvDTE.Project Project
    {
        get
        {
            return project;
        }
    }

    protected string InputFileContents
    {
        get
        {
            return bstrInputFileContents;
        }
    }

    protected string InputFilePath
    {
        get
        {
            return wszInputFilePath;
        }
    }

    private ServiceProvider SiteServiceProvider
    {
        get
        {
            if (serviceProvider == null)
            {
                IServiceProvider oleServiceProvider =
                    site as IServiceProvider;
                serviceProvider = new ServiceProvider(oleServiceProvider);
            }
            return serviceProvider;
        }
    }

The class uses a generic type declaration, defining IterativeElementType as the type that will be passed to our generation methods.
This class also implements IEnumerable<>, which also receives the IterativeElementType type for its enumeration type. The result of this is, our class will provide strongly typed enumeration facilities, to retrieve each element type from the underlying file. This type may be an System.Xml.XmlNode, or some other type if you are performing some custom deserialization of the stream.

The Visual Studio specific interfaces are IVsSingleFileGenerator, and IObjectWithSite, these interfaces supply Visual Studio with the hooks it needs to actually initialize and execute the Custom Tool.

Our class then declares a few private instance variables to provide as service fields to our concrete sub-class.

1. bstrInputFileContents
   This variable is populated from Visual Studio, it generally contains the actual string contents of the source file. We will place it in this variable so that our Interface methods (such as the IEnumerable interface) have access to it.
2. wszInputFilePath
   This is the physical location of the source file on disk, again populated from Visual Studio. For the same reasons as above, we have placed it into an instance field, and provided a read-only protected Property for our concrete base-class to access.
3. project
   Because we are essentially rewriting the functionality of Visual Studios' Single File Generator, we need access to the IDE object model to be able to associate the files we are going to create, with the source file, so that they may appear as child nodes in the Solution Explorer.
   We are creating an instance declaration for this field, rather than just hiding its implementation inside of the generation methods, so that we can provide access to it in the concrete subclass. There's no point in requiring our sub-class to double the overhead of obtaining another reference to DTE, when we've already done it!

Finally, we have a variable called newFileNames, this stores a list of the filenames we are generating when our Custom Tool executes. The purpose is to allow our code, after all our child files have been generated, to make sure that any child-files whose names have changed during the generation process, have their previous file deleted.
This sort of clean-up is not necessary when using the simple IVsSingleFileGenerator, as the filename stays the same every time the Custom Tool executes, thus ensuring it will always overwrite the old file. Our VsMultipleFileGenerator may change the number of files, and their names, each time it executes, so it must clean-up any old files after it is done.

public VsMultipleFileGenerator()
{
    EnvDTE.DTE dte = (EnvDTE.DTE)Package.GetGlobalService(typeof(EnvDTE.DTE));
    Array ary = (Array)dte.ActiveSolutionProjects;
    if (ary.Length > 0)
    {
        project = (EnvDTE.Project)ary.GetValue(0);
    }
    newFileNames = new List<STRING><string>();
} 

The constructor

Here we perform some basic field initialization. After obtaining a reference to the DTE object, we can grab the active Project from the solution. Finally, we must instantiate the newFileNames collection.

Enumeration

public abstract IEnumerator<IterativeElementType> GetEnumerator();

System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
    return GetEnumerator();
}

These methods satisfy the IEnumerable<IterativeElementType> interface, and our concrete subclass is forced to implement them.

VsMultipleFileGenerator requires that any concrete subclass implements these methods to return an IEnumerator containing a list of whatever type the subclass was declared to represent. You will see how to implement these in "Using the code" section.

More Methods to Override

Our generator needs to know some more things about the files we will be generating. Firstly, we need to know what to call each file:

protected abstract string GetFileName(IterativeElementType element);   

The implementation class must override this, and interrogate whatever type their 'element' variable is, to determine what filename to return. This is an unqualified filename, so should look something like "MyFirstFile.txt".

We also need to actually generate some content for each of these target files, which is what the following abstract method is for:

public abstract byte[] GenerateContent(IterativeElementType element); 

Finally, we have to deal with some legacy stuff. The interface we have implemented, IVsSingleFileGenerator, is obviously intended to generate only a single file. So, unfortunately, our code has to satisfy this requirement, and generate content for this file, as well as all our other files. We don't actually have much control over the name of this 'single' file, Visual Studio will call it whatever the source filename is, with the 'Default Extension' string appended to the end of it. This is most likely to reduce the risk of Custom Tools generating naming conflicts, whereby they may generate files that overwrite existing ones in the solution - something WE must be wary of.

I have taken the approach that this single file can contain summary information about our generation process, so I have named the method accordingly. I usually declare this file as a ".txt" file, and fill it with some auto-generated descriptive information regarding the set of files that have generated, and the name of the tool responsible.

GetDefaultExtension tells Visual Studio what extension to append to the filename of our single file. This can return something like ".txt"

GenerateSummaryContent returns a byte array of data, used to populate the content of this file.

public abstract string GetDefaultExtension();

public abstract byte[] GenerateSummaryContent();

Where all the Magic Happens

The Generate method is called by Visual Studio, and is the root of the generation process. Here, we initialize our instance variables, iterate over the elements in our source file (as defined by our enumeration methods), create the target files, populate them, add them to the solution, then clean up any stale target files, and generate our summary file. The embedded comments explain more clearly step-by-step what is going on.

public void Generate(string wszInputFilePath, string bstrInputFileContents,
    string wszDefaultNamespace, out IntPtr rgbOutputFileContents,
    out int pcbOutput, IVsGeneratorProgress pGenerateProgress)
{
    this.bstrInputFileContents = bstrInputFileContents;
    this.wszInputFilePath = wszInputFilePath;
    this.newFileNames.Clear();

    int iFound = 0;
    uint itemId = 0;
    EnvDTE.ProjectItem item;
    VSDOCUMENTPRIORITY[] pdwPriority = new VSDOCUMENTPRIORITY[1];

    // obtain a reference to the current project as an IVsProject type

    Microsoft.VisualStudio.Shell.Interop.IVsProject VsProject =
                        VsHelper.ToVsProject(project);
    // this locates, and returns a handle to our source file, as a ProjectItem

    VsProject.IsDocumentInProject(InputFilePath, out iFound,
                        pdwPriority, out itemId);

    // if our source file was found in the project (which it should have been)

    if (iFound != 0 && itemId != 0)
    {
        Microsoft.VisualStudio.OLE.Interop.IServiceProvider oleSp = null;
        VsProject.GetItemContext(itemId, out oleSp);
        if (oleSp != null)
        {
            ServiceProvider sp = new ServiceProvider(oleSp);
            // convert our handle to a ProjectItem

            item = sp.GetService(typeof(EnvDTE.ProjectItem))
                        as EnvDTE.ProjectItem;
        }
        else
            throw new ApplicationException
            ("Unable to retrieve Visual Studio ProjectItem");
    }
    else
        throw new ApplicationException
            ("Unable to retrieve Visual Studio ProjectItem");

    // now we can start our work,

    // iterate across all the 'elements' in our source file

    foreach (IterativeElementType element in this)
    {
        try
        {
            // obtain a name for this target file

            string fileName = GetFileName(element);
            // add it to the tracking cache

            newFileNames.Add(fileName);
            // fully qualify the file on the filesystem

            string strFile = Path.Combine( wszInputFilePath.Substring(0,
                    wszInputFilePath.LastIndexOf
            (Path.DirectorySeparatorChar)), fileName);
            // create the file

            FileStream fs = File.Create(strFile);
            try
            {
                // generate our target file content

                byte[] data = GenerateContent(element);

                // write it out to the stream

                fs.Write(data, 0, data.Length);

                fs.Close();

                // add the newly generated file to the solution,

                // as a child of the source file...

                EnvDTE.ProjectItem itm =
                item.ProjectItems.AddFromFile(strFile);
                /*
                 * Here you may wish to perform some addition logic
                 * such as, setting a custom tool for the target file if it
                 * is intended to perform its own generation process.
                 * Or, set the target file as an 'Embedded Resource' so that
                 * it is embedded into the final Assembly.

                EnvDTE.Property prop = itm.Properties.Item("CustomTool");
                //// set to embedded resource
                itm.Properties.Item("BuildAction").Value = 3;
                if (String.IsNullOrEmpty((string)prop.Value) ||
                    !String.Equals((string)prop.Value, typeof
                        (AnotherCustomTool).Name))
                {
                    prop.Value = typeof(AnotherCustomTool).Name;
                }
                */
            }
            catch (Exception)
            {
                fs.Close();
                if ( File.Exists( strFile ) )
                    File.Delete(strFile);
            }
        }
        catch (Exception ex)
        {
        }
    }

    // perform some clean-up, making sure we delete any old

    // (stale) target-files

    foreach (EnvDTE.ProjectItem childItem in item.ProjectItems)
    {
       if (!(childItem.Name.EndsWith(GetDefaultExtension()) ||
                newFileNames.Contains(childItem.Name)))
            // then delete it

            childItem.Delete();
    }

    // generate our summary content for our 'single' file

    byte[] summaryData = GenerateSummaryContent();

    if (summaryData == null)
    {
        rgbOutputFileContents = IntPtr.Zero;

        pcbOutput = 0;
    }
    else
    {
        // return our summary data, so that Visual Studio may write it to disk.

        rgbOutputFileContents = Marshal.AllocCoTaskMem(summaryData.Length);

        Marshal.Copy(summaryData, 0,
                rgbOutputFileContents, summaryData.Length);

        pcbOutput = summaryData.Length;
    }
}

Using the code

Now that we have written our extensible base class, we can begin the fun part, actually providing an implementation.

As an example, I am going to generate a custom tool that takes an HTML file as its source, and retrieves all the images (<a href=""> tags) in that file, downloads them from the internet, and embeds them into the assembly. I can't think of any practical use for a tool like this, but hey, it's a fun example.

[Guid("6EE05D8F-AAF9-495e-A8FB-143CD2DC03F5")]
public class HtmlImageEmbedderCustomTool : VsMultipleFileGenerator<STRING>
{
    public override IEnumerator<STRING> GetEnumerator()
    {
        Stream inStream = File.OpenRead(base.InputFilePath);
        Regex regAnchor = new Regex("<img src=[\"']([^\"']+)[\"'][^>]+[/]?>", 
                            RegexOptions.IgnoreCase);
        try
        {
            StreamReader reader = new StreamReader(inStream);
            string line = null;
            while ((line = reader.ReadLine()) != null)
            {
                MatchCollection mc = regAnchor.Matches(line);
                foreach (Match match in mc)
                {
                    // yield each element to the enumerator

                    yield return match.Groups[1].Value;
                }
            }
        }
        finally
        {
            inStream.Close();
        }
    }

    protected override string GetFileName(string element)
    {
        return element.Substring(element.LastIndexOf('/') + 1);
    }

    public override byte[] GenerateContent(string element)
    {
        // create the image file

        WebRequest getImage = WebRequest.Create(element);

        return StreamToBytes( getImage.GetResponse().GetResponseStream() );
    }

    public override byte[] GenerateSummaryContent()
    {
        // I'm not going to put anything in here...

        return new byte[0];
    }

    public override string GetDefaultExtension()
    {
        return ".txt";
    }

    protected byte[] StreamToBytes(Stream stream)
    {
        MemoryStream outBuffer = new MemoryStream();

        byte[] buffer = new byte[1024];
        int count = 0;
        while( (count = stream.Read( buffer, 0, buffer.Length )) > 0 )
        {
            outBuffer.Write( buffer, 0, count );
        }

        return outBuffer.ToArray();
    }
}

You can then perform the usual registry additions to register the custom tool assembly (remember to add it to the GAC first), then activate the Custom Tool against an HTML file that has image tags with absolute URLs. This will result in all the images linked from the HTML, being downloaded and saved into the project as child items of the HTML node.

Notes

If you are using the sample solution, I have already created a pre/post build event which adds the assembly to the GAC every time you compile (removing it first - to refresh any changes). There is also a .reg file in the solution folder which will register the custom tool for you.

If you have downloaded the demo zip, the .reg file is also included, but you will need to manually drag the assembly into 'c:\windows\assembly', to register it in the GAC.

Remember: If you are running Microsoft Vista, you will need to run Visual Studio as Administrator for the build events to work.

Once you have registered the custom tool, start up a new instance of Visual Studio (so that it may re-load the registry configuration), and add an HTML file to the solution (make sure all the 'img' tags in the HTML are absolute references: i.e. 'http://...').
Then, select the HTML file in the solution explorer, and set its 'Custom Tool' attribute (in the property grid) to "HtmlImageEmbedder" (without quotes).
As soon as you hit enter, you should see children nodes of the HTML file appear in the solution explorer, with filenames the same as the images in the HTML. Open some of them, they're the images - downloaded from the website!

Additional Code

The downloadable sample project contains additional helper code in the form of a static class named VsHelper.

History

• 2006.11.25 Initial Publication