Click here to Skip to main content
Click here to Skip to main content
Go to top

Creating a Custom Tool to Generate Multiple Files in Visual Studio 2005

, 30 Nov 2006
Rate this:
Please Sign up or sign in to vote.
Visual Studio provides interfaces for extending the environment to generate one file from another through a "Custom Tool". Now, learn how to generate multiple files, easily, and extensibly.
Solution Explorer: An Existing Custom Tool

Property Grid: Setting Your Custom Tool

The End Result

Introduction

Visual Studio is a great environment for Rapid Application Development. It also provides a rich extensibility API for customizing it for your specific needs.
A good example of this is the highly utilized "Custom Tool" facility.

When a file is selected in the Solution Explorer, the property grid displays several attributes, common across any file type. One of these is the "Custom Tool" attribute. When this attribute is set correctly, it refers to an Assembly/Type that can generate a new file (whenever the source file is saved), based upon the source file as input, and store it as a child node of the source file, in the Solution Explorer.
Examples of this are the MSDiscoGenerator, MSDataSetGenerator and ResXFileCodeGenerator.

The complexities behind creating and deploying your own Custom Tools are surprisingly few. The name that is placed in the "Custom Tool" attribute, is simply referring to a node in the registry, which in turn, points to an Assembly and a Type in that Assembly (in the case of Custom Tools written in Managed Code). This allows Visual Studio to instantiate the Custom Tool, and execute its interface methods to generate the required output.

One pre-requisite of a Custom Tool is that it implements a specific Managed Interface, to allow Visual Studio to call the transformation, for a given file. This interface, Microsoft.VisualStudio.TextTemplating.VSHost.IVsSingleFileGenerator, unfortunately, only contains hooks for generating a single destination file from each source file.

Background

This document does not explain the procedure for creating or deploying a Custom Tool, you are expected to already know this. This document explains how to create a reusable base class, which overcomes the 'Single File' limitation.
For information on creating Custom Tools, please read Jasmin Muharemovics article titled "VS.NET CodeDOM-Based Custom Tool for String Resource Management".

An Extensible Approach to Multiple File Generation

Obviously, generating multiple files from a single source is going to have some inherent application specific dependencies.
For example, if your input file is HTML, and you are going to generate a child file for each occurrence of the <IMG> tag in that HTML, you are going to need specific code to find each instance, and then iteratively perform the 'generation' routines.

I have used .NET 2.0, and generics to make the implementation highly flexible, allowing you to simply override the base class, named 'VsMultipleFileGenerator', and implement 3 simple methods.

To begin, I will explain the inner workings of the base class.

Under the Hood

The attached project implements an abstract base class (an implementation of IVsSingleFileGenerator) which is easily subclassed to implement multiple file generation.

Firstly, let's begin the class declaration:

Required References and Prerequisites

Before you begin, you will need to make sure that you have the Visual Studio SDK installed, to have access to the assemblies you will need.

You can download this from the Microsoft Visual Studio Extensibility website.

You will need the following references added to your project to use the required classes and interfaces.

  1. EnvDTE
  2. Microsoft.VisualStudio.OLE.Interop
  3. Microsoft.VisualStudio.Shell
  4. Microsoft.VisualStudio.Shell.Interop
  5. Microsoft.VisualStudio.Shell.Interop.8.0
  6. Microsoft.VisualStudio.TextTemplating.VSHost

Our Type Declaration

We define our class as abstract, so that we force the implementation to declare certain methods required for the iterative and generation process.

public abstract class VsMultipleFileGenerator<IterativeElementType><T> :
                                   IEnumerable<T><IterativeElementType>,
                                   IVsSingleFileGenerator,
                                   IObjectWithSite
{
    #region Visual Studio Specific Fields
    private object site;
    private ServiceProvider serviceProvider = null;
    #endregion

    #region Our Fields
    private string bstrInputFileContents;
    private string wszInputFilePath;
    private EnvDTE.Project project;

    private List<string><STRING> newFileNames;
    #endregion

    protected EnvDTE.Project Project
    {
        get
        {
            return project;
        }
    }

    protected string InputFileContents
    {
        get
        {
            return bstrInputFileContents;
        }
    }

    protected string InputFilePath
    {
        get
        {
            return wszInputFilePath;
        }
    }

    private ServiceProvider SiteServiceProvider
    {
        get
        {
            if (serviceProvider == null)
            {
                IServiceProvider oleServiceProvider =
                    site as IServiceProvider;
                serviceProvider = new ServiceProvider(oleServiceProvider);
            }
            return serviceProvider;
        }
    }

The class uses a generic type declaration, defining IterativeElementType as the type that will be passed to our generation methods.
This class also implements IEnumerable<>, which also receives the IterativeElementType type for its enumeration type. The result of this is, our class will provide strongly typed enumeration facilities, to retrieve each element type from the underlying file. This type may be an System.Xml.XmlNode, or some other type if you are performing some custom deserialization of the stream.

The Visual Studio specific interfaces are IVsSingleFileGenerator, and IObjectWithSite, these interfaces supply Visual Studio with the hooks it needs to actually initialize and execute the Custom Tool.

Our class then declares a few private instance variables to provide as service fields to our concrete sub-class.

  1. bstrInputFileContents
    This variable is populated from Visual Studio, it generally contains the actual string contents of the source file. We will place it in this variable so that our Interface methods (such as the IEnumerable interface) have access to it.
  2. wszInputFilePath
    This is the physical location of the source file on disk, again populated from Visual Studio. For the same reasons as above, we have placed it into an instance field, and provided a read-only protected Property for our concrete base-class to access.
  3. project
    Because we are essentially rewriting the functionality of Visual Studios' Single File Generator, we need access to the IDE object model to be able to associate the files we are going to create, with the source file, so that they may appear as child nodes in the Solution Explorer.
    We are creating an instance declaration for this field, rather than just hiding its implementation inside of the generation methods, so that we can provide access to it in the concrete subclass. There's no point in requiring our sub-class to double the overhead of obtaining another reference to DTE, when we've already done it!

Finally, we have a variable called newFileNames, this stores a list of the filenames we are generating when our Custom Tool executes. The purpose is to allow our code, after all our child files have been generated, to make sure that any child-files whose names have changed during the generation process, have their previous file deleted.
This sort of clean-up is not necessary when using the simple IVsSingleFileGenerator, as the filename stays the same every time the Custom Tool executes, thus ensuring it will always overwrite the old file. Our VsMultipleFileGenerator may change the number of files, and their names, each time it executes, so it must clean-up any old files after it is done.

public VsMultipleFileGenerator()
{
    EnvDTE.DTE dte = (EnvDTE.DTE)Package.GetGlobalService(typeof(EnvDTE.DTE));
    Array ary = (Array)dte.ActiveSolutionProjects;
    if (ary.Length > 0)
    {
        project = (EnvDTE.Project)ary.GetValue(0);
    }
    newFileNames = new List<STRING><string>();
} 

The constructor

Here we perform some basic field initialization. After obtaining a reference to the DTE object, we can grab the active Project from the solution. Finally, we must instantiate the newFileNames collection.

Enumeration

public abstract IEnumerator<IterativeElementType><iterativeelementtype /> GetEnumerator();

System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
    return GetEnumerator();
}

These methods satisfy the IEnumerable<IterativeElementType> interface, and our concrete subclass is forced to implement them.

VsMultipleFileGenerator requires that any concrete subclass implements these methods to return an IEnumerator containing a list of whatever type the subclass was declared to represent. You will see how to implement these in "Using the code" section.

More Methods to Override

Our generator needs to know some more things about the files we will be generating. Firstly, we need to know what to call each file:

protected abstract string GetFileName(IterativeElementType element);   

The implementation class must override this, and interrogate whatever type their 'element' variable is, to determine what filename to return. This is an unqualified filename, so should look something like "MyFirstFile.txt".

We also need to actually generate some content for each of these target files, which is what the following abstract method is for:

public abstract byte[] GenerateContent(IterativeElementType element); 

Finally, we have to deal with some legacy stuff. The interface we have implemented, IVsSingleFileGenerator, is obviously intended to generate only a single file. So, unfortunately, our code has to satisfy this requirement, and generate content for this file, as well as all our other files. We don't actually have much control over the name of this 'single' file, Visual Studio will call it whatever the source filename is, with the 'Default Extension' string appended to the end of it. This is most likely to reduce the risk of Custom Tools generating naming conflicts, whereby they may generate files that overwrite existing ones in the solution - something WE must be wary of.

I have taken the approach that this single file can contain summary information about our generation process, so I have named the method accordingly. I usually declare this file as a ".txt" file, and fill it with some auto-generated descriptive information regarding the set of files that have generated, and the name of the tool responsible.

GetDefaultExtension tells Visual Studio what extension to append to the filename of our single file. This can return something like ".txt"

GenerateSummaryContent returns a byte array of data, used to populate the content of this file.

public abstract string GetDefaultExtension();

public abstract byte[] GenerateSummaryContent();

Where all the Magic Happens

The Generate method is called by Visual Studio, and is the root of the generation process. Here, we initialize our instance variables, iterate over the elements in our source file (as defined by our enumeration methods), create the target files, populate them, add them to the solution, then clean up any stale target files, and generate our summary file. The embedded comments explain more clearly step-by-step what is going on.

public void Generate(string wszInputFilePath, string bstrInputFileContents,
    string wszDefaultNamespace, out IntPtr rgbOutputFileContents,
    out int pcbOutput, IVsGeneratorProgress pGenerateProgress)
{
    this.bstrInputFileContents = bstrInputFileContents;
    this.wszInputFilePath = wszInputFilePath;
    this.newFileNames.Clear();

    int iFound = 0;
    uint itemId = 0;
    EnvDTE.ProjectItem item;
    VSDOCUMENTPRIORITY[] pdwPriority = new VSDOCUMENTPRIORITY[1];

    // obtain a reference to the current project as an IVsProject type
    Microsoft.VisualStudio.Shell.Interop.IVsProject VsProject =
                        VsHelper.ToVsProject(project);
    // this locates, and returns a handle to our source file, as a ProjectItem
    VsProject.IsDocumentInProject(InputFilePath, out iFound,
                        pdwPriority, out itemId);

    // if our source file was found in the project (which it should have been)
    if (iFound != 0 && itemId != 0)
    {
        Microsoft.VisualStudio.OLE.Interop.IServiceProvider oleSp = null;
        VsProject.GetItemContext(itemId, out oleSp);
        if (oleSp != null)
        {
            ServiceProvider sp = new ServiceProvider(oleSp);
            // convert our handle to a ProjectItem
            item = sp.GetService(typeof(EnvDTE.ProjectItem))
                        as EnvDTE.ProjectItem;
        }
        else
            throw new ApplicationException
            ("Unable to retrieve Visual Studio ProjectItem");
    }
    else
        throw new ApplicationException
            ("Unable to retrieve Visual Studio ProjectItem");

    // now we can start our work,
    // iterate across all the 'elements' in our source file
    foreach (IterativeElementType element in this)
    {
        try
        {
            // obtain a name for this target file
            string fileName = GetFileName(element);
            // add it to the tracking cache
            newFileNames.Add(fileName);
            // fully qualify the file on the filesystem
            string strFile = Path.Combine( wszInputFilePath.Substring(0,
                    wszInputFilePath.LastIndexOf
            (Path.DirectorySeparatorChar)), fileName);
            // create the file
            FileStream fs = File.Create(strFile);
            try
            {
                // generate our target file content
                byte[] data = GenerateContent(element);

                // write it out to the stream
                fs.Write(data, 0, data.Length);

                fs.Close();

                // add the newly generated file to the solution,
                // as a child of the source file...
                EnvDTE.ProjectItem itm =
                item.ProjectItems.AddFromFile(strFile);
                /*
                 * Here you may wish to perform some addition logic
                 * such as, setting a custom tool for the target file if it
                 * is intended to perform its own generation process.
                 * Or, set the target file as an 'Embedded Resource' so that
                 * it is embedded into the final Assembly.

                EnvDTE.Property prop = itm.Properties.Item("CustomTool");
                //// set to embedded resource
                itm.Properties.Item("BuildAction").Value = 3;
                if (String.IsNullOrEmpty((string)prop.Value) ||
                    !String.Equals((string)prop.Value, typeof
                        (AnotherCustomTool).Name))
                {
                    prop.Value = typeof(AnotherCustomTool).Name;
                }
                */
            }
            catch (Exception)
            {
                fs.Close();
                if ( File.Exists( strFile ) )
                    File.Delete(strFile);
            }
        }
        catch (Exception ex)
        {
        }
    }

    // perform some clean-up, making sure we delete any old
    // (stale) target-files
    foreach (EnvDTE.ProjectItem childItem in item.ProjectItems)
    {
       if (!(childItem.Name.EndsWith(GetDefaultExtension()) ||
                newFileNames.Contains(childItem.Name)))
            // then delete it
            childItem.Delete();
    }

    // generate our summary content for our 'single' file
    byte[] summaryData = GenerateSummaryContent();

    if (summaryData == null)
    {
        rgbOutputFileContents = IntPtr.Zero;

        pcbOutput = 0;
    }
    else
    {
        // return our summary data, so that Visual Studio may write it to disk.
        rgbOutputFileContents = Marshal.AllocCoTaskMem(summaryData.Length);

        Marshal.Copy(summaryData, 0,
                rgbOutputFileContents, summaryData.Length);

        pcbOutput = summaryData.Length;
    }
}

Using the code

Now that we have written our extensible base class, we can begin the fun part, actually providing an implementation.

As an example, I am going to generate a custom tool that takes an HTML file as its source, and retrieves all the images (<a href=""> tags) in that file, downloads them from the internet, and embeds them into the assembly. I can't think of any practical use for a tool like this, but hey, it's a fun example.

[Guid("6EE05D8F-AAF9-495e-A8FB-143CD2DC03F5")]
public class HtmlImageEmbedderCustomTool : VsMultipleFileGenerator<STRING>
{
    public override IEnumerator<STRING> GetEnumerator()
    {
        Stream inStream = File.OpenRead(base.InputFilePath);
        Regex regAnchor = new Regex("<img src=[\"']([^\"']+)[\"'][^>]+[/]?>", 
                            RegexOptions.IgnoreCase);
        try
        {
            StreamReader reader = new StreamReader(inStream);
            string line = null;
            while ((line = reader.ReadLine()) != null)
            {
                MatchCollection mc = regAnchor.Matches(line);
                foreach (Match match in mc)
                {
                    // yield each element to the enumerator
                    yield return match.Groups[1].Value;
                }
            }
        }
        finally
        {
            inStream.Close();
        }
    }

    protected override string GetFileName(string element)
    {
        return element.Substring(element.LastIndexOf('/') + 1);
    }

    public override byte[] GenerateContent(string element)
    {
        // create the image file
        WebRequest getImage = WebRequest.Create(element);

        return StreamToBytes( getImage.GetResponse().GetResponseStream() );
    }

    public override byte[] GenerateSummaryContent()
    {
        // I'm not going to put anything in here...
        return new byte[0];
    }

    public override string GetDefaultExtension()
    {
        return ".txt";
    }

    protected byte[] StreamToBytes(Stream stream)
    {
        MemoryStream outBuffer = new MemoryStream();

        byte[] buffer = new byte[1024];
        int count = 0;
        while( (count = stream.Read( buffer, 0, buffer.Length )) > 0 )
        {
            outBuffer.Write( buffer, 0, count );
        }

        return outBuffer.ToArray();
    }
}

You can then perform the usual registry additions to register the custom tool assembly (remember to add it to the GAC first), then activate the Custom Tool against an HTML file that has image tags with absolute URLs. This will result in all the images linked from the HTML, being downloaded and saved into the project as child items of the HTML node.

Notes

If you are using the sample solution, I have already created a pre/post build event which adds the assembly to the GAC every time you compile (removing it first - to refresh any changes). There is also a .reg file in the solution folder which will register the custom tool for you.

If you have downloaded the demo zip, the .reg file is also included, but you will need to manually drag the assembly into 'c:\windows\assembly', to register it in the GAC.

Remember: If you are running Microsoft Vista, you will need to run Visual Studio as Administrator for the build events to work.

Once you have registered the custom tool, start up a new instance of Visual Studio (so that it may re-load the registry configuration), and add an HTML file to the solution (make sure all the 'img' tags in the HTML are absolute references: i.e. 'http://...').
Then, select the HTML file in the solution explorer, and set its 'Custom Tool' attribute (in the property grid) to "HtmlImageEmbedder" (without quotes).
As soon as you hit enter, you should see children nodes of the HTML file appear in the solution explorer, with filenames the same as the images in the HTML. Open some of them, they're the images - downloaded from the website!

Additional Code

The downloadable sample project contains additional helper code in the form of a static class named VsHelper.

History

  • 2006.11.25 Initial Publication

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author

Adam Langley
Software Developer (Senior)
New Zealand New Zealand
"Find a job you love, and you'll never work a day in your life."
 
Adam Langley is a software engineer in Auckland, New Zealand.
 
Software development is his personal passion, he takes pride in his work, and likes to share his experiences with the development community.
 
When he's not coercing computers, you'll find him riding his motorcycle, or attempting to carve something creative from bone.

Comments and Discussions

 
QuestionHas anyone used this with VS2010 or later? PinmemberQwertie27-Oct-13 18:50 
AnswerRe: Has anyone used this with VS2010 or later? PinmemberQwertie28-Oct-13 8:14 
QuestionCustom tool MultipleFileGenerator Pinmembersamsulhuda29-Jul-13 0:02 
GeneralMy vote of 5 Pinmemberrdkleine23-Oct-12 4:37 
QuestionProblem with VsMultipleFileGenerator.Generate method (and solution) PinmemberNotre29-May-12 12:46 
BugGenerate does not work unless project is saved with source file in it. Pinmemberpistolspete18-Jul-11 10:48 
QuestionLicense? PinmemberSinnerG12-Dec-10 13:39 
AnswerRe: License? PinmemberAdam Langley12-Dec-10 14:06 
General64bit Windows Pinmemberasdf4616-Nov-09 13:23 
GeneralRe: 64bit Windows PinmemberNotre16-Mar-10 10:08 
QuestionForcing Generation PinmemberBiggerNoise8-Jul-09 5:50 
AnswerRe: Forcing Generation PinmemberAdam Langley29-Sep-09 15:01 
GeneralRe: Forcing Generation PinmemberBiggerNoise29-Sep-09 18:52 
JokeAhh, the sweet memories PinmemberDmitri Nesteruk17-Nov-08 10:49 
GeneralExcellent artcile Pinmembertalamoni10-Jan-08 21:29 
QuestionHow to debug? PinmemberAndrewCsy27-Apr-07 0:35 
Nice code sample - it's very helpful.
How would I debug this though?

 
Andrew
AnswerRe: How to debug? PinmemberAdam Langley27-Apr-07 11:24 
Questionabout: Microsoft.VisualStudio.TextTemplating.VSHost Pinmemberffhust28-Nov-06 15:19 
AnswerRe: about: Microsoft.VisualStudio.TextTemplating.VSHost PinmemberAdam Langley28-Nov-06 15:56 
GeneralRe: about: Microsoft.VisualStudio.TextTemplating.VSHost Pinmemberffhust30-Nov-06 15:59 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140916.1 | Last Updated 1 Dec 2006
Article Copyright 2006 by Adam Langley
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid