Combres - WebForm & MVC Client-side Resource Combine Library

• Download source code - 849 KB
• Download Combres binary from CodePlex
• Download latest Combres source from CodePlex
• Download samples from CodePlex

Introduction

Many web developers probably know about various website optimization techniques described in the YSlow documentation and Steve Souders' book. Most of these techniques are very simple, yet bring about huge difference to the downloading time of most web pages. As simple as they are, applying some of these rules again and again in all .NET web applications can easily become a tedious task. As I couldn't find an existing solution that meets my needs from the Internet, I went about implementing Combres (formerly known as Client-side Resource Combine Library), a very easy-to-use library which can be used to automate many steps that you would have to do yourself when applying certain optimization techniques in your MVC and Web Form ASP.NET applications.

Key features of Combres

• Organize resource files, including JavaScript and CSS, into separate resource sets; each may share the same or use different configuration settings.
• Configuration settings are specified in an XML file which is monitored by Combres so that changes get noticed and applied immediately.
• Resource files can be static files in the web server, dynamically generated files, or remote files from external servers or web applications.
• Allow files in resource sets to be combined, minified, and gzipped before sending to the browser. All of this is done using a single HTTP request per resource set. (Refer to Yslow's performance rules #1, #4, and #10 to know why this is useful.)
• The minification part is performed by the great YUI Compressor library.
• Generate proper ETag and Expires/Cache-Control headers for every response as well as support server-side caching. (Refer to Yslow's performance rules #3 and #13 to know why this is useful.)
• Integrated with ASP.NET routing engine and thus work equally well for both ASP.NET MVC and ASP.NET WebForm applications.
• Support Debugging mode, which won't cache or minify contents at all to facilitate debugging.
• Extensibility via the filtering architecture. Anyone can easily add more functionality to the Combres engine by writing a custom filter. There are two built-in filters in the 1.0 beta release, which I will describe in this article.

Steps to use Combres

Step 1. Modify your web.config file to register the path of the Combres configuration file

Add the following elements to your web.config file:

You can use any path for the definitionUrl, as long as it is a partial ASP.NET path (i.e., starting with "~/"). You don't have to put it in the App_Data folder if you don't want to, but it's a good convention to put application data files there.

In this example, the file combres.xml in the App_Data folder will be the configuration file for the Combres library. You might wonder why I chose to put configuration settings in yet another file instead of leveraging the existing web.config file. The reason is that any change to web.config forces an application restart, and I don't think anybody would want to have such a time-consuming process take place just because they make simple changes to their CSS and JavaScript files, like renaming a file, or moving it to another folder.

Step 2. Create the Combres configuration file in the location you specified in the previous step

Let's look at a sample Combres configuration file, and I'll explain what each element and attribute means.

The above file (optionally) defines the list of filters to be used to extend the standard behaviors of Combres. I'll explain what filters are later; for now, just forget about them for a minute.

The most important thing in the Combres configuration file is the resource sets definition. As I mentioned earlier, you can organize your JavaScript and CSS files into separate resource sets. This sample configuration file defines three resource sets: a CSS set and two JavaScript sets (notice the value of the type attribute of the resourceSet element). There is, of course, no limit to the number of resource sets you can define.

Each resource set contains one or more resources of the same type (CSS or JavaScript). All these files are to be combined, minified, gzipped, and cached together, as well as requested in one single HTTP request.

Let's look at the attributes of the resourceSet element:

• name: the name of the resource set. Resource sets must not have the same name, and Combres will throw a validation exception if there are two resource sets with the same name. The resource set's name is used to form the path of the URL to request the combined content of all resources in the set.
• type: either 'css' or 'js'.
• duration: duration in days that the browser and server should cache the computed result of a resource set. This is a huge time-saver, knowing that the process for all resources in a set to be combined, minified, and gzipped is costly. Why support both client-side caching and server-side caching? Because the users can easily invalidate their browser cache in various ways (like hitting Ctrl-F5, or deleting their browser's data etc.), in which case the browser will send a brand new HTTP request asking for the data, and you don't want to go through the whole computation process just because of one particular user 's action.
• If duration is not specified for a resource set, it inherits from the defaultDuration attribute of the resourceSets element. At least one of these must be specified.
• version: each resource set must have a version, which can be any number or string. The version, if changed, will invalidate the browser's cache and the server's cache. So, if one or more files in the resource set is updated and ready to be tested or pushed into production, you would want to change the resource set's version so that the new content can be used.
• If version is not specified for a resource set, it inherits from the defaultVersion attribute of the resourceSets element. At least one of these must be specified.
• debugEnabled: during development time, you might not want the contents to be served from the cache and force you to keep increasing the resource set's version. Plus, if you are going to do any client-side debugging, you don't want to see the minified version of your CSS and JavaScript. By default, debugEnabled is false; if set to true, Combres will disable all caching mechanisms including not emitting cache headers to the browser as well as ignore the minification step.
• If debugEnabled is not specified for a resource set, it inherits from the defaultDebugEnabled attribute of the resourceSets element. Both can be omitted, in which case, a false value is assumed.

Now, let's look at the attributes of the resource element.

• path: the URL to the JavaScript or CSS resource. If the resources are deployed with your web application, their URLs must start with ~/ so that Combres can resolve them into correct URLs regardless of whether your application is deployed in a virtual directory or not. If the resources are remote (e.g., from a CDN), their URLs can be anything.
• mode: either Remote, LocalDynamic, or LocalStatic (default).
• Remote: Combres uses HTTP to request and retrieve the resource from the remote server (or web application).
• LocalDynamic: same as Remote although Combres resolves the URLs differently.
• LocalStatic: Combres reads the resource directly from the file system.

Step 3. Register the route

Notice the attribute URL in the element resourceSets. Whatever value you specify here will be used by the ASP.NET routing engine to map with the Combres processing pipeline. In the example, I use combres.axd although you can use any name and extension (as long as you configure IIS to route that extension to ASP.NET; the axd extension is already configured out of the box). The next thing you need to do is register that route with the ASP.NET routing engine. Depending on whether you're working on an ASP.NET MVC or an ASP.NET WebForms application, the steps are a little bit different.

ASP.NET MVC

The routing module is registered by default for an ASP.NET MVC application, so there are only a few steps you need to do. In the routing registration routine in global.asax, invoke routes.AddCombresRoute("Combres"). (AddCombresRoute() is an extension method defined in a type in the Combres namespace, so you need to import the Combres namespace into your global.asax, i.e., "using Combes".) Note that since I use the axd extension, I need to put the call before routes.IgnoreRoute("{resource}.axd/{*pathInfo}"), which is added by default by the ASP.NET MVC project template; otherwise, the routing engine will ignore our Combres route.

Combres is now ready to serve combined contents via the 'combres.axd' path. Next, in views that we need to use JS and CSS files, add the links to the resource sets. The following examples show an MVC view which uses two resource sets we defined in the configuration file:

CombresUrl and CombresLink are extension methods defined in the Combres namespace, therefore import this namespace to your page first. CombresUrl generates only the URL, while CombresLink generates the full script or link tag, depending on the type of the resource set. This is the generated code:

Note the nice URLs generated: combres.axd is followed by the resource set name and its current version. Including version as part of the URL is one of the things Combres does in order to control the caching behavior of the client browser. That should be it. If you run your application and use tools like FireBugs to disassemble the HTTP request and response, say the request to the siteCss resource set, you'll see the following:

There are a couple of things in the above screenshots that are worth noticing:

• The content is gzipped. (It wouldn't have been gzipped if your browser didn't support that. Combres detects that automatically.)
• Proper Cache-Control and Expires headers are sent as per your configuration. ETag is also computed and sent to the browser.
• The CSS resources are combined into one request and minified.

ASP.NET WebForm

As of ASP.NET 3.5 SP1, the routing module is not registered by default in the ASP.NET project template. Therefore, there are a number of steps you need to follow to register that module. There is a great step-by-step tutorial that shows you how to do this, so I won't repeat it here. Once the routing module is integrated into the ASP.NET pipeline, register the Combres route as follows:

To use Combres in a ASPX web page, do as follows:

WebExtensions is a type defined in the Combres namespace, therefore import this namespace to your page first. Things will work exactly the same with the ASP.MVC example above.

Understanding filters

Filter is a mechanism enabled by Combes to help developers add custom logic to the standard Combres' processing pipeline via means of intercepting key transformation phases. In order to implement a filter, you implement the ICombresFilter interface by overriding its three methods:

The diagram below shows how Combres interacts with filters during its processing pipeline, which is simplified in the diagram to show only the relevant parts. At each of the interception points, all the filters registered with Combres will be instantiated and invoked. Each of these filters has a chance to modify the output of the previous phase, and the modified content will be fed into the next phase.

At each interception point, an instance of a filter type is instantiated on the fly and one of its methods is invoked, depending on the current phase. This invocation idiom is to make it easy to develop custom filters: you can have instance variables if you want to, and you don't have to consider thread-safety when implementing your filters. You also don't have to worry about performance (for using a lot of Reflection here), because Fasterflect, another library I developed to turn Reflection invocations into close-to-native invocations, is used internally by Combres.

After developing a filter, you need to register it with Combres via the configuration file. (Now, you can refer back to the sample configuration file to look at the filters element.) Each filter needs to appear in a filter element with its type specified.

Combres ships with two built-in filters: HandleCssVariablesFilter and FixUrlsInCssFilter.

Let's look at what they do and how they are implemented.

HandleCssVariablesFilter

The idea of this filter comes from a blog post by Rory Neopoleon. Basically, it allows you to define variables in CSS files. For example, you can have a CSS with the following @define block:

@define
{
    boxColor: #345131;
    boxWidth: 150px;
}
p
{
    color: @boxColor;
    width: @boxWidth;
}

The filter will turn the content into the following:

p
{
    color: #345131;
    width: 150px;
}

This simple technique is a great way to refactor your CSS files. Let's look at the implementation of this filter.

public class HandleCssVariablesFilter : ICombresFilter
{
    public string TransformSingleContent(Settings settings, 
                  Resource resource, string content)
    {
        if (resource.ParentSet.Type != ResourceType.CSS)
            return content;

        // Remove comments because it may mess up the result
        content = Regex.Replace(content, @"/\*.+?\*/", "", 
                                RegexOptions.Singleline);
        var regex = new Regex(@"@define\s*{(?<define>.*?)}", 
                              RegexOptions.Singleline);
        var match = regex.Match(content);
        if (!match.Success)
            return content;

        var value = match.Groups["define"].Value;
        var variables = value.Split(';');
        var sb = new StringBuilder(content);
        variables.ToList().ForEach(var =>
                      {
                         if (string.Empty == var.Trim())
                             return;
                         var pair = var.Split(':');
                         sb.Replace("@" + pair[0].Trim(), pair[1].Trim());
                      });

        // Remove the variables declaration,
        // it's not needed in the final output
        sb.Replace(match.ToString(), string.Empty);
        return sb.ToString();
    }

    public string TransformCombinedContent(Settings settings, 
                  ResourceSet set, string content)
    {
        return content;
    }

    public string TransformMinifiedContent(Settings settings, 
                  ResourceSet set, string content)
    {
        return content;
    }
}

Note that TransformSingleContent() is overridden because we want this filter to work on a per CSS file basis. The other methods simply return the original input. Some might be scared by the amount of text manipulation done and its implication to performance. In reality, this is hardly a problem, thanks to the caching mechanism supported by Combres. Chances are that the processing is done once and the content is served from either the browser's or the server's cache for days or even months, depending on your configuration settings and how frequently you update your contents.

FixUrlsInCssFilter

This is a neat filter that addresses a problem many users of Combres reported. The problem is that URLs referenced in CSS files are interpreted by browsers as relative to the location of the CSS file. Because of that, when Combres is used to serve CSS content, unless the Combres' registered route starts in the same folder as the CSS file (unlikely, given that you might have a lot of CSS files in different folders in your application), the browser might not be able to resolve the correct paths to the referenced URLs (which result in issues like images not being shown). The FixUrlsInCssFilter is built to fix this problem. Even more than that, it allows you to use the standard ASP.NET partial part, e.g., URLs starting with '~/'.

For example, assume the path of the CSS file is ~/content/site.css, then the following transformations are done by the filter to each URL reference in the CSS file:

• Absolute path
• /path/to/image.gif becomes /path/to/image.gif
• Relative from CSS location -> Relative from CSS location
• path/to/image.gif becomes /content/path/to/imag2.gif
• ../path/to/style1.css becomes /path/to/style1.css
• Starting from application root
• ~/path/to/style2.css becomes /content/path/to/style2.css

Let's look at how the filter is implemented:

public class FixUrlsInCssFilter : ICombresFilterreadonly {
    ILog Log = LogManager.GetLogger(
                   System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);

    public string TransformSingleContent(Settings settings, 
                  Resource resource, string content)
    {
        if (resource.ParentSet.Type == ResourceType.CSS)
            return Regex.Replace(content, @"url\((?<url>.*?)\)", 
                match => FixUrl(resource.Path, match),
                RegexOptions.IgnoreCase | RegexOptions.Singleline | 
                RegexOptions.ExplicitCapture);
        return content;
    }

    private static string FixUrl(string cssPath, Match match)
    {
        try
        {
            const string template = "url(\"{0}\")";
            var url = match.Groups["url"].Value.Trim('\"', '\'');
            if (url.StartsWith("/"))
                return string.Format(template, url);
            if (url.StartsWith("~"))
                return string.Format(template, url.ResolveUrl());

            var cssFolder = cssPath.Substring(0, cssPath.LastIndexOf("/"));
            var backFolderCount = Regex.Matches(url, @"\.\./").Count;
            for (int i = 0; i < backFolderCount; i++)
            {
                url = url.Substring(3); // skip a '../'
                cssFolder = cssFolder.Substring(0, cssFolder.LastIndexOf("/"));
                // move back 1 folder
            }
            return string.Format(template, (cssFolder + "/" + url).ResolveUrl());
        }
        catch (Exception ex)
        {
            // Be lenient here, only log. After all,
            // this is just an image in the CSS file
            // and it should't be the reason to stop loading that CSS file.
            if (Log.IsWarnEnabled) 
                Log.Warn("Cannot fix url " + match.Value, ex);
            return match.Value;
        }
    }

    public string TransformCombinedContent(Settings settings, 
                  ResourceSet set, string content)
    {
        return content;
    }

    public string TransformMinifiedContent(Settings settings, 
                  ResourceSet set, string content)
    {
        return content;
    }
}

Conclusion

In this article, I have shown how easy it is to use Combres to employ many website performance optimization techniques in your ASP.NET and ASP.NET MVC applications. I have also demonstrated the extensibility of Combres via the filtering mechanism. With this mechanism in place, you can easily extend Combres to fit your own needs.

The source code of Combres comes with two sample applications, an ASP.NET MVC and an ASP.NET WebForm. You can take the configuration and code in these samples to quickly get your web application up and running with Combres.

I love to hear from you. Either post your comments here or in the project's discussion board at CodePlex.