Click here to Skip to main content
Click here to Skip to main content

Simple Sitemaps in ASP.NET

, 21 Apr 2007
Rate this:
Please Sign up or sign in to vote.
Easiest way to create sitemaps for Google, Yahoo, and Ask. Provides a simple class which implements an XML sitemap file and the search engine ping.

Introduction

The simple sitemap library is, as stated, an easy way to add sitemap support to your ASP.NET application. Just add the SitemapLib.cs file to your project and you are ready to go. The library implements the sitemaps.org standard for creating an XML sitemap file, and includes Ping support for all the search engines that currently support it: Google.com, Yahoo.com and Ask.com.

Please feel free to modify, reuse or redistribute the code as you see fit.

Background on Sitemaps

The sitemap standard was created by Google, Yahoo and Microsoft to provide an easy way for web site owners to inform search engines about pages on their site that are available for crawling.

Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data to allow crawlers that support sitemaps to pick up all URLs in the sitemap and learn about those URLs using the associated metadata. Using sitemaps does not guarantee that web pages are included in search engines, but provides hints for web crawlers to do a better job of crawling your site.

Using SitemapLib

The library supports two main aspects of the standard, creating XML sitemaps, and Pinging the major search engines to notify them of updates in the sitemap.

Creating an XML Sitemap

The code snippet below demonstrates all the code required to create a sitemap using the library. Since the sitemap will be an XML file, it is better to use a "Generic Handler" file type (ashx) than an ASPX file. ASPX is optimized to generate HTML files, which doesn't work in this case. The ASHX file on the other hand, allows you to create any type of file you would like.

using System;
using stem.Web;

using SitemapLib;

public class Handler : IHttpHandler 
{
    public void ProcessRequest (HttpContext context)
    {
        SitemapLib.Sitemap sitemap = new SitemapLib.Sitemap();

        sitemap.AddLocation("http://mysite.com/default.aspx");
        sitemap.AddLocation(http://mysite.com/products/shoes/default.aspx, 
                                DateTime.Today);
        sitemap.AddLocation(http://mysite.com/products/coats/coat.aspx?id=3, 
                DateTime.Today, "0.8", ChangeFrequency.Monthly);

        context.Response.ContentType = "text/xml";
        context.Response.Write(sitemap.GenerateSitemapXML());
    }

    public bool IsReusable 
    {
        get 
        {
            return false;
        }
    }
}

Once you have instantiated a new SitemapLib.Sitemap class, you can use one of the many versions of the AddLocation() method to add URLs to your sitemap, depending on which metadata you would like to include in your sitemap file.

Note

The library validates the input URLs and the output file for common issues like malformed URLs, max URL length, max Filesize length, etc. To be a good programmer, you should use try{} catch(Exception e) {} and handle all errors thrown. Failure to manage these errors could mean that the search engines disregard your sitemap file.

Notifying Search Engines with a Ping

You have the option to notify each search engine that supports sitemaps of updates by using an HTTP Ping command. The following code shows you how to use the SitemapLib to ping all the Search Engines that support it.

// Replace with the fully qualified domain name of your sitemap file, 
// and your yahoo application id.
// You can provision one here: 
// http://developer.yahoo.com/search/siteexplorer/V1/updateNotification.html

Sitemap.Ping("http://mysite.com/sitemap.ashx", "Yahoo Application ID");

The first parameter should be the fully qualified path to your sitemap.xml file. Note that your XML file does not need to have an "XML" extension to be valid. The only odd thing about using the Ping is that Yahoo breaks the standard by requiring that developers include an application id with each ping. You can easily provision one here: http://developer.yahoo.com/search/siteexplorer/V1/updateNotification.html and include it in your program.

Notes on Implementing Sitemaps on your Website

  • Don't forget to add the "sitemap:" directive to your Robots.txt file

    Instead of registering your sitemap with every search engine, you can support all of them by adding one line to your Robots.txt file, "Sitemap: <fully qualified path to your sitemap file, or sitemap index file>".

  • Sitemaps are a suggestion, not a command

    The search engines will take all of the information provided by your sitemaps as one of many inputs into determining what pages should be included in their index, what is the relative priority of each page on your site, and how frequently your pages should be crawled.

  • Generally, sitemaps will not impact your page rank

    They are most likely to affect the number of pages that are crawled and indexed, as well as the speed at which they are indexed. However, there are some cases where sitemaps have significantly impacted the page rank of a site, and that is when the sites were not getting fully crawled, and the sitemap led to the inclusion of some really good content into the index.

  • Give your converting pages the highest priority

    Product pages, newsletter signup pages and other pages that generate the most value from your customers should receive the highest priority. Also include the pages that you know your customers will be the most interested in.

  • Don't give all your pages 1.0 priority

    This just tells all the search engines that all your pages are of equal importance, it doesn't help boost the importance of any of your pages specifically.

  • Give sitemaps time to see impact

    You may not see an impact right away from implementing sitemaps, but that doesn't mean that they are not being used. Search engines are still optimizing their implementations and figure out how exactly to use all of this information.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author

Nathan Buggia
Marketing
United States United States
Hello, I have been working in various aspects of web technology since my first web dev job in 1997, doing PERL and CGI. Since then I have worked in Java, C++/CGI (no, really), Systems Administration, Systems Architecture, and finally moved into Marketing for Microsoft's Live Search.
 
You can also find me at my blog: nathanbuggia.com

Comments and Discussions

 
GeneralVB code to use XmlTextWriter instead of StringBuilder: Pinmembery2kisdead19-Sep-07 8:13 
GeneralVB Translation PinmemberMike Joseph20-Jun-07 7:54 
GeneralRe: VB Translation PinmemberNathan Buggia20-Jun-07 8:06 
GeneralRe: VB Translation PinmemberWhittet25-Jun-08 9:22 
Question'&' not allowed Pinmemberjariaman27-May-07 20:32 
AnswerRe: '&' not allowed PinmemberNathan Buggia27-May-07 21:54 
Jariaman, the ampersand (&) is not allowed because sitemaps must be in the UTF-8 encoding to be properly processed.
 
However, you shouldn't have to worry about that, the SitemapLib takes care of this encoding process for you, so you can just give it any valid URL and it will properly encode to UTF-8. For example, if your URL was the following:
 
http://mysite.com/products?id=3&color=red
 
you can just use the following method from the library like this:
 
sitemap.AddLocation("http://mysite.com/products?id=3&color=red", DateTime.Today, "0.8", ChangeFrequency.Monthly);
 
regards,
nate
GeneralSuggestions PinmemberRichard Deeming24-Apr-07 7:32 
GeneralRe: Suggestions PinmemberNathan Buggia24-Apr-07 8:04 
GeneralRe: Suggestions [modified] PinmemberRichard Deeming25-Apr-07 1:21 
NewsSource Code Posted! PinmemberNathan Buggia24-Apr-07 5:57 
QuestionRe: Source Code Posted! PinmemberSteveC-A94-May-07 5:20 
QuestionWhere's the beef? PinmemberPCBender24-Apr-07 5:14 
QuestionWhere can one access the SitemapLib.cs file? PinmemberTheBigBadWolf23-Apr-07 21:04 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web04 | 2.8.140827.1 | Last Updated 22 Apr 2007
Article Copyright 2007 by Nathan Buggia
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid