An XML sitemap is a great way to help crawlers such as Google, MSN Search, Yahoo and ASK.com to crawl your site and also allow you to tell them how important pages on your site are relative to each other.

I created a ASP.NET handler that generates such an XML sitemap for this site, and of course you are welcome to download the source code (a Visual Studio 2008 project).

The project contains two files, a sample robots.txt which help crawlers find your sitemap and the handler, Sitemap.ashx. To implement it on your own site modify the GetSitemapPages method  in Sitemap.ashx.cs to return a list of pages in your site.

For the impatient it is also possible to tell the major search engines about your sitemap by using their webmaster tools. You can also ping Google and MSN by making HTTP requests to http://www.google.com/webmasters/tools/ping?sitemap=http://www.yoursite.com/XmlSitemap.ashx and  http://webmaster.live.com/ping.aspx?siteMap=http://www.yoursite.com/XmlSitemap.ashx. Note that you shouldn't ping them more than once per hour though, or you may anger the all mighty search engine gods.

PS. For updates about new posts, sites I find useful and the occasional rant you can follow me on Twitter. You are also most welcome to subscribe to the RSS-feed.