Daniel Coburn

Today I read an article about how sitemap.xml files are not as useful as some people say they are in the world of SEO.  The argument was just because you have a page in the Google, Yahoo, MSN etc. index doesn’t get it ranked.  And I completely agree with them that just because it’s there doesn’t give it rank.  So the question needs to be why use a sitemap.xml file, and once you have them what do you do? Also what about a human consumable sitemap? What’s it’s function and why should you have one that is bot friendly?

First lets tackle the sitemap.xml.  These things are must haves, for a few of reasons.  They help you get your full site indexed, they help you get new pages into the indexes fast, they allow navigation of pages that might be isolated because of no-follow strategies,  and they let you set information about each page, such as how often it’s modified and it’s page priority.

A sitemap.xml is only going to get your pages into the search engine, it will be up to the public (and to some level you) to “vote” on how important those pages are to the world. 

What are the first steps to creating a sitemap.xml file? There are several ways to create a file, the simplest is to find an application that will spider your site and create an xml file for you… easy enough, but what happens if you have massive amounts of pages? Firstly you cannot exceed 50k links per file or 10 mb max, so if you do have more you will have to split your map up into multiple pieces.  The syntax for the “root” is pretty simple

<?xml version=”1.0″ encoding=”UTF-8″?>
<sitemapindex xmlns=”http://www.sitemaps.org/schemas/sitemap/0.9″>
  <sitemap>
    <loc>http://www.yoursite.com/sitemap_01.xml</loc>
    <lastmod>2007-01-08</lastmod>
  </sitemap>
 <sitemap>
    <loc>http://www.yoursite.com/sitemap_02.xml</loc>
    <lastmod>2007-01-08</lastmod>
  </sitemap>
</sitemapindex>

This will allow a company to get million’s of pages indexed into search engines, vs. thousands.  Once you get the pages in the index, now the magic begins.  Let us jump to user consumable sitemaps and their SEO benefit or purpose.

A user consumable sitemap is one that a person could go to and click around to view pages, ALL pages on your website.  One of the major sin’s I’ve seen is the development of really cool looking sitemaps, that are dhtml/javascript heavy and use those technologies to render elements on clicks, or throw AJAX content into a div for user consumption.  The biggest problem here is that search engines cannot read any of it.  The pages need to be bot friend and user consumable.  This means, no more than 100 links per page and keyword rich links. 

“Daniel WHAT? ONLY 100 LINKS!” — yes 100 links otherwise you get link spammy and that’s not very good, it makes the spiders think you are a link farm.  So there’s this really cool thing called Pagination! And yes those links need to be bot friendly too so they can find the next page.

What this will do is help start creating an internal linking that will allow spiders to find your pages, and start building internal link juice.  This will begin to start allowing them to be one of the more popular kids on the block.  Now all you have to do is start working on external content…. but that’s down the road a bit.