Daniel Coburn

Today I read an article about how sitemap.xml files are not as useful as some people say they are in the world of SEO.  The argument was just because you have a page in the Google, Yahoo, MSN etc. index doesn’t get it ranked.  And I completely agree with them that just because it’s there doesn’t give it rank.  So the question needs to be why use a sitemap.xml file, and once you have them what do you do? Also what about a human consumable sitemap? What’s it’s function and why should you have one that is bot friendly?

First lets tackle the sitemap.xml.  These things are must haves, for a few of reasons.  They help you get your full site indexed, they help you get new pages into the indexes fast, they allow navigation of pages that might be isolated because of no-follow strategies,  and they let you set information about each page, such as how often it’s modified and it’s page priority.

A sitemap.xml is only going to get your pages into the search engine, it will be up to the public (and to some level you) to “vote” on how important those pages are to the world. 

What are the first steps to creating a sitemap.xml file? There are several ways to create a file, the simplest is to find an application that will spider your site and create an xml file for you… easy enough, but what happens if you have massive amounts of pages? Firstly you cannot exceed 50k links per file or 10 mb max, so if you do have more you will have to split your map up into multiple pieces.  The syntax for the “root” is pretty simple

<?xml version=”1.0″ encoding=”UTF-8″?>
<sitemapindex xmlns=”http://www.sitemaps.org/schemas/sitemap/0.9″>
  <sitemap>
    <loc>http://www.yoursite.com/sitemap_01.xml</loc>
    <lastmod>2007-01-08</lastmod>
  </sitemap>
 <sitemap>
    <loc>http://www.yoursite.com/sitemap_02.xml</loc>
    <lastmod>2007-01-08</lastmod>
  </sitemap>
</sitemapindex>

This will allow a company to get million’s of pages indexed into search engines, vs. thousands.  Once you get the pages in the index, now the magic begins.  Let us jump to user consumable sitemaps and their SEO benefit or purpose.

A user consumable sitemap is one that a person could go to and click around to view pages, ALL pages on your website.  One of the major sin’s I’ve seen is the development of really cool looking sitemaps, that are dhtml/javascript heavy and use those technologies to render elements on clicks, or throw AJAX content into a div for user consumption.  The biggest problem here is that search engines cannot read any of it.  The pages need to be bot friend and user consumable.  This means, no more than 100 links per page and keyword rich links. 

“Daniel WHAT? ONLY 100 LINKS!” — yes 100 links otherwise you get link spammy and that’s not very good, it makes the spiders think you are a link farm.  So there’s this really cool thing called Pagination! And yes those links need to be bot friendly too so they can find the next page.

What this will do is help start creating an internal linking that will allow spiders to find your pages, and start building internal link juice.  This will begin to start allowing them to be one of the more popular kids on the block.  Now all you have to do is start working on external content…. but that’s down the road a bit.

Anyone who has even taken a physics class knows that matter is neither created nor destroyed….  The same sort rings true for pagerank.  All of the pagerank in the world is equal to 1.  If you add 1 million pages, the universal sum is still 1.  Since you cannot change the universe, you need to know how to massage it.  If you have a website with 5 pages, you cannot control a lot of the virtual real estate, and will have to create more depth to control more of the pagerank available.  If you want to grow pagerank on certain pages you need to funnel traffic through links to them.  For example pages that should not have pagerank are contact us, privacy policy, etc.  they don’t earn you money so don’t give them the benefit of pushing rank to them.  So how do you push the rank internally?  Simple provide links.

All page link to your home page, so the home page of a site typically gets more rank since more pages both internally and externally are pushing rank to it.  While pages that should also be considered are category pages etc.  Many of these will be part of your universal navigation so they will also get rank pushed to them.  It’s important that you use appropriate keywords were possible to allow search engines to know the phrase that is important to the page you are pointing too.

You also want to have links to your contact us page etc since you do want to allow customers to get there, and you need them for better ranking in Google, but instead of use <a href=”contactusurl”>contact us</a> use a no follow <a href=”contacturl” rel=”nofollow”>contact us</a> this will prevent you from passing rank from a page to that un “wanted” page.