Continued from page 1
A sitemap page is a very good way of giving search engine robot every opportunity to reach your website pages. Since robots click through links of your web pages, make sure that at least your most important pages are included in sitemap; you may even want to include all your pages there, depending on size of your site. Be sure to add a link to sitemap page from each page on your site.
Another important consideration is that of keeping all of your pages within a small number of "clicks" from your top page. Many robots will not follow links more than two or three levels deep, so if your "widgets" page can only be reached from your home page by following multiple links (e.g. home page >> about us page >> products page >> widgets page), robot may not crawl deep enough to get to widgets page.
Testing Your Website For Search Engine Robot Accessibility To get an idea just what search engine robot "sees" on your page, you can look at Sim Spider tool. You may be surprised at how different your site looks to robot. You can find this tool at http://www.searchengineworld.com/cgi-bin/sim_spider.cgi
You will see text and ALT image text show up in results. If your entire website is built in Flash, you will see nothing at all because robots don't understand Flash movies.
The Bottom Line When it comes to search engine robots, think simply. Lots of good content and text, hyperlinks robots can follow, optimization of your pages, topical links pointing back to your site and a sitemap will help insure best results when robots come visiting.
SpiderSpotting - Search Engine Watch http://searchenginewatch.com/webmasters/spiders.html
Robotstxt.org List of robots and protocols for setting up a robots.txt file. http://www.robotstxt.org/
Spider-Food Tutorials, forums and articles about Search Engine spiders and Search Engine Marketing. http://spider-food.net/
Spiderhunter.com Articles and resources about tracking Search Engine spiders. http://www.spiderhunter.com/
Sim Spider Search Engine Robot Simulator Search Engine World has a spider that simulates what Search Engine robots read from your website. http://www.searchengineworld.com/cgi-bin/sim_spider.cgi
Daria Goetsch is the founder and Search Engine Marketing Consultant for Search Innovation Marketing (www.searchinnovation.com), a Search Engine Promotion company serving small businesses. She has specialized in search engine optimization since 1998, including three years as the Search Engine Specialist for O'Reilly & Associates, a technical book publishing company.