Continued from page 1
A sitemap page is a very good way of giving
search engine robot every opportunity to reach your website pages. Since robots click through
links of your web pages, make sure that at least your most important pages are included in
sitemap; you may even want to include all your pages there, depending on
size of your site. Be sure to add a link to
sitemap page from each page on your site.
Another important consideration is that of keeping all of your pages within a small number of "clicks" from your top page. Many robots will not follow links more than two or three levels deep, so if your "widgets" page can only be reached from your home page by following multiple links (e.g. home page >> about us page >> products page >> widgets page),
robot may not crawl deep enough to get to
widgets page.
Testing Your Website For Search Engine Robot Accessibility To get an idea just what
search engine robot "sees" on your page, you can look at
Sim Spider tool. You may be surprised at how different your site looks to
robot. You can find this tool at http://www.searchengineworld.com/cgi-bin/sim_spider.cgi
You will see text and ALT image text show up in
results. If your entire website is built in Flash, you will see nothing at all because robots don't understand Flash movies.
The Bottom Line When it comes to search engine robots, think simply. Lots of good content and text, hyperlinks
robots can follow, optimization of your pages, topical links pointing back to your site and a sitemap will help insure
best results when
robots come visiting.
Resources
SpiderSpotting - Search Engine Watch http://searchenginewatch.com/webmasters/spiders.html
Robotstxt.org List of robots and protocols for setting up a robots.txt file. http://www.robotstxt.org/
Spider-Food Tutorials, forums and articles about Search Engine spiders and Search Engine Marketing. http://spider-food.net/
Spiderhunter.com Articles and resources about tracking Search Engine spiders. http://www.spiderhunter.com/
Sim Spider Search Engine Robot Simulator Search Engine World has a spider that simulates what
Search Engine robots read from your website. http://www.searchengineworld.com/cgi-bin/sim_spider.cgi
