Playing in Googlebot's Sandbox with Slurp, Teoma & MSNbot

Written by Mike Banks Valentine

Continued from page 1

* June 4 - Teoma returns and crawls 590 pages! No others.

* June 5 - Teoma returns and crawls 1902 pages! No others. * June 6 - Teoma returns and crawls 290 pages. No others. * June 7 - Teoma returns and crawls 471 pages. No others.

* June 8-14 Odd spider behavior, looking at robots.txt only.

* June 15 - Slurp gets thirsty, gulps 1396 pages! No others.

* June 16 - Slurp still thirsty, gulps 1379 pages! No others.

So we'll take a break here atrepparttar 5 weeks point and take note ofrepparttar 147807 very different behavior ofrepparttar 147808 top crawlers. Googlebot visits once and looks at a substantial number of pages but doesn't return for over a month. Slurp finds bad links and seems addicted to them as it stops crawling good pages until it is told to lay offrepparttar 147809 bad liquor, er that is links by getting robots.txt to slap slurp to its senses. MSNbot visits looking for that robots.txt and won't crawl any pages until told what NOT to do byrepparttar 147810 robots.txt file. Teoma just crawls like crazy, takes breaks, then comes back for more.

This behavior may imitaterepparttar 147811 differing personalities ofrepparttar 147812 software engineers who designed them. Teoma is tenacious and hard working. MSNbot is timid and needs instruction and some reassurance it is doingrepparttar 147813 right thing, picks up pages slowly and carefully. Slurp has addictive personality and performs erratically on a random schedule. Googlebot takes a good long look and leaves. Who knows whether it will be back and when.

Now let's look at indexing by each engine. As of this writing on July 7, each engine also shows differing indexing behavior as well. Google shows no pages indexed although it crawled 250 pages nearly two months ago. Yahoo has three pages indexed in a clear aging routine that doesn't list any ofrepparttar 147814 nearly 8,000 pages it has crawled to date (not all itemized above.) MSN has 187 pages indexed while crawling fewer pages than any ofrepparttar 147815 others. Ask Jeeves has crawled more pages to date than any search engine, yet has not indexed a single page.

Each ofrepparttar 147816 engines will showrepparttar 147817 number of pages indexed if you userepparttar 147818 query operator "" withoutrepparttar 147819 quotes. MSN 187 pages, Ask none, Yahoo 3 pages, Google none.

The daily activity not listed inrepparttar 147820 three weeks since June 16 above has not varied dramatically, with Teoma crawling a bit more than other engines, Slurp erratically up and down and MSN slowly gathering 30 to 50 pages daily. Google is absent.

Linking campaign has been minimal with posts to discussion lists, a couple of articles and some blog activity. Looking back over this time it is apparent that a listing delay is actually quite sensible fromrepparttar 147821 view ofrepparttar 147822 search engines. Our site restructuring and bobbled robots.txt implementation seems to have abruptly stalled crawling butrepparttar 147823 indexing behavior of each engine displays distinctly differing policy by each major player.

The sandbox is apparently not just Google's playground, but it is certainly tiresome after nearly two months. I think I'd like to leave for home, have some lunch and take a nap now.

Back to class before we leave forrepparttar 147824 day kiddies. What did we learn today? Watch early crawler activity and be certain to implement robots.txt early and adjust often for bad bots. Oh yes, andrepparttar 147825 sandbox belongs to all search engines.

Mike Banks Valentine is a search engine optimization specialist who operates and will continue reports of case study chronicling search indexing of

The Fundamentals of Inbound Links

Written by Justine Curtis, Enable UK

Continued from page 1

Buying Links Buying links is also becoming more commonplace. It is a form of purchased advertising that is less obvious than banner adverts or pay per click listings. Oftenrepparttar link will be placed on multiple pages, or all pages withinrepparttar 147533 site. If purchasing a link, make sure you are clear about why are you buying it? Is it for search engine ranking purposes only, or to get traffic directly fromrepparttar 147534 link?

Make surerepparttar 147535 link is worth what you are paying in terms ofrepparttar 147536 traffic it will send you and not solely to improve your page’s ranking. Ifrepparttar 147537 link partners page is well ranked inrepparttar 147538 search engines under a specific key phrase but receives few visitors, it won’t be sending many visitors to your web site. Ifrepparttar 147539 page is well ranked, it may help with your ranking in Google andrepparttar 147540 rest, but that would berepparttar 147541 icing onrepparttar 147542 cake.

Forums Join forums and discussion groups related to your business and place links to your web site in your signature line. Use your main search terms asrepparttar 147543 link text. But before spending time writing lots of posts with your signature line in each post, make sure thatrepparttar 147544 forum is spiderable by checkingrepparttar 147545 robots.txt file. Also make sure that links in signature lines are not hidden from spiders (viewrepparttar 147546 source code to make sure that signature links are in plain HTML and point directly torepparttar 147547 site).

Link Exchange Centres Find and join free link exchange centres like There you can find a categorised directory of websites that also want to exchange links. Be careful not to sign up with FFA (Free-For-All) sites because they are mostly email address gatherers and you can expect a sudden increase in email sp@m soon after you sign up. Also, only sign up with centres where you can approach other sites personally, and where they can approach you personally.

Controlling Your Links When providing your link partners withrepparttar 147548 text to use in your inbound link, make sure it containsrepparttar 147549 key phrase you are targeting. Search engines will take into accountrepparttar 147550 words that are included inrepparttar 147551 actual link as well atrepparttar 147552 text around it so include a brief paragraph about your business and if possible link your top key phrase to your URL. Make sure that each link points torepparttar 147553 page you are promoting for that particular search term. This should improve your ranking in their listings.

Pointing allrepparttar 147554 links to your home page, with various target search terms, won't achieverepparttar 147555 same as pointing targeted links at specific pages withinrepparttar 147556 site.

How not to link Under no circumstances join any link farms! Link farms, such as, sound simple and effective, but search engines, Google in particular, disapprove of them as blatant attempts to manipulaterepparttar 147557 rankings and they will penalise sites that use them. Once a site has been penalised, it is very difficult to getrepparttar 147558 penalty lifted, so avoid all link farms at all costs.

Search engine software engineers always look for spam tactics, when a software engineer discovers a link farm; allrepparttar 147559 domains inrepparttar 147560 link farm are banned. Result? Link popularity vanishes.

Summary Acquiring links can be time-consuming and tedious. There are a number of ways to get them, some being easier than others. However you get them, inbound links are important for websites that want to move uprepparttar 147561 rankings. Inbound links, withrepparttar 147562 right link text, are essential for achieving top rankings when there is any competition forrepparttar 147563 search terms.

Link development's main advantage is dynamic, cumulative, and difficult to imitate. Many sites maintain search engine visibility andrepparttar 147564 resulting qualified traffic because of successful link development, not just based onrepparttar 147565 number of keywords on a page.

For top rankings, it is essential to takerepparttar 147566 time to get as many quality inbound links as possible. It should be treated as an ongoing task. Getting your first link is like getting your first sale. It is just as hard - and just as satisfying.

Linking strategies are just one of a wide range of subjects covered in our guide “Start atrepparttar 147567 Beginning”. It covers allrepparttar 147568 basics of building and optimising your web site for both your customers and organic search engine rankings. For an excerpt and contents list visit:

Is your web site driving high quality, targeted customers to your business? Our proven Internet Marketing solutions make your web site work harder for you. For a complete range of Internet marketing and advertising resources to improve search engine positions visit Enable UK. Stop losing customers to your competitors and make more money from your web site TODAY.

    <Back to Page 1 © 2005
Terms of Use