60 Day Sandbox for Google & AskJeeves. MSN Quickest, Yahoo Next

Written by Mike Banks Valentine

Continued from page 1

Slurp gets strangely inactive then alternately hyperactive for periods of time. The Yahoo crawler will look at 40 pages one day and then 4000repparttar next, then simply look atrepparttar 150099 home page for a few days and then jump back in for 3000 pagesrepparttar 150100 next day and back to only reviewing robots.txt for two days. Consistency is not a curse suffered by Slurp. Yahoo now shows 6 pages in their index, one an errors page and another is a "index/of" page as we have not posted a home page to several subdomains. But Slurp has crawled easily 15,000 pages to date.

Lessons learned inrepparttar 150101 first 60 days on a new site follow:

1) Google crawls 250 pages on first discovery of links to site. Then they don't return until they find more links and crawl slowly. Google has failed to index new domain for 60 days.

2) Yahoo looks for errors pages and once they find bad links will crawl them ceaselessly until you tell them to stop it. Then won't crawl at all for weeks until crawling heavily one day and lightlyrepparttar 150102 next in random fashion.

3) MSNbot requires robots.txt files and once they decide they like your site, may crawl too fast, requiring "crawl-delay" instructions in that robots.txt file. Implement immediately.

4) Bad bots can strain resources and hit too many pages too quickly until you tell them to stay out. We banned 3 bots outright after they slammed our servers for a day or two. Noted "aipbot" crawled first then "BecomeBot" came along and then "Pbot" from Picsearch.com crawled heavily looking for image files we don't have. Bad bots, stay out. Best to implement robots.txt exclusions for all but top engines if their crawlers strain your server resources. We considered excludingrepparttar 150103 Chinese search engine named Baidu.com when they began crawling heavily early on. We don't expect much traffic from China, but why exclude one billion people? Especially since Google is rumored to be considering a possible purchase of Baidu.com as entry to Chinese market.

The bottom line is that we've discovered all engines seem to delay indexing of new domain names for at least thirty days. Google so far has delayed indexing THIS new domain for 60 days since first crawling it. AskJeeves has crawled thousands of pages, while indexing none of them. MSN indexes faster than all engines but requires robots.txt file. Yahoo's Slurp crawls on again off again for 60 days, but indexes only six of total 15,000 or more pages crawled to date.

We seem to have settled that there is a clear indexing delay, but whether this site specifically is "Sandboxed" and whether delays apply universally is less clear. Many webmasters claim that they have been indexed fully within 30 days of first posting a new domain. We'd love to see others track spiders through new sites following launch to document their results publicly so that indexing and crawling behavior are proven.

Mike Banks Valentine is a search engine optimization specialist who operates WebSite101 Ecommerce Tutorial and will continue reports of case study chronicling search indexing of Publish101 Article Resource http://www.seoptimism.com/SEO_Contact.htm

5 hot tips to boost your website traffic using free tools

Written by Chris K.

Continued from page 1

Once you know where your clients tend to meet or congregate onrepparttar web, you will also be inrepparttar 150098 perfect position of knowing exactly where to post your free articles. Writing and posting articles is a much more powerful online marketing strategy than most people realize. This is because, done properly, it will also have an amazing and powerful; impact ion improving your search engine rankings and thereforerepparttar 150099 traffic you receive from search engines.

Make sure that apart from postingrepparttar 150100 articles at your blog, also post them at article directories, ezine publishers and announcement lists.

e) Use a traffic exchange site

Some people look down on traffic exchange sites. This is a big mistake. In traffic exchange sites, you win traffic to your own site by visiting others belonging to other members who have signed up atrepparttar 150101 traffic exchange site. While it is true that not everybody who visit from a traffic exchange site will be interested in what you have to offer, some will inevitably be. It is a great way to launch a campaign for increased traffic because it will give your site more exposure. It is like advertising your product on prime time TV. Many will not be interested and even fewer will respond. Butrepparttar 150102 tiny fraction that shows interest and ends up buying can be a significant figure. Sorepparttar 150103 idea should be to win as much traffic to your site as possible.


About the author: Chris K. is a Technical Executive Writer for Website Source, Inc. http://www.websitesource.com. His established writing skills coupled with experience in the website hosting industry have provided internet professionals with marketing, product and service ideas for many years.

    <Back to Page 1
ImproveHomeLife.com © 2005
Terms of Use