Slurp gets strangely inactive then alternately hyperactive for periods of time. The Yahoo crawler will look at 40 pages one day and then 4000 next, then simply look at home page for a few days and then jump back in for 3000 pages next day and back to only reviewing robots.txt for two days. Consistency is not a curse suffered by Slurp. Yahoo now shows 6 pages in their index, one an errors page and another is a "index/of" page as we have not posted a home page to several subdomains. But Slurp has crawled easily 15,000 pages to date.

Lessons learned in first 60 days on a new site follow:

1) Google crawls 250 pages on first discovery of links to site. Then they don't return until they find more links and crawl slowly. Google has failed to index new domain for 60 days.

2) Yahoo looks for errors pages and once they find bad links will crawl them ceaselessly until you tell them to stop it. Then won't crawl at all for weeks until crawling heavily one day and lightly next in random fashion.

3) MSNbot requires robots.txt files and once they decide they like your site, may crawl too fast, requiring "crawl-delay" instructions in that robots.txt file. Implement immediately.

4) Bad bots can strain resources and hit too many pages too quickly until you tell them to stay out. We banned 3 bots outright after they slammed our servers for a day or two. Noted "aipbot" crawled first then "BecomeBot" came along and then "Pbot" from Picsearch.com crawled heavily looking for image files we don't have. Bad bots, stay out. Best to implement robots.txt exclusions for all but top engines if their crawlers strain your server resources. We considered excluding Chinese search engine named Baidu.com when they began crawling heavily early on. We don't expect much traffic from China, but why exclude one billion people? Especially since Google is rumored to be considering a possible purchase of Baidu.com as entry to Chinese market.

The bottom line is that we've discovered all engines seem to delay indexing of new domain names for at least thirty days. Google so far has delayed indexing THIS new domain for 60 days since first crawling it. AskJeeves has crawled thousands of pages, while indexing none of them. MSN indexes faster than all engines but requires robots.txt file. Yahoo's Slurp crawls on again off again for 60 days, but indexes only six of total 15,000 or more pages crawled to date.

We seem to have settled that there is a clear indexing delay, but whether this site specifically is "Sandboxed" and whether delays apply universally is less clear. Many webmasters claim that they have been indexed fully within 30 days of first posting a new domain. We'd love to see others track spiders through new sites following launch to document their results publicly so that indexing and crawling behavior are proven.

Once you know where your clients tend to meet or congregate on web, you will also be in perfect position of knowing exactly where to post your free articles. Writing and posting articles is a much more powerful online marketing strategy than most people realize. This is because, done properly, it will also have an amazing and powerful; impact ion improving your search engine rankings and therefore traffic you receive from search engines.

Make sure that apart from posting articles at your blog, also post them at article directories, ezine publishers and announcement lists.

e) Use a traffic exchange site

Some people look down on traffic exchange sites. This is a big mistake. In traffic exchange sites, you win traffic to your own site by visiting others belonging to other members who have signed up at traffic exchange site. While it is true that not everybody who visit from a traffic exchange site will be interested in what you have to offer, some will inevitably be. It is a great way to launch a campaign for increased traffic because it will give your site more exposure. It is like advertising your product on prime time TV. Many will not be interested and even fewer will respond. But tiny fraction that shows interest and ends up buying can be a significant figure. So idea should be to win as much traffic to your site as possible.