GOOGLEBOT WON'T GO HOMEI have 'Googlebot' crawl my site every day like a dispossessed spirit that can't leave.
It was not always like this, I would go for a month or more before he came to my site and then would only crawl a few pages and leave again.
This has been going on now for two months or more, I've been watching
stats on my server everyday seeing just where he goes. He will crawl a few pages one day, and
next some other pages, and then finally leave until
morrow. After just three or four weeks, I noticed I got an infestation of lesser bots calling as well, some I have never heard of before, they seem to follow 'Googlebot' as though he is
Pied Piper of Hamelin.
What caused this infestation of bots?
Well, I got interested in rss feeds, I thought "What a good tool for drawing traffic to my site."
So I studied up on rss feeds until I understood what was involved. Now I am no 'techie', and
most programming I do is html in 'Notepad', making my own web pages.
Rss feeds use a similar language called xml, so it was easy to understand for me, and I was able to quickly make up an rss feed page in 'Notepad', placed it on my server, downloaded a 'feed reader' software, opened it up, placed
address of my feed in
'feed reader', and up came my rss feed in
reader. It was so easy I wrote an ebook to tell everyone how to do it, it's called "Really Simple RSS".
There are many rss directories and syndication services that I submitted
address of my rss feed to, so other people can look at
feed in their 'feed reader' and see what articles I am posting.
Some of
'subscribers' to
feed actually come and visit my site to see what it's all about, which happened almost straight away after I submitted my feed.
In less than 36 hours, 'Googlebot' found my feed!
In a little over two days later Yahoo's 'Slurp' bot found my feed, and from there on they have both been crawling my site with
lesser bots every single day.