Creating a Robots.txt file

Written by Sumantra Roy


Continued from page 1

For simplicity, let's assume that you are targeting only two keywords: "tourism in Australia" and "travel to Australia". Also, let's assume that you are targeting only three ofrepparttar major search engines: AltaVista, HotBot and Google.

Now, suppose you have followedrepparttar 132364 following convention for namingrepparttar 132365 files: Each page is named by separatingrepparttar 132366 individual words ofrepparttar 132367 keyword for whichrepparttar 132368 page is being optimized by hyphens. To this is addedrepparttar 132369 first two letters ofrepparttar 132370 name ofrepparttar 132371 search engine for whichrepparttar 132372 page is being optimized.

Hence,repparttar 132373 files for AltaVista are

tourism-in-australia-al.html travel-to-australia-al.html

The files for HotBot are

tourism-in-australia-ho.html travel-to-australia-ho.html

The files for Google are

tourism-in-australia-go.html travel-to-australia-go.html

As I noted earlier, AltaVista's spider is called Scooter and Google's spider is called Googlebot.

A list of spiders forrepparttar 132374 major search engines can be found here.

Now, we know that HotBot uses Inktomi and from this list, we find that Inktomi's spider is called Slurp.

Using this knowledge, here's whatrepparttar 132375 robots.txt file should contain:

User-Agent: Scooter Disallow: /tourism-in-australia-ho.html Disallow: /travel-to-australia-ho.html Disallow: /tourism-in-australia-go.html Disallow: /travel-to-australia-go.html

User-Agent: Slurp Disallow: /tourism-in-australia-al.html Disallow: /travel-to-australia-al.html Disallow: /tourism-in-australia-go.html Disallow: /travel-to-australia-go.html

User-Agent: Googlebot Disallow: /tourism-in-australia-al.html Disallow: /travel-to-australia-al.html Disallow: /tourism-in-australia-ho.html Disallow: /travel-to-australia-ho.html

When you putrepparttar 132376 above lines inrepparttar 132377 robots.txt file, you instruct each search engine not to spiderrepparttar 132378 files meant forrepparttar 132379 other search engines.

When you have finished creatingrepparttar 132380 robots.txt file, double-check to ensure that you have not made any errors anywhere in it. A small error can have disastrous consequences - a search engine may spider files which are not meant for it, in which case it can penalize your site for spamming, or, it may not spider any files at all, in which case you won't get top rankings in that search engine.

An useful tool to checkrepparttar 132381 syntax of your robots.txt file can be found here. While it will help you correct syntactical errors inrepparttar 132382 robots.txt file, it won't help you correct any logical errors, for which you will still need to go throughrepparttar 132383 robots.txt thoroughly, as mentioned above.



Article by Sumantra Roy. Sumantra is one of the most respected and recognized search engine positioning specialists on the Internet. For more articles on search engine placement, subscribe to his 1st Search Ranking Newsletter by going to: http://the-easy-way.com/newsletter.html


Creating Your First Web Site, Part 1

Written by Lisa Maliga


Continued from page 1

Contact Information Who are you and where are you located? Proving you’re not a fly-by-net operation is easier if you have provided a name, street address, phone number, toll-free is a huge bonus, and fax number. Also, make absolutely certain that your e-mail address is easy to locate.

Testimonials Sell Products Use real testimonials, with real names of people living in actual cities and towns acrossrepparttar world. People who have tried your product/service and have been impressed enough to jot down a line or two…these missives should be displayed on your main page in order to create an element of trust. You can also have a testimonials page. Always askrepparttar 132363 person if it’s okay to use their testimonial.

Speed Sells Your site must be able to load quickly – check outrepparttar 132364 home page for Google and see how fast that loads. Please keep in mind that not everyone with an Internet connection is able to get online via cable modem or DSL. Some people still have dial-up, and are also charged byrepparttar 132365 minute so for them a quickly loading web page is important.

Hosting It Since you already have a reliable ISP such as AOL, MSN, Earthlink, Adelphia, Yahoo, etc., your next step is to find a place where you can put your web site out there. The above-mentioned ISP’s provide web hosting for various prices, but you are advised to always compare hosts. Make sure you readrepparttar 132366 reviews and check out: http://www.webhostingratings.com Here arerepparttar 132367 six things you should look for when considering a hosting company.

1.Monthly cost – If you’re shopping around for a free web site, be aware ofrepparttar 132368 number of limitations and restrictions you’ll encounter. Oftentimes free means fee, but that’s hidden inrepparttar 132369 fine print [if at all] onrepparttar 132370 hosting company’s site. As in everything, you get what you pay for. 2.Setup Fee – Sometimes a hosting company will charge a small one-time fee, other times they will have a rather large fee. The more services you require such as: lots of e-mail addresses, shopping cart, streaming audio/video, extra disk space, etc.,repparttar 132371 higherrepparttar 132372 amount. 3.Disk Space – How many MB [millions of bytes] will you need? You can get as few as 5 MB and as many as 1000 for less than $20 per month. 4.Transfer Rate – GB [billions of bytes] of pictures, text, etc. you will be allowed to upload onto your site per month. It also applies to what a visitor on your site will be viewing and if you have an exceptionally high rate of traffic you will have to pay for it. 5.Technical support availability – Whether by e-mail, online chat or toll-free number, this is a service that many consider vital, especially if they have no experience with web sites. 6.Good uptime – If your host is on downtime that means your site will be unavailable to anyone. Look for a host with no less than 99% uptime

In Creating Your First Web Site, Part 2, you will learn about getting your site online and intorepparttar 132373 search engines and directories, along with ways to promote it and exchange links.

http://www.lisamaliga.com ~ Lisa’s Library of Writing offers advice on publishing, adult figure skating, aromatherapy, web design, and Internet promotion. This is the home of fiction and nonfiction writer Lisa Maliga.


    <Back to Page 1
 
ImproveHomeLife.com © 2005
Terms of Use