Search Engine Robots - How They Work, What They Do (Part II)
Why Isn't My Website In The Search Engine? If your site isn't found in search engines, it is probably because robots couldn't deal with it. It could be something as simple as not being able to find site, or it may be more complicated issues involving robot's not being able to crawl site or figure out what your pages are all about.
Submitting your site to major search engines: that will help deal with "can't find it" problem. Even having links pointing back to your site can be enough to attract search engine robots. Google, for example, suggests that you may not have to submit your pages; they will find your site if you have a link pointing back to it from at least one other site on web.
If robots can find your site but can't make sense of it, then you may need to look at content and technology used on your pages. Frames, Flash, dynamically generated pages, and invalid HTML source code can cause problems when search engine robot tries to access your web pages. While some search engines are beginning to be able to index dynamically generated pages and Flash (e.g. Google and AllTheWeb), use of some of these technologies can hinder your ability to be indexed by search engine robots.
Text in images cannot be read by search engine robots. Using ALT image text is an important way to help robots "read" your images. Websites with extensive images rely heavily on ALT text to present their content.
How Do I Get The Most Out Of Indexing? If you know what to "feed" spidering robots you will help yourself with search engine ranking.
Having a website full of good content is major factor. Search engines exist to serve their visitors, not to rank your website. You need to be sure to present yourself in your site in way that will be most useful to search engine visitor. Each search engine has its own idea of what is important in a page, but they all value text highly. Making sure that text on your pages includes your most important keyword phrases will help search engine evaluate content of those pages.
Making sure that you have good title and meta tags will further assist search engines in understanding what your page is about. If text on page is about widgets, title is about widgets, and meta tags are about widgets, search engine will have a pretty good idea that you are all about widgets. When their visitors search for widgets, search engines know to list your site in results.