The Search Engine Strategies conference and show, sponsored by AltaVista, Search Engine Watch and Internet.com on August 16-17 provided a glimpse of several emerging search trends, biggest trend is toward "Paid Inclusion." The show, held in San Francisco at Fairmont Hotel on Nob Hill, provided a look at where search is headed. Paid inclusion is latest of several revenue producing models considered by search engines as cost of indexing a much larger and more complex web increases and advertising revenue declines. Paid inclusion was started quietly by several previously free search engines as a revenue producing move recently.
Virtually everyone scrambled for new business models when banner advertising effectiveness declined steeply in 2001. Search engines had relied heavily on banner advertising to generate only income they were seeing in sparse times. When tech economy took a nosedive in 2001, justification given by advertising companies for continued banner advertising was now as a "Branding" strategy rather than a sales technique.
To satisfy investor demands for such absurd things as "profit" search engines began probing for new cash infusions and realized that they could no longer rely on advertising to support free submissions. Since nobody was looking at banner ads any more (or at least not clicking through) support for free search listing nearly disappeared.
Who benefits from search engines most? Those who receive traffic from searches. Simple. Who should pay for that traffic? Those who gain that traffic. Simple. How do you charge them? Ah! Now things get more complex.
Goto.com was first to introduce novel idea of charging businesses for traffic generation directly with "Keyword Bids" starting at a penny per click-through. Now companies began to bid for top positions knowing that being at top of list was worth more traffic than being further down page.
Reaction to CPC or cost per click model in 1998 was nearly unanimous from web community, especially while banner advertising revenues were still a viable business model for search engines. It was, "You can't charge for search listings! It'll never fly!" "Searchers won't trust listings they receive on a pay-per-click basis because they are now 'tainted' with commercial results!" At that time, in 1998, nobody believed first, that people would pay for traffic and second, that searchers would trust paid listings.
It very quickly became apparent after decline in revenue from banner advertising in 2000 and dismal performance of banners in delivering traffic that pay-per-click was actually going to work for GoTo. At that point several new pay for performance models were adopted by start-up PPC search engines, and being first to market with idea, GoTo dominated crowd.
It was then that 900 pound gorilla, Yahoo stunned web with a $199 charge for a review of a commercial web site. Wait, not $199 to be listed, but $199 to be LOOKED AT by a reviewer. It was still not a guarantee of inclusion in directory! Search engines started thinking about that and realized that Yahoo had clout to demand money to be reviewed, not listed, but reviewed.
While major search engines stewed on dramatic move by Yahoo, they quickly realized that demanding money to be reviewed would not work for them because they actively sought out sites by sending "spiders" out to "crawl" web and index pages. Search engines wanted to be exhaustive in their coverage of web and catalog entire thing, not just good sites, but ALL of them!
When Google surpassed 1 billion page mark they tooted loudly that they had indexed more of web than anyone else. This was seen as a milestone in search history and became goal of many of top search engines. Let's index entire web! That prospect becomes very expensive for search companies and someone has to pay for it. But how? We can't all become pay-per-click engines if we want to index ALL web, because only a limited number will pay to be listed. So they all stewed about it some more.
Meanwhile, several other directories followed lead of Yahoo and began to charge to be listed. It became accepted at multiple directory sites, notably at LookSmart and NBCi. Several special interest and topical directories had been charging for listings longer, but directories that listed everyone had a harder time justifying those charges. Yahoo can do it becuase entire world knows of Yahoo and wants to be seen there. Vertical portals can charge because they draw a very targeted searcher seeking specific businesses. It's worth paying for that targeted traffic.