I've been following
discussion about Google and mirrored information for some time. It is "common knowledge" that Google penalizes page rank when it determines that content is duplicated somewhere else. In fact, I've read many experts stating that there should be no duplicate domain names and no duplicate content anywhere.On
face of it
arguments appear to be sound. Google obviously has several billion pages in it's database and could, it appears, easily determine if content is duplicated. It also seems, again on
face of it, that it's reasonable to check for duplicate content, as this is
"mark of a spammer" and not necessary on
web with hyperlinking available. At least, this is
common wisdom.
However, sometimes what seems reasonable and possible is not: not by a long shot.
Let's begin with
technical side of things. You've got domain x and domain y with exactly
same content. How on earth would Google be able to figure that out? Let's say Google had 3 billion pages in it's database. To compare every page to every page would be an enormous task - quadrillions of comparisons.
Now, if site x had page "page1" which linked to site y which also had "page1", then it would be possible for Google to determine
duplicate content. Conceivably, it could check this out.
Not only is
task enormous, but
benefit is so tiny as to be insignificant. Duplicate content does not imply in any way shape or form spamming. In actual fact, a duplicate site is generally going to lower page rank of BOTH sites. Instead of having 100 links to one site, there will presumably be 50 links to one and 50 to another. This would tend (all things being equal) to lower
page ranking of both sites. So Google gains nothing by this incredible expenditure of resources.
There are several reasons for duplicate content which have nothing to do with spamming. Sometimes
content is actually duplicated, and sometimes it's just that there are several different domains (at least
www and non-www versions) for
same website
Mirroring a site for load balancing - This is very common. The purpose is to split up
traffic between two copies of
site.
Mirroring for region - Sometimes site mirroring is done simply to make it more efficient on
internet backbone itself. You might put an identical copy of a site in Europe, for example, to reduce traffic across
Atlantic, which should make it faster in European countries.
Viral marketing - It's extremely common to allow other sites to republish articles in return for a link.
Different domain names - Sometimes a site might be referenced on many different domain names. You might want to allow
.com, .net and .org versions of
name to all work
same, you might allow for common misspellings or you might cover different keywords (sewing-tips and sewing-secrets are examples of possible combinations).