Duplicate Content Example: Why It Is an Issue for SEO
When the portcullis cut my horse in two, both halves went on drinking at the fountain, and I could not say which one was the original.
The short answer
- What it is
- Duplicate content means two or more URLs show the same text. The match can be across paths, or across http and https or www and non-www versions.
- Why it matters
- Search engines will usually show just one copy, and it may not be the one you want. Links and other signals can also split across the copies.
- How a crawler should report it
- A crawler should group matching pages under one content hash and list every URL in the group. Each row should get a duplicate page warning that names the other URLs.
- Live examples
/lab/duplicate-pages/a/(positive)/lab/duplicate-pages/b/(positive)/lab/duplicate-pages/near-miss/(near miss)
What happens when two URLs serve the same words
Why is having duplicate content an issue for SEO? Search engines want to show one version of a page, so they must choose among the copies. If they pick the wrong URL, the wrong page ranks. Links may point to several versions, which splits their value. Crawlers also spend time on copies when they could fetch new pages.
Most duplicates are accidents. A site may answer on both http and https, or on www and the bare domain. Trailing slashes, tracking parameters, print views, and session IDs can each make a new URL for the same page. A CMS may publish one post under a date path and a category path. Copying a product text across many items adds more.
Browsers do not care. They load each URL and show it. Google groups similar pages into a cluster, picks one as canonical, and tends to show that one. Google has said there is no penalty for plain duplicates. It may act if copying is meant to trick users or game results. So the usual cost is lost control, split signals, and wasted crawls.
To detect exact copies, hash the visible text and title of each page, then group the URLs that share a hash. Also test each page on the other scheme and host. To fix them, choose one URL. Send the others to it with a 301 redirect. If a copy must stay live, add rel="canonical" that points to the main URL. Link only to the main URL, and list only that URL in your sitemap.