Canonicals and Duplicate Content

There's no duplicate content penalty. There's a much more mundane problem: Google picks one URL from a set to index, and if you haven't told it which, it picks for you, sometimes wrong, and the ranking signals split across the set in the meantime.

Canonicals are a hint, not a directive. Google overrides them regularly, and when it does, that's information: it disagrees with your declaration for a reason you can usually find.

Pick the right tool

Choosing the wrong mechanism is the actual source of most problems here.

The most consequential confusion is between noindex and robots.txt. Blocking a URL in robots.txt prevents Google reading it, which means it can't see your noindex or your canonical. An already-indexed page blocked in robots.txt stays indexed indefinitely with no snippet. To remove a page: allow the crawl, serve noindex, wait for the recrawl, then block it if you must.

Canonical rules

1. Every indexable page self-canonicalises unless there's a deliberate reason otherwise. This is the default and it's frequently missing.
2. Absolute URLs. Relative canonicals resolve unpredictably.
3. One per page. Two canonical tags means both are ignored.
4. Point at a 200-status, indexable URL. Canonicals to redirects, 404s or noindexed pages are discarded.
5. Don't chain. If A canonicals to B and B canonicals to C, point A at C.
6. Emit server-side. Canonicals injected by JavaScript are unreliable, and invisible to AI retrieval agents that don't render.
7. Keep it consistent with the sitemap, internal links and hreflang. All four should name the same URL. Contradiction is how Google ends up choosing for itself.

<link rel="canonical" href="https://example.com/pricing" />

The standard duplicate set

Every site generates these. Pick one form and enforce it everywhere.

https://example.com/pricing        ← canonical
http://example.com/pricing         → 301
https://www.example.com/pricing    → 301
https://example.com/pricing/       → 301 (or the reverse; just pick one)
https://example.com/Pricing        → 301
https://example.com/pricing?utm_source=x   → canonical to clean URL
https://example.com/index.html     → 301

Protocol, host and trailing slash get redirects, because there's no reason for the variants to exist. Tracking parameters get canonicals, because the URLs need to keep working for the campaigns that use them.

URL parameters

Sort every parameter into one of three groups and handle the whole group consistently.

Tracking (utm_*, gclid, fbclid, ref) — content is identical. Self-canonical to the clean URL. Don't redirect; you'd break attribution.

Content-changing (?page=2, ?category=shoes, ?sort=price) — genuinely different content. Decide per parameter whether the variants should be indexable. Sorting almost never should. Filtering sometimes should. See ecommerce-seo for the faceted-navigation decision tree.

Junk (session IDs, ?view=grid, timestamps) — normalise them away server-side, or block the pattern in robots.txt if they generate an unbounded URL space.

Also normalise parameter order. ?a=1&b=2 and ?b=2&a=1 are two URLs to a crawler unless you canonicalise or sort them.

Pagination

The old rel=next / rel=prev markup is retired. Google no longer uses it.

Current handling:

• Each paginated page self-canonicalises. Page 2 is not a duplicate of page 1 and canonicalising it there deindexes everything on it.
• Don't canonicalise a series to a "view all" unless the view-all page genuinely loads all items and is fast.
• Keep paginated pages crawlable so the items on them get discovered.
• Make sure items are reachable another way too — category pages, tags, a sitemap. Relying on page 40 of a paginated list to get a product indexed is fragile.
• Consider noindex, follow on deep pages if they're generating index bloat, but only if the items are discoverable elsewhere.

Cross-domain and syndication

When your content is republished elsewhere:

• Ask the republisher for a cross-domain canonical pointing at your original. This is the correct answer and the hardest to get.
• Failing that, ask for noindex on their copy.
• Failing that, ask for a prominent link to the original. Weakest, still helps.
• Publish first, and give search engines time to index your version before syndicating.

When you're the republisher, canonical to the original. It's the honest signal and it doesn't cost you what people fear it does.

Diagnosing "Google chose a different canonical"

This report means Google read your declaration and disagreed. It's not a bug to be worked around; it's a signal about which page it thinks is the real one.

Work through:

1. Is the content actually near-identical? If two pages are 95% the same, Google is right to consolidate and the fix is to make them genuinely different or merge them.
2. Do internal links contradict the canonical? If 200 links point at URL A and A canonicals to B, Google weighs the links. Fix the links.
3. Does the sitemap list the non-canonical version?
4. Is the canonical target weaker — slower, thinner, fewer links? Google prefers the stronger page.
5. Do redirects contradict it? A canonical pointing at a URL that redirects elsewhere is self-cancelling.
6. Is the canonical rendered server-side? Check curl, not the browser inspector.

The fix is nearly always making every signal agree, rather than restating the canonical more forcefully.

Verify

# Canonical, robots meta, and final URL for a set of pages
while read -r u; do
  html=$(curl -sL "$u")
  can=$(printf '%s' "$html" | grep -oP '(?<=rel="canonical" href=")[^"]+' | head -1)
  rob=$(printf '%s' "$html" | grep -oP '(?<=name="robots" content=")[^"]+' | head -1)
  printf '%s\n  canonical: %s\n  robots:    %s\n' "$u" "${can:-MISSING}" "${rob:-default}"
done < urls.txt

Anything printing MISSING needs a self-canonical.

Don't do these

• Don't canonicalise different content together. Canonicals are for duplicates. Using them to consolidate genuinely different pages gets ignored, and if it doesn't, you've deindexed a page you wanted.
• Don't canonicalise everything to the homepage. It doesn't consolidate authority; it removes pages from the index.
• Don't combine noindex with a canonical pointing elsewhere. Contradictory, and the noindex may propagate.
• Don't block in robots.txt to deindex. Covered above, and it's the most common mistake in this area.
• Don't worry about a duplicate content penalty. It isn't a thing. The problem is signal splitting and index selection, which is what this skill fixes.

Related skills

technical-seo-audit, redirect-migration-plan, ecommerce-seo, hreflang-international-seo, keyword-cannibalization, nextjs-seo