agentleFS
JakeLabateJakeLabate/Claude-SEO-Skills/skills/sitemap-audit/SKILL.md

sitemap-audit

Skill3 starsChanged 5 months agoMITView on GitHub

Audit a website's XML sitemaps for SEO. Discovers sitemaps from robots.txt, follows sitemap index files, and validates every listed URL to find broken (non-200) URLs, redirecting URLs, noindexed or robots-blocked URLs, URLs canonicalized to a different page, wrong-host or wrong-protocol URLs, duplicate URLs, oversized sitemaps (over 50,000 URLs or 50MB), parse errors, and missing or invalid lastmod/priority/changefreq values. Use when the user asks to audit, analyze, check, validate, or fix an XML sitemap, sitemap index, sitemap.xml, or sitemap coverage of a website.

  • Installs packages

4 other projects use a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

7 rules. 0 are common, 3 are distinctive, and 4 are very specific to this project.

  • 0

    Common

    Most of the 4 edited versions we read keep these.

  • 3

    Distinctive

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use this skill when the user asks to:

    line 12
  2. Distinctive:

    Check that sitemaps are declared in robots.txt and stay under the 50,000-URL / 50MB limits

    line 17
  3. Distinctive:

    Before starting, confirm with the user:

    line 23
  4. Very specific:

    Write a report following the structure in references/report-template.md:

    line 80
  5. Very specific:

    Not in robots.txt: add a Sitemap: line to robots.txt for each sitemap (or the index).

    line 94
  6. Very specific:

    If the user wants the report as a .docx (for example, to share with stakeholders or attach to a ticket), save the Markdown report to a file …

    line 102
  7. Very specific:

    If the user wants the raw findings as a spreadsheet (for filtering, sorting, or triage in Sheets or Excel), convert the audit's …

    line 112

“Common” is measured: kept by at least 6 in 10 of the 4 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Sitemap Audit

Audit a website's XML sitemaps and produce an actionable SEO report.

When to use this skill

Use this skill when the user asks to:

  • Audit, validate, or check an XML sitemap or sitemap index
  • Find sitemap URLs that 404, redirect, are noindexed, or are blocked by robots.txt
  • Find URLs in the sitemap that canonicalize elsewhere or sit on the wrong host/protocol
  • Check that sitemaps are declared in robots.txt and stay under the 50,000-URL / 50MB limits
  • Clean up lastmod, priority, and changefreq metadata
  • Improve the quality and trustworthiness of the URL set submitted to search engines

Inputs to collect

Before starting, confirm with the user:

  1. Site URL or sitemap URL: a live site root (e.g., https://example.com, which auto-discovers sitemaps via robots.txt and common paths) or a specific sitemap URL (--sitemap).
  2. Scope: how many listed URLs to probe (default 2000). Large sitemaps can be sampled with --max-urls.
  3. Probing: whether to fetch each listed URL to check status/canonical/noindex (default on). Use --no-probe for a fast structural-only pass (parse errors, size limits, lastmod hygiene).

Note on the indexable contract: every URL in an XML sitemap should be a final, canonical, indexable 200 page. A URL that redirects, 404s, is noindexed, is blocked by robots.txt, or canonicalizes elsewhere does not belong in the sitemap. Most checks here enforce that contract.

Workflow

Step 1: Discover, parse, and probe the sitemaps

Use scripts/collect_sitemap.py:

python3 scripts/collect_sitemap.py https://example.com --max-urls 2000 --output sitemap_inventory.json

Point directly at a sitemap with --sitemap:

python3 scripts/collect_sitemap.py https://example.com/sitemap_index.xml --sitemap --output sitemap_inventory.json

The script reads robots.txt for Sitemap: directives (falling back to /sitemap.xml and /sitemap_index.xml), follows sitemap index files, handles gzipped sitemaps, and records each sitemap's type, URL count, byte size, and any parse/fetch error. For every listed URL it captures the sitemap's lastmod/changefreq/priority, then probes the URL for status, redirect target, rel=canonical, meta robots, the X-Robots-Tag header, and whether robots.txt disallows it.

Step 2: Run the audit checks

Use scripts/audit_sitemap.py:

python3 scripts/audit_sitemap.py sitemap_inventory.json --output audit_report.json

Step 3: Evaluate the audit checks

Evaluate each check defined in references/audit-checks.md. The core checks are:

CheckSeverity
No parseable sitemap foundHigh
Sitemap parse error / fetch errorHigh
Sitemap exceeds 50,000 URLsHigh
Sitemap exceeds 50MB uncompressedHigh
Listed URL returns 4xx/5xx/errorHigh
Listed URL redirects (not a final 200)High
Listed URL is noindex (meta robots or X-Robots-Tag)High
Listed URL is blocked by robots.txtHigh
Listed URL canonicalizes to a different URLMedium
Listed URL is on the wrong host or protocolMedium
Sitemaps not declared in robots.txtMedium
Duplicate URL across sitemapsLow
Missing / invalid / future lastmodLow
Invalid priority or changefreqLow

Step 4: Produce the report

Write a report following the structure in references/report-template.md:

  1. Summary: sitemaps found, total URLs listed, URLs probed, issue counts by severity
  2. Sitemap files: table of each sitemap: type, URL count, size, gzip, errors
  3. Issues: one section per failing check, listing the URL, the problem, and the fix
  4. Prioritized action list: ordered by severity and impact

Step 5: Recommend fixes

For each issue, give a concrete fix:

  • Non-200, redirecting, noindex, robots-blocked, or canonicalized URLs: remove them from the sitemap (or, for redirects, replace each with its final destination URL). The sitemap should list only final indexable pages.
  • Wrong host/protocol: rewrite URLs to the canonical host and https://.
  • Size limits: split the sitemap into multiple files under 50,000 URLs / 50MB and reference them from a sitemap index.
  • Not in robots.txt: add a Sitemap: line to robots.txt for each sitemap (or the index).
  • lastmod hygiene: emit a valid W3C datetime reflecting the real last-modified date; drop priority/changefreq if they are arbitrary (Google ignores them).
  • Base every recommendation on the observed data; do not invent URLs.

Optional: export the report

As a Word document (.docx)

If the user wants the report as a .docx (for example, to share with stakeholders or attach to a ticket), save the Markdown report to a file and convert it:

python3 scripts/md_to_docx.py report.md --output report.docx

scripts/md_to_docx.py uses only the Python standard library (no pip install) and renders headings, tables, lists, links, bold/italic, and code blocks. Offer this whenever a user asks for a Word doc, a .docx, or a shareable/downloadable report.

As a CSV of findings (.csv)

If the user wants the raw findings as a spreadsheet (for filtering, sorting, or triage in Sheets or Excel), convert the audit's audit_report.json directly:

python3 scripts/findings_to_csv.py audit_report.json --output findings.csv

scripts/findings_to_csv.py is also standard-library only. It writes one row per finding, with check and severity columns prepended and list fields (e.g. the pages sharing a duplicate value) joined with ; . Unlike the .docx, which reformats the written report, the CSV is a direct dump of the structured findings: offer it whenever a user wants the data itself, a spreadsheet, or to slice findings by check or severity.

Resources

  • scripts/md_to_docx.py: convert the Markdown report into a Word (.docx) document (standard library only)
  • scripts/findings_to_csv.py: flatten the audit findings JSON into a CSV, one row per finding (standard library only)
  • references/audit-checks.md: full definitions, thresholds, and rationale for every audit check
  • references/report-template.md: report output structure
  • scripts/collect_sitemap.py: discover, parse, and probe a site's XML sitemaps
  • scripts/audit_sitemap.py: run audit checks against a sitemap inventory JSON file

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.