seo-technical
Skill0 starsChanged 5 months agoMITView on GitHub
Technical SEO audit across 9 categories: crawlability, indexability, security, URL structure, mobile, Core Web Vitals, structured data, JavaScript rendering, and IndexNow protocol. Use when user says "technical SEO", "crawl issues", "robots.txt", "Core Web Vitals", "site speed", or "security headers".
41 other projects use a version of this, 23 of them edited.
Adopted by
Most recently taken up by
- yotamwiesel/claude-skillsedited, first seen Nov 16, 2026
- mia-cx/skillstheir own version, first seen Nov 15, 2026
- joaoguirunas/team-osedited, first seen Nov 15, 2026
- YerayRodri/claude-skillsedited, first seen Nov 13, 2026
- alpit-patel/seo-aeo-geo-kitedited, first seen Nov 8, 2026
- brainbytes-dev/everything-claude-marketingtheir own version, first seen Nov 1, 2026
Used with
Projects with it also use seo-agentic.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
AgriciDanielSkillseo-agentic
Audit and fix agent readiness: the Lighthouse Agentic Browsing fraction, accessibility tree for agents, robots.txt and Content-Signal for AI agents, WAF treatment of agent traffic, llms.txt, Markdown delivery, ai-catalog.json, /.well-known discovery files, and WebMCP tools. Use when the user asks whether AI agents can use or browse the site, or about llms.txt or WebMCP. Exclude AI citability and brand signals (seo-geo) and commerce protocol depth (seo-ecommerce).
Also in 5 projects that use this
Other versions41 projects use a version of this.
Word for word · 4
- agrivisionai/claude-skills★ 1100% same
- mahiunfiltered/claude-code-skills★ 1100% same
- bbeliso/belisodw-★ 0100% same
- Biznomad/claude-skills★ 0100% same
Edited · 23
- AgriciDaniel/claude-seo★ 30k26% same
- zhtyyx/codex-seo★ 15232% same
- coopersimson96/ai-content-system★ 1794% same
- joaoguirunas/team-os★ 644% same
- 01clauding/claude-seo-skill★ 494% same
- avalonreset/legends-seo-dungeon★ 457% same
- megastep/codex-skills★ 494% same
- realwigu/claude-skills★ 494% same
- alpit-patel/seo-aeo-geo-kit★ 231% same
- get-proofpilot/claude-skills-catalog★ 194% same
13 more not listed
Wrote their own version · 14
- shawnpang/startup-founder-skills★ 331same name
- ronaldships/foundvia-discovery★ 29same name
- Thibaultbm/claude-seo-geo★ 18same name
- ZachArticulateV/designer-pro-and-seo★ 10same name
- brainbytes-dev/everything-claude-marketing★ 9same name
- Maudeunfledged834/startup-founder-skills★ 8same name
- YepAPI/skills★ 8same name
- oegeyilmaz9/seo-aeo-geo-ultimate★ 6same name
- rampstackco/claude-skills-seo★ 5same name
- mia-cx/skills★ 4same name
4 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository73 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- ab-testingSkill
- ad-creativeSkill
- adsSkill
- ai-seoSkill
- analyticsSkill
- asoSkill
- churn-preventionSkill
- cold-emailSkill
- co-marketingSkill
- community-marketingSkill
- competitor-profilingSkill
- competitorsSkill
- content-strategySkill
- copy-editingSkill
- copywritingSkill
- croSkill
- customer-researchSkill
- directory-submissionsSkill
- emailsSkill
- free-toolsSkill
- imageSkill
- launchSkill
- lead-magnetsSkill
- marketing-ideasSkill
- marketing-loopsSkill
- marketing-planSkill
- marketing-psychologySkill
- offersSkill
- onboardingSkill
- paywallsSkill
- popupsSkill
- pricingSkill
- product-marketingSkill
- programmatic-seoSkill
- prospectingSkill
- public-relationsSkill
- referralsSkill
- revopsSkill
- sales-enablementSkill
- schemaSkill
- seo-auditSkill
- signupSkill
- site-architectureSkill
- smsSkill
- socialSkill
- videoSkill
- cyber-neoSkill
- seo-auditSkill
- seo-backlinksSkill
- seo-clusterSkill
- seo-competitor-pagesSkill
- seo-content-briefSkill
- seo-contentSkill
- seo-dataforseoSkill
- seo-driftSkill
- seo-ecommerceSkill
- seo-flowSkill
- seo-geoSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
9 rules. 4 are common, 1 is distinctive, and 4 are very specific to this project.
4
Common
Most of the 34 edited versions we read keep these.
1
Distinctive
This author's own choices. Worth a look.
4
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 67
Recommendation: Consider your AI visibility strategy before blocking.
- Common: line 107
Use PageSpeed Insights API or CrUX data if MCP available
- Common: line 115
Check if content visible in initial HTML vs requires JS
- Distinctive: line 118
Verify dynamic rendering setup if applicable
- Common: line 124
Canonical conflicts: If a canonical tag in raw HTML differs from one injected by JavaScript, Google may use EITHER one.
- Very specific: line 125
noindex with JavaScript: If raw HTML contains <meta name="robots" content="noindex"> but JavaScript removes it, Google MAY still honor the …
- Common: line 132
Check if site supports IndexNow for Bing, Yandex, Naver
- Very specific: line 189
If DataForSEO MCP tools are available, use on_page_instant_pages for real page analysis (status codes, page timing, broken links, on-page …
- Very specific: line 193
If Google API credentials are configured, use python3 scripts/pagespeed_check.py <url> --json for real PSI + CrUX field data (replaces …
“Common” is measured: kept by at least 6 in 10 of the 34 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Technical SEO Audit
Categories
1. Crawlability
- robots.txt: exists, valid, not blocking important resources
- XML sitemap: exists, referenced in robots.txt, valid format
- Noindex tags: intentional vs accidental
- Crawl depth: important pages within 3 clicks of homepage
- JavaScript rendering: check if critical content requires JS execution
- Crawl budget: for large sites (>10k pages), efficiency matters
AI Crawler Management
As of 2025-2026, AI companies actively crawl the web to train models and power AI search. Managing these crawlers via robots.txt is a critical technical SEO consideration.
Known AI crawlers:
| Crawler | Company | robots.txt token | Purpose |
| GPTBot | OpenAI | GPTBot | Model training |
| ChatGPT-User | OpenAI | ChatGPT-User | Real-time browsing |
| ClaudeBot | Anthropic | ClaudeBot | Model training |
| PerplexityBot | Perplexity | PerplexityBot | Search index + training |
| Bytespider | ByteDance | Bytespider | Model training |
| Google-Extended | Google-Extended | Gemini training (NOT search) | |
| CCBot | Common Crawl | CCBot | Open dataset |
Key distinctions:
- Blocking
Google-Extendedprevents Gemini training use but does NOT affect Google Search indexing or AI Overviews (those useGooglebot) - Blocking
GPTBotprevents OpenAI training but does NOT prevent ChatGPT from citing your content via browsing (ChatGPT-User) - ~3-5% of websites now use AI-specific robots.txt rules
Example, selective AI crawler blocking:
# Allow search indexing, block AI training crawlers
User-agent: GPTBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: Bytespider
Disallow: /
# Allow all other crawlers (including Googlebot for search)
User-agent: *
Allow: /Recommendation: Consider your AI visibility strategy before blocking. Being cited by AI systems drives brand awareness and referral traffic. Cross-reference the seo-geo skill for full AI visibility optimization.
2. Indexability
- Canonical tags: self-referencing, no conflicts with noindex
- Duplicate content: near-duplicates, parameter URLs, www vs non-www
- Thin content: pages below minimum word counts per type
- Pagination: rel=next/prev or load-more pattern
- Hreflang: correct for multi-language/multi-region sites
- Index bloat: unnecessary pages consuming crawl budget
3. Security
- HTTPS: enforced, valid SSL certificate, no mixed content
- Security headers:
- Content-Security-Policy (CSP)
- Strict-Transport-Security (HSTS)
- X-Frame-Options
- X-Content-Type-Options
- Referrer-Policy
- HSTS preload: check preload list inclusion for high-security sites
4. URL Structure
- Clean URLs: descriptive, hyphenated, no query parameters for content
- Hierarchy: logical folder structure reflecting site architecture
- Redirects: no chains (max 1 hop), 301 for permanent moves
- URL length: flag >100 characters
- Trailing slashes: consistent usage
5. Mobile Optimization
- Responsive design: viewport meta tag, responsive CSS
- Touch targets: minimum 48x48px with 8px spacing
- Font size: minimum 16px base
- No horizontal scroll
- Mobile-first indexing: Google indexes mobile version. Mobile-first indexing is 100% complete as of July 5, 2024. Google now crawls and indexes ALL websites exclusively with the mobile Googlebot user-agent.
6. Core Web Vitals
- LCP (Largest Contentful Paint): target <2.5s
- INP (Interaction to Next Paint): target <200ms
- INP replaced FID on March 12, 2024. FID was fully removed from all Chrome tools (CrUX API, PageSpeed Insights, Lighthouse) on September 9, 2024. Do NOT reference FID anywhere.
- CLS (Cumulative Layout Shift): target <0.1
- Evaluation uses 75th percentile of real user data
- Use PageSpeed Insights API or CrUX data if MCP available
7. Structured Data
- Detection: JSON-LD (preferred), Microdata, RDFa
- Validation against Google's supported types
- See seo-schema skill for full analysis
8. JavaScript Rendering
- Check if content visible in initial HTML vs requires JS
- Identify client-side rendered (CSR) vs server-side rendered (SSR)
- Flag SPA frameworks (React, Vue, Angular) that may cause indexing issues
- Verify dynamic rendering setup if applicable
JavaScript SEO: Canonical & Indexing Guidance (December 2025)
Google updated its JavaScript SEO documentation in December 2025 with critical clarifications:
- Canonical conflicts: If a canonical tag in raw HTML differs from one injected by JavaScript, Google may use EITHER one. Ensure canonical tags are identical between server-rendered HTML and JS-rendered output.
- noindex with JavaScript: If raw HTML contains
<meta name="robots" content="noindex">but JavaScript removes it, Google MAY still honor the noindex from raw HTML. Serve correct robots directives in the initial HTML response. - Non-200 status codes: Google does NOT render JavaScript on pages returning non-200 HTTP status codes. Any content or meta tags injected via JS on error pages will be invisible to Googlebot.
- Structured data in JavaScript: Product, Article, and other structured data injected via JS may face delayed processing. For time-sensitive structured data (especially e-commerce Product markup), include it in the initial server-rendered HTML.
Best practice: Serve critical SEO elements (canonical, meta robots, structured data, title, meta description) in the initial server-rendered HTML rather than relying on JavaScript injection.
9. IndexNow Protocol
- Check if site supports IndexNow for Bing, Yandex, Naver
- Supported by search engines other than Google
- Recommend implementation for faster indexing on non-Google engines
Agent-Friendly Pages (forward-looking)
AI agents (not just AI summarizers) increasingly read sites through three
channels: vision models on screenshots, raw HTML/DOM, and the accessibility
tree (the cleanest signal). Audit criteria — semantic HTML (real <button>
and <a>, not <div onclick>), label associations, interactive target sizing,
layout stability across templates, cursor: pointer correctness — live in
references/agent-friendly-pages.md.
Audit command
# Render with Playwright + capture accessibility tree, then score
python3 scripts/agent_ux_check.py https://example.com --jsonThe scanner outputs an Agent-UX score (0-100) plus itemized issues:
- HTML findings: real buttons / anchors,
<div onclick>widgets, semantic landmarks, inputs without<label for>, inputs without ARIA labels - Accessibility tree findings: total nodes, interactive nodes, unnamed
interactive elements,
role="generic"ratio
The accessibility-tree snapshot uses Playwright's
page.accessibility.snapshot(interesting_only=False). To capture the tree
without scoring, use python3 scripts/render_page.py <url> --a11y-tree --json.
Surface findings as opportunities, not failures. The standards (WebMCP, agent UX heuristics) are early — don't gate audits on a sub-100 score.
Output
Technical Score: XX/100
Category Breakdown
| Category | Status | Score |
| Crawlability | pass/warn/fail | XX/100 |
| Indexability | pass/warn/fail | XX/100 |
| Security | pass/warn/fail | XX/100 |
| URL Structure | pass/warn/fail | XX/100 |
| Mobile | pass/warn/fail | XX/100 |
| Core Web Vitals | pass/warn/fail | XX/100 |
| Structured Data | pass/warn/fail | XX/100 |
| JS Rendering | pass/warn/fail | XX/100 |
| IndexNow | pass/warn/fail | XX/100 |
Critical Issues (fix immediately)
High Priority (fix within 1 week)
Medium Priority (fix within 1 month)
Low Priority (backlog)
DataForSEO Integration (Optional)
If DataForSEO MCP tools are available, use on_page_instant_pages for real page analysis (status codes, page timing, broken links, on-page checks), on_page_lighthouse for Lighthouse audits (performance, accessibility, SEO scores), and domain_analytics_technologies_domain_technologies for technology stack detection.
Google API Integration (Optional)
If Google API credentials are configured, use python3 scripts/pagespeed_check.py <url> --json for real PSI + CrUX field data (replaces lab-only CWV estimates), python3 scripts/crux_history.py <url> --json for 25-week CWV trends, and python3 scripts/gsc_inspect.py <url> --json for real indexation status per URL.
Error Handling
| Scenario | Action |
| URL unreachable | Report connection error with status code. Suggest verifying URL, checking DNS resolution, and confirming the site is publicly accessible. |
| robots.txt not found | Note that no robots.txt was detected at the root domain. Recommend creating one with appropriate directives. Continue audit on remaining categories. |
| HTTPS not configured | Flag as a critical issue. Report whether HTTP is served without redirect, mixed content exists, or SSL certificate is missing/expired. |
| Core Web Vitals data unavailable | Note that CrUX data is not available (common for low-traffic sites). Suggest using Lighthouse lab data as a proxy and recommend increasing traffic before re-testing. |
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

