aeo-audit
Skill0 starsChanged 4 days agoMITView on GitHub
Audit whether AI crawlers and answer engines can read a website, with supereo's free MCP tools. Checks robots.txt for each AI crawler (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot) and Google's Google-Extended token, the site's llms.txt, the page's HTML without JavaScript, and what the site serves each crawler's user agent. Use when someone asks whether ChatGPT, Claude or Perplexity can read their site, why AI answers don't cite their pages, or for a technical AEO or GEO audit of a site or page.
4 other projects use a version of this.
Adopted by
Most recently taken up by
- teachskillofskills-ai/DigitalMarketingPro-techshutheir own version, first seen Oct 11, 2026
- ifitsmanu/landing-studiotheir own version, first seen Jul 15, 2026
- mgicoda/icoda-skillstheir own version, first seen Apr 3, 2026
- indranilbanerjee/digital-marketing-protheir own version, first seen Feb 12, 2026
Other versions4 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 4
- indranilbanerjee/digital-marketing-pro★ 934same name
- mgicoda/icoda-skills★ 5same name
- ifitsmanu/landing-studio★ 1same name
- teachskillofskills-ai/DigitalMarketingPro-techshu★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository4 other files it gives its agents.
- ai-visibility-auditSkill
- ai-visibility-resultsSkill
- ship-recommendationSkill
- track-questionsSkill
The file
We split this skill into the rules it gives an agent
2 rules. 0 are common, 0 are distinctive, and 2 are very specific to this project.
0
Common
Most of the 4 edited versions we read keep these.
0
Distinctive
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 15
If check_ai_access, audit_page and fetch_as_bot are among your tools (supereo's MCP server at https://supereo.app/api/mcp, or supereo-audit …
- Very specific: line 22
If it says the person isn't signed in, ask them to run npx -y github:superagnt/supereo-audit login in their own terminal.
“Common” is measured: kept by at least 6 in 10 of the 4 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
AEO audit
Tools: check_ai_access, audit_page and fetch_as_bot, from supereo's MCP server or the supereo-audit CLI, which calls the same three. All three are free.
Find what keeps AI engines from reading a page, with supereo's three free, read-only tools: check_ai_access, audit_page and fetch_as_bot. They read robots.txt, llms.txt and the page itself, and change nothing.
1. Get the tools
- If
check_ai_access,audit_pageandfetch_as_botare among your tools (supereo's MCP server athttps://supereo.app/api/mcp, orsupereo-audit mcp), call them directly. Otherwise run the CLI, which calls the same tools (Node 20 or later):
npx -y github:superagnt/supereo-audit <site or page> --jsonIf it says the person isn't signed in, ask them to run
npx -y github:superagnt/supereo-audit loginin their own terminal. It opens supereo's sign-in in their browser; it's free, and supereo emails a code. Then run the audit again.
2. Run the audit
Audit the page the person asks about, or the home page when they name only a site.
check_ai_accesswith theurlof the page: robots.txt's verdict for each crawler on that path, the rule that decided it, and whether the site serves an llms.txt.audit_pagewith theurl, andquerywhen the page should answer a particular question: the page's HTML without JavaScript, the way crawlers that don't run it read the page, with flags for what keeps AI from reading it.fetch_as_botwith theurl,botandmax_html_charsat 1000 (unless you need the HTML), once per crawler that matters:- ChatGPT's search:
OAI-SearchBotandChatGPT-User(OpenAI trains onGPTBot's fetches) - Claude:
Claude-SearchBotandClaude-User(Anthropic trains onClaudeBot's) - Perplexity:
PerplexityBotandPerplexity-User
- ChatGPT's search:
The CLI runs all three steps at once: by default as GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot, or --bots all for all eight. Each page fetch counts toward supereo's 120 an hour per person, so audit the pages that matter rather than a whole site.
3. Read the results
- A crawler blocked in robots.txt.
ruleis the line that decided it, andgroupthe user agent it sits under (*when no group names the crawler). Blocking a training crawler (GPTBot, ClaudeBot) is a choice some sites make on purpose. Blocking a search crawler (OAI-SearchBot, Claude-SearchBot, PerplexityBot) keeps the page out of that engine's search results, which its answers draw on. The fetchers a person's question triggers (ChatGPT-User, Claude-User, Perplexity-User) follow robots.txt differently from company to company, so read theirfetch_as_botresults too. Google-Extended isn't a crawler but Google's token for Gemini: blocking it keeps the page out of Gemini's training and its grounded answers, not out of Google Search. - Issues from
audit_page:- javascript_shell: the content needs JavaScript, and AI crawlers don't run it. Render the page on the server or ahead of time.
- noindex: the page asks not to be indexed.
- canonical_elsewhere: its canonical points at another URL, so that one is what gets cited.
- ai_crawlers_blocked: robots.txt blocks the crawlers it names.
- robots_unknown: robots.txt couldn't be read, and crawlers assume the worst then.
- no_h1: the page has no H1 heading.
- no_date or not_fresh: no date found on the page (its text, structured data, meta tags or Last-Modified header), or the newest is more than 30 days old. Engines favor pages that show when they were updated.
- not_readable: the page didn't answer 2xx HTML.
fetch_as_bot: a crawler that gets a 4xx or 5xx while others get 200, or much less HTML than the others, is being turned away or sent a challenge page, usually by the CDN's or firewall's bot settings. Onlyfetch_as_botsends a crawler's user agent:audit_pageand the robots.txt and llms.txt reads send an ordinary browser's. supereo fetches from its own address, so a site that checks crawler IP addresses may still answer the real crawler differently.- llms.txt is optional. Say whether it's served, and don't claim it changes how engines rank or cite a page.
4. Report
Lead with what blocks reading, then the rest, each with its evidence from the tools (the rule, the status, the flag) and the fix. Don't guess beyond what the tools returned.
These tools tell you whether AI can read a page. How often ChatGPT, Gemini, Perplexity and Google actually recommend the brand, and who they pick instead, is supereo's free audit: the ai-visibility-audit skill runs it from your agent over supereo's MCP server, and https://supereo.app runs it in a browser (about 2 minutes, no card).
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

