ai-readiness
Skill1 starsChanged 3 days agoMITView on GitHub
Check whether AI apps can read and quote a website, and turn each failed check into an exact change. Covers robots.txt rules for AI search crawlers, indexing, snippets, and text that needs JavaScript. Use for "can ChatGPT read my site", "why does AI not cite my site", or an AI crawler audit. Works with a Frontmost AI project, or with only a website address.
4 other projects use a version of this.
Adopted by
Most recently taken up by
- Grubbomatic/crawl-readiness-mcptheir own version, first seen Oct 13, 2026
- nirholas/AuditKittheir own version, first seen Oct 11, 2026
- anthropics/financial-services-pluginstheir own version, first seen Oct 8, 2026
- anthropics/financial-servicestheir own version, first seen Jul 22, 2026
Other versions4 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 4
- anthropics/financial-services-plugins★ 41ksame name
- anthropics/financial-services★ 41ksame name
- nirholas/AuditKit★ 19same name
- Grubbomatic/crawl-readiness-mcp★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository18 other files it gives its agents.
- ai-search-strategySkill
- answer-accuracySkill
- answer-pagesSkill
- community-repliesSkill
- customer-surveySkill
- directory-listingsSkill
- listicle-outreachSkill
- mention-checkSkill
- name-checkSkill
- page-draftingSkill
- placement-trackerSkill
- question-draftingSkill
- question-gapsSkill
- rival-gapsSkill
- scan-reviewSkill
- site-auditSkill
- source-mapSkill
- tactic-checkSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
13 rules. 0 are common, 8 are distinctive, and 5 are very specific to this project.
0
Common
Most of the 3 edited versions we read keep these.
8
Distinctive
This author's own choices. Worth a look.
5
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 8
Use this guidance to find what stops an AI app from reading a website, and to give the exact change for each problem.
- Very specific: line 14
Call get_readiness for the project and the scan in the request.
- Very specific: line 15
If checked is false, the scan has no check.
- Very specific: line 16
Read status: ready, not_verified, or blocked.
- Distinctive: line 17
Read the five checks in the table below.
- Very specific: line 22
If a tool returns setup_required, give the person the url from the result.
- Very specific: line 38
Use the group whose User-agent names the crawler.
- Distinctive: line 40
No robots.txt, or an empty one, allows every crawler.
- Distinctive: line 49
If a fetch fails or a file does not load, mark that check as not verified.
- Distinctive: line 60
Use evidence that you saw, such as "robots.txt line 277".
- Distinctive: line 61
Not verified means that a check did not run.
- Distinctive: line 63
Do not say that a fix will raise citations or mentions.
- Distinctive: line 75
If nothing fails, say that all checks passed, and name the AI apps that can read the site.
“Common” is measured: kept by at least 6 in 10 of the 3 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
AI readiness
Use this guidance to find what stops an AI app from reading a website, and to give the exact change for each problem.
There are two paths. Use the first when the person has a Frontmost AI project. Use the second when they have only a website address.
Path 1: with Frontmost AI
- Call
get_readinessfor the project and the scan in the request. - If
checkedis false, the scan has no check. Say that, and use Path 2. - Read
status:ready,not_verified, orblocked. Readapps_readyofapps_total. - Read the five checks in the table below.
- For each check that fails, copy its evidence and its change from the tool. Make the change exact for this website: name the file, the line, and the new text.
- Each AI app shows how many of its answers cite the site. Report the number with its counts.
- Link the
urlof the AI Readiness tab. The next scan checks the website again.
If a tool returns setup_required, give the person the url from the result. If a tool returns plan_required, say that this data needs a paid Frontmost AI plan. In both cases, continue with Path 2.
Path 2: with only a website address
Fetch two things yourself: https://<site>/robots.txt and the home page, with its response headers. Then do the five checks. Say in the result that this is your own check of two files, made today.
| Check | Passes when | How to check |
| Search crawlers allowed | robots.txt lets each AI app's search crawler read the home page | Test / for OAI-SearchBot (ChatGPT), Googlebot (AI Overviews and AI Mode), PerplexityBot (Perplexity), and Bingbot (Copilot) |
| Gemini allowed | robots.txt does not block Google-Extended | Test / for Google-Extended |
| Home page indexable | The page has no noindex | Read the robots and googlebot meta tags and the X-Robots-Tag header |
| Home page quotable | The page allows a snippet | Fails on nosnippet or max-snippet:0 in those same places |
| Text readable without JavaScript | The HTML that the server sends holds the page's text | Count the words of visible text in the raw HTML. Compare with the page in a browser when you can |
How to read robots.txt (RFC 9309):
- Use the group whose
User-agentnames the crawler. If no group names it, use the*group. - In that group, the longest rule that matches the path wins. If an
Allowand aDisalloware equally long,Allowwins. - No robots.txt, or an empty one, allows every crawler.
- Quote the line number of each rule that you report.
How to read the JavaScript check:
- The crawlers of OpenAI and Perplexity do not run JavaScript. They see only the HTML that the server sends.
- The check fails when a browser shows at least 20 words and the raw HTML holds fewer than half of them.
- If you cannot open the page in a browser, compare the raw HTML with what the page's title and description promise. If the raw HTML is an empty app shell, report it as a probable failure and say that you did not confirm it in a browser.
If a fetch fails or a file does not load, mark that check as not verified. Do not guess.
Order of the changes
- robots.txt rules that block an AI app's search crawler, then a rule that blocks
Google-Extended. These are first because a blocked crawler cannot read any other fix. - A home page that search engines cannot index.
- A home page that does not allow a snippet.
- Text that needs JavaScript to appear.
Rules
- Use evidence that you saw, such as "robots.txt line 277". Do not invent a rule.
- Not verified means that a check did not run. It is not a failure. Say which check did not run, and give no fix for it.
- A search crawler and a training crawler are different.
OAI-SearchBotlets ChatGPT search read the site.GPTBotcollects training data. Whether to allowGPTBotis the owner's choice. Do not list it as a fix. - Do not say that a fix will raise citations or mentions. A readable site is a condition, not a result.
- Path 2 reads the home page only. Say so. A rule that blocks another part of the site can exist.
Never suggest
llms.txtfiles or "AI schema". Google states that its AI features need no new files and no special markup.- A change to a file that you did not see evidence for.
Output
A numbered list in the order above. For each item: the check, the evidence, the exact change, and who usually makes it (the person who manages the website or its host).
If nothing fails, say that all checks passed, and name the AI apps that can read the site.
With Path 2, end with one line: these checks show whether AI apps can read the site. They do not show whether AI apps mention the business. That needs the answers themselves.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

