llms-txt-generator
Skill1 starsChanged 6 days agoMITView on GitHub
Generate or refresh a site's llms.txt (and, on request, llms-full.txt) from its sitemap, route tree, docs navigation, or content directory, following the llmstxt.org conventions: an H1 with the site name, a blockquote summary, H2 sections of links each with a one-line description taken from the page's own metadata, and an Optional section for secondary pages. Use when a site or docs repository needs the file created, or an existing one checked against the pages that actually exist today.
6 other projects use a version of this.
Adopted by
Most recently taken up by
- jqaisystems/jqai-ai-skillstheir own version, first seen Nov 1, 2026
- solana-foundation/solana-comtheir own version, first seen Oct 8, 2026
- UXMagicai/seosorted-skillstheir own version, first seen Oct 4, 2026
- GianlucaNaarden/claude-skills-nederlandstheir own version, first seen Aug 10, 2026
- famzila/agent-skills-settheir own version, first seen Mar 25, 2026
- akhilkannur/marketing-agent-blueprintstheir own version, first seen Feb 4, 2026
Other versions6 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 6
- solana-foundation/solana-com★ 538same name
- akhilkannur/marketing-agent-blueprints★ 4same name
- jqaisystems/jqai-ai-skills★ 3same name
- famzila/agent-skills-set★ 2same name
- GianlucaNaarden/claude-skills-nederlands★ 1same name
- UXMagicai/seosorted-skills★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository163 other files it gives its agents.
- agents.mdAGENTS.md
- 3d-asset-handoff-sheetSkill
- accessibility-regression-auditorSkill
- adr-writerSkill
- agent-memory-designerSkill
- agent-trajectory-evaluatorSkill
- alerting-rules-tunerSkill
- analysis-memo-writerSkill
- annotation-disagreement-ledgerSkill
- annotation-task-spec-builderSkill
- api-deprecation-plannerSkill
- api-error-contract-designerSkill
- architecture-diagram-generatorSkill
- asset-metadata-review-ledgerSkill
- auth-flow-reviewerSkill
- background-job-reliability-auditorSkill
- branch-rebaserSkill
- brand-voice-profilerSkill
- bundle-analyzerSkill
- cache-policy-designerSkill
- canary-release-plannerSkill
- caption-correction-ledgerSkill
- changelog-from-prsSkill
- chart-chooserSkill
- chunking-strategy-optimizerSkill
- circular-dependency-breakerSkill
- claude-settings-auditorSkill
- code-example-verifierSkill
- cold-email-deliverability-auditorSkill
- cold-start-optimizerSkill
- commit-splitterSkill
- competitor-teardownSkill
- component-spec-writerSkill
- connection-pool-tunerSkill
- content-repurposerSkill
- contract-obligation-registerSkill
- contract-test-designerSkill
- conventional-commitsSkill
- cors-configuratorSkill
- coverage-gap-finderSkill
- crm-export-auditorSkill
- dashboard-designerSkill
- data-retention-auditorSkill
- dataset-first-lookSkill
- dead-code-finderSkill
- deadlock-diagnoserSkill
- decision-log-reconcilerSkill
- dependency-auditSkill
- dependency-upgrade-plannerSkill
- design-brief-writerSkill
- design-critique-checklistSkill
- design-token-extractorSkill
- devcontainer-designerSkill
- distributed-tracing-instrumenterSkill
- dockerfile-optimizerSkill
- email-sequence-drafterSkill
- embedding-index-tunerSkill
- embedding-set-inspectorSkill
- entity-match-candidate-ledgerSkill
- exit-ticket-builderSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
2 rules. 0 are common, 1 is distinctive, and 1 is very specific to this project.
0
Common
Most of the 5 edited versions we read keep these.
1
Distinctive
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 29
Prefer Markdown targets: If the site serves a Markdown rendition of each page (a .md twin, a ?format=md query, or a dedicated route), link …
- Distinctive: line 40
Refresh mode: When the file already exists, parse its links, diff them against the inventory, and rebuild only the lists.
“Common” is measured: kept by at least 6 in 10 of the 5 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
llms.txt is a proposed convention, published at llmstxt.org, for a Markdown file at a site's root that tells a language model what the site is and which pages matter. Writing one by hand goes stale the week after. This skill builds the file from what the site already declares (its sitemap, its route tree, or its docs navigation and content directory), takes every description from the page itself, and writes the result to public/llms.txt or wherever the project serves static files. Run it again later and it refreshes the lists instead of starting over.
When to use this skill
- A docs site or content site has no llms.txt and you want one that matches the real page set.
- The file exists but links to pages that moved, or new sections were never added.
- You are asked for llms-full.txt, the single-file variant that inlines page content rather than linking to it.
- A seo-technical-auditor pass is done and the next step is readability for models.
When NOT to use this skill
- Controlling which AI crawlers may fetch the site: that is
robots.txtand its user-agent rules. llms.txt is a reading guide, not access control. - Deciding whether a page's prose is quotable: the geo-content-auditor skill.
- Writing page descriptions from nothing. The skill copies each page's own description or first sentence and reports pages that have neither; write those first.
Instructions
- Find the inventory. In order of preference: a docs navigation config (
mkdocs.ymlnav, Docusaurussidebars.*, Starlightsidebarinastro.config.*, VitePressthemeConfig.sidebar), because it already encodes the grouping a human chose; thensitemap.xml, static or generated; then the route tree (app/**/page.*in Next.js,src/pagesin Astro) or a content directory with frontmatter. Note the canonical production origin so every link can be absolute. - Write the header.
# <site name>on the first line, then one blockquote (>) of one or two sentences taken from the README or home page, then, if useful, a short paragraph or two of plain prose: what the product is, a versioning note, where to start. No headings inside that prose; llmstxt.org reserves H2 for link sections. - Build the sections. Each
##section is a flat list of- [Title](https://absolute-url): one-line description. Group by what a reader needs (Getting started, Guides, Reference, Examples); for a content site, group by content type. The description comes from frontmatterdescription, the meta description, or the first sentence of the page. Pages with none go into a "needs description" list in your report, not into the file. - Add
## Optional. Per the convention, links here can be skipped when a reader's context is short: changelogs, older versions, legal pages, long reference tables. - Prefer Markdown targets. If the site serves a Markdown rendition of each page (a
.mdtwin, a?format=mdquery, or a dedicated route), link to that; a model reads it better than rendered HTML. - Emit llms-full.txt only when asked. Concatenate the Markdown body of each page in section order, each preceded by its H1 and a
Source: <url>line. Include the main sections, never Optional. If the result runs to several megabytes, say so and trim to the first sections. - Place and serve.
public/llms.txtworks for Next.js, Astro, and most static builders. Use a route handler (app/llms.txt/route.tsin Next.js) when the file must be generated at build time from a content layer. Serve it astext/plain; charset=utf-8ortext/markdown. Verify.
curl -s https://example.com/llms.txt | head -20, count the links, and check that each resolves:rg -o 'https?://[^) ]+' public/llms.txt | sort -u | while read -r u; do printf '%s %s\n' "$(curl -s -o /dev/null -w '%{http_code}' "$u")" "$u" done | rg -v '^200'- Refresh mode. When the file already exists, parse its links, diff them against the inventory, and rebuild only the lists. Keep the hand-written header prose. Report what was added, removed, and moved.
[!NOTE] llmstxt.org is a proposal. Whether any particular assistant or crawler reads the file is up to that vendor, so treat it as a low-cost courtesy for readers that do, not as a ranking lever. AI content and search in 2026 covers where it fits.
[!WARNING] List only pages you want quoted, and describe each one truthfully. A description that promises more than the page delivers is a hallucination you wrote yourself, and a model that trusts the file will repeat it.
[!TIP] Wire the generation into the build so the file cannot drift. The /llms-txt command runs this procedure from inside Claude Code and is the easy thing to call from a
prebuildscript review.
Output
The llms.txt file (and llms-full.txt if requested) written to the static directory, the diff when refreshing, the list of pages that still need a description, and the link-verification result. If any link returned a non-200 status, the report says which and the file keeps the previous good entry.
Example
Generated file for a fictional command-line tool, "Orbit CLI":
# Orbit CLI
> Orbit is a command-line tool for deploying static sites to object storage. These docs cover installation, configuration, and the deploy pipeline.
## Getting started
- [Install](https://orbit.example/docs/install.md): Install with Homebrew, npm, or a prebuilt binary.
- [First deploy](https://orbit.example/docs/first-deploy.md): Deploy a folder in three commands.
## Reference
- [orbit.toml](https://orbit.example/docs/config.md): Every configuration key with its default and type.
## Optional
- [Changelog](https://orbit.example/changelog.md): Release notes back to the first public version.Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

