llms-txt
Skill8 starsChanged 8 months agoMITView on GitHub
Generates llms.txt and llms-full.txt files following the llmstxt.org specification. Provides LLM-friendly content curation for AI coding assistants (Cursor, Windsurf, Claude Code) and AI search engines. Use when generating or updating llms.txt for a repository.
14 other projects use a version of this.
Adopted by
Most recently taken up by
- sitetiller/ai-visibility-skillstheir own version, first seen Nov 15, 2026
- 6t9xstar/one-step-seotheir own version, first seen Nov 8, 2026
- genericService/claude-skillstheir own version, first seen Nov 8, 2026
- akii-technologies-ltd/akii-seo-ai-search-optimizertheir own version, first seen Nov 1, 2026
- Anas660/claude-seo-desktheir own version, first seen Nov 1, 2026
- aelena/claude-skillstheir own version, first seen Oct 26, 2026
Other versions14 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 14
- akii-technologies-ltd/akii-seo-ai-search-optimizer★ 80same name
- jsath/claude-skills★ 14same name
- 6t9xstar/one-step-seo★ 5same name
- Anas660/claude-seo-desk★ 3same name
- AgentCTO/claude-skills★ 2same name
- jstanx/aeo-toolkit★ 2same name
- marcelo-earth/llms-txt-generator★ 2same name
- genericService/claude-skills★ 1same name
- howseen-ai/howseen-geo-skills★ 1same name
- try-era/geo-skills★ 1same name
4 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository20 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- .github/copilot-instructions.mdCopilot instructions
- llms-full.txtllms.txt
- llms.txtllms.txt
- api-referenceSkill
- changelogSkill
- doc-refreshSkill
- docs-verifySkill
- feature-benefitsSkill
- geo-optimisationSkill
- launch-artifactsSkill
- package-registrySkill
- pitchdocs-suiteSkill
- platform-profilesSkill
- public-readmeSkill
- roadmapSkill
- user-guidesSkill
- visual-standardsSkill
- pitchdocsSkill
The file
We split this skill into the rules it gives an agent
8 rules. 0 are common, 8 are distinctive, and 0 are very specific to this project.
0
Common
Most of the 14 edited versions we read keep these.
8
Distinctive
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 31
No other heading levels are used: Only H1 (one, at the top) and H2 (for sections).
- Distinctive: line 44
Read the primary manifest for the project name and description:
- Distinctive: line 56
Check for these files and directories:
- Distinctive: line 89
Use the feature-benefits approach: describe what the reader gains from reading that file.
- Distinctive: line 188
After adding or removing documentation files
- Distinctive: line 189
After major version releases
- Distinctive: line 190
After restructuring the docs directory
- Distinctive: line 191
Add to the release checklist alongside CHANGELOG updates
“Common” is measured: kept by at least 6 in 10 of the 14 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
llms.txt Generator
Generate structured, LLM-friendly content indexes following the llmstxt.org specification.
Background
llms.txt was proposed by Jeremy Howard (Answer.AI) in September 2024. It provides a curated Markdown file that gives LLMs a structured map of a project's most important content — solving the problem of context windows being too small to process entire websites or repositories.
Adopted by: Anthropic, Cloudflare, Stripe, Vercel, Cursor, Mintlify, GitBook, Fern.
Used by: Cursor, Windsurf, Context7 MCP, Claude Code (reading local files), AI search engines.
Specification (llmstxt.org)
An llms.txt file is a Markdown document with sections in this exact order:
- H1 heading (required) — the name of the project or site
- Blockquote (optional) — short summary with key information for understanding the rest of the file
- Body text (optional) — zero or more Markdown sections of any type except headings
- H2 sections with file lists (optional) — each contains a Markdown list where every item has:
- A required hyperlink:
[name](url) - Optionally a
:followed by notes about the file
- A required hyperlink:
## Optionalsection (special) — URLs here can be skipped when shorter context is needed
No other heading levels are used. Only H1 (one, at the top) and H2 (for sections).
Two Output Files
| File | Content | Size Target | Use Case |
llms.txt | Index with links and descriptions | Under 10K tokens | Real-time AI assistants navigating quickly |
llms-full.txt | Concatenated Markdown of all referenced files | Varies (can be 100K+ tokens) | RAG ingestion, IDE indexing, full-context tools |
Generation Workflow
Step 1: Gather Project Metadata
Read the primary manifest for the project name and description:
| File | Name Field | Description Field |
package.json | name | description |
pyproject.toml | [project].name | [project].description |
Cargo.toml | [package].name | [package].description |
go.mod | module path | First line of README |
.claude-plugin/plugin.json | name | description |
Step 2: Scan for Documentation Files
Check for these files and directories:
Primary docs:
README.mddocs/directory (hub page, guides, API reference)examples/directory
Supporting docs:
CONTRIBUTING.mdCHANGELOG.mdSECURITY.mdCODE_OF_CONDUCT.mdROADMAP.mdLICENSE
Code entry points (include only if the project is a library/framework):
src/index.*orlib/index.*- Config files with schema documentation
Step 3: Write Descriptive Annotations
For each file, write a benefit-focused description — not just the file name:
Good:
- [Getting Started](./docs/guides/getting-started.md): Install, configure, and deploy your first worker in under 5 minutesBad:
- [Getting Started](./docs/guides/getting-started.md): Getting started guideUse the feature-benefits approach: describe what the reader gains from reading that file.
Step 4: Assemble llms.txt
For repositories (local paths):
# [Project Name]
> [Description from manifest or README first paragraph]
[Optional body text: language, framework, key technical context]
## Docs
- [README](./README.md): Project overview, value proposition, and quick start
- [API Reference](./docs/api.md): Complete endpoint documentation with authentication and error codes
- [Configuration](./docs/configuration.md): All config options with defaults and examples
## Guides
- [Getting Started](./docs/guides/getting-started.md): Install, configure, and run your first example in under 5 minutes
- [Deployment](./docs/guides/deployment.md): Production deployment to Docker, AWS Lambda, and Cloudflare Workers
## Examples
- [Basic Usage](./examples/basic/): Minimal working examples for common use cases
- [Advanced Patterns](./examples/advanced/): Complex integrations and performance optimisation
## Optional
- [Changelog](./CHANGELOG.md): Version history with user-facing change descriptions
- [Contributing](./CONTRIBUTING.md): Development setup, coding standards, and PR workflow
- [Code of Conduct](./CODE_OF_CONDUCT.md): Community behaviour standards (Contributor Covenant v3.0)
- [Security](./SECURITY.md): Vulnerability reporting process and response timeline
- [License](./LICENSE): MIT license termsFor documentation sites (full URLs):
# [Project Name]
> [Description]
## Docs
- [Getting Started](https://docs.example.com/getting-started): Installation and first steps
- [API Reference](https://docs.example.com/api): Complete API documentation
## Optional
- [Changelog](https://docs.example.com/changelog): Version history
- [Contributing](https://github.com/org/repo/blob/main/CONTRIBUTING.md): How to contributeStep 5: Generate llms-full.txt (if requested)
Concatenate all referenced files in the same order as llms.txt, with clear separators:
# [Project Name] — Full Documentation
> Complete documentation content for LLM ingestion.
---
## README.md
[Full contents of README.md]
---
## docs/api.md
[Full contents of docs/api.md]
---
[Continue for all referenced files, excluding the Optional section unless specifically requested]Size management:
- Skip binary files (images, PDFs)
- For very large files (>50K tokens), include only the first section or a summary
- Note the total token count at the end of the file
When to Generate
| Project Type | llms.txt | llms-full.txt |
| Public repo with docs site | Always | Always (host on docs site) |
| Public GitHub repo | Recommended | Optional (large repos benefit) |
| Claude Code plugin | Recommended | Optional |
| Small utility / internal tool | Optional | Skip |
Regeneration
llms.txt should be updated when documentation changes significantly:
- After adding or removing documentation files
- After major version releases
- After restructuring the docs directory
- Add to the release checklist alongside CHANGELOG updates
Real-World Examples
| Project | llms.txt | llms-full.txt | Notable Pattern |
| Anthropic | docs.anthropic.com/llms.txt (~8K tokens) | 481K tokens | Organised by product area |
| Cloudflare | developers.cloudflare.com/llms.txt | Per-product files (~3.7M total) | Product-specific full files |
| Stripe | docs.stripe.com/llms.txt | Yes | Uses Optional for niche products |
| Vercel | vercel.com/docs/llms.txt | ~400K words | Multi-product structure |
Specification Reference
- Spec: llmstxt.org
- Creator: Jeremy Howard, Answer.AI
- Reference repo: AnswerDotAI/llms-txt
- Directory: llms-txt-hub
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

