crawl-apis
Skill38 starsChanged 9 months agoMITView on GitHub
Scan a target codebase for API definitions (OpenAPI, REST routes, GraphQL schemas) and propose api_contract and api_endpoint registry entries. Presents findings for review before writing files.
In this repository8 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- crawl-dataSkill
- dashboardSkill
- deploySkill
- enterprise-platform-archiSkill
- new-entrySkill
- scaffold-componentSkill
- validateSkill
The file
We split this skill into the rules it gives an agent
10 rules. About 0 are common, 5 are distinctive, and 5 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈5
Distinctive · estimate
This author's own choices. Worth a look.
5
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 19
If no path is provided, ask the user which directory to scan.
- Very specific: line 25
Read models/registry-mapping.yaml to find:
- Distinctive: line 30
Never hardcode paths: Always derive from the YAML.
- Distinctive: line 34
Search the target directory for API-related files.
- Very specific: line 45
Look in: /routes/, /controllers/, /handlers/, /api/
- Distinctive: line 72
Before proposing entries, check existing registry entries:
- Very specific: line 104
Read the _template.md for the target type
- Distinctive: line 106
Write to the correct registry folder
- Very specific: line 132
Always propose as status: draft — never auto-promote to active
- Very specific: line 134
If the scanned codebase is this repo itself, skip catalog-ui/ and docs-site/ (they're the UI, not the modeled system)
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Crawl APIs — Discover and Register API Definitions
Scan a codebase directory for API definitions and propose registry entries.
Arguments
$1— Path to scan (required). Absolute or relative path to the codebase to crawl.--domain <name>— Domain to assign discovered APIs to (optional, will ask if omitted).--write— Write proposed entries to registry immediately (default: preview only).
If no path is provided, ask the user which directory to scan.
Workflow
1. Discover Type-to-Folder Mapping
Read models/registry-mapping.yaml to find:
- The folder path for
api_contractentries - The folder path for
api_endpointentries - The
_template.mdin each folder for frontmatter structure
Never hardcode paths. Always derive from the YAML.
2. Scan for API Definitions
Search the target directory for API-related files. Use these detection patterns:
OpenAPI / Swagger specs:
# Find OpenAPI/Swagger YAML and JSON files- Glob:
**/{openapi,swagger}*.{yaml,yml,json},**/api-spec*.{yaml,yml,json} - Content match: files containing
openapi:orswagger:at the top level
REST route definitions (Node.js/Express/Fastify/Hono):
- Grep for patterns:
router\.(get|post|put|delete|patch),app\.(get|post|put|delete|patch),@Get|@Post|@Put|@Delete(NestJS decorators) - Look in:
**/routes/**,**/controllers/**,**/handlers/**,**/api/**
GraphQL schemas:
- Glob:
**/*.graphql,**/*.gql - Content match: files containing
type Query,type Mutation,schema {
gRPC / Protobuf:
- Glob:
**/*.proto - Content match:
service <Name>blocks
Python (FastAPI/Flask/Django REST):
- Grep for:
@app.route,@router.get|post|put|delete,@api_view,class.*ViewSet
3. Extract API Information
For each discovered API, extract:
| Field | Source |
name | OpenAPI info.title, route prefix, GraphQL type name, proto service name |
description | OpenAPI info.description, JSDoc/docstring if available |
protocol | REST, GraphQL, gRPC, async (inferred from file type) |
endpoints | Individual route paths / operations / queries+mutations |
status | draft (always — human reviews and promotes) |
4. Check for Duplicates
Before proposing entries, check existing registry entries:
# List existing api-contract and api-endpoint entriesCompare by name (case-insensitive). Flag potential duplicates.
5. Present Findings
Show the user a summary table:
**API Discovery Results** — scanned: <path>
Found X API definitions:
| # | Name | Protocol | Endpoints | Status |
|---|------|----------|-----------|--------|
| 1 | User API | REST | 5 routes | NEW |
| 2 | Payment API | GraphQL | 3 queries, 2 mutations | NEW |
| 3 | Billing API | REST | 8 routes | DUPLICATE (exists) |
**Proposed Registry Entries:**
For each NEW API, show the proposed frontmatter:
- 1 `api_contract` entry (logical grouping)
- N `api_endpoint` entries (one per route/operation, or grouped by resource)6. Write Entries (if --write or user confirms)
For each approved entry:
- Generate kebab-case filename from the API name
- Read the
_template.mdfor the target type - Fill in discovered fields, leave unknowns as TBD
- Write to the correct registry folder
- Report what was written
7. Post-Scan Report
**Written X entries:**
- api_contract: registry-v2/<path>/api-name.md
- api_endpoint: registry-v2/<path>/endpoint-name.md
**Next steps:**
1. Review and fill in TBD fields (owner, relationships)
2. Wire relationships: `implements_api_contract`, `parent_software_subsystem`
3. Run `/validate` to check model consistencyDetection Priority
- OpenAPI/Swagger specs (highest confidence — structured, complete)
- GraphQL schema files (high confidence — typed, discoverable)
- Protobuf service definitions (high confidence — typed)
- Framework route definitions (medium confidence — may need human review)
- Generic endpoint patterns (low confidence — present as suggestions)
Notes
- Always propose as
status: draft— never auto-promote to active - Group related endpoints under a single
api_contractwhen they share a prefix/resource - If the scanned codebase is this repo itself, skip
catalog-ui/anddocs-site/(they're the UI, not the modeled system) - Large codebases: limit scan to first 50 API definitions and suggest narrowing the path
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

