chunking-strategy-optimizer
Skill1 starsChanged 5 days agoMITView on GitHub
Find the chunking strategy and size that maximizes retrieval quality for a specific corpus, by sweeping configurations against a fixed eval set instead of guessing. Use when RAG answers miss obvious content, when standing up a new corpus, or when picking chunk size/overlap.
In this repository163 other files it gives its agents.
- agents.mdAGENTS.md
- 3d-asset-handoff-sheetSkill
- accessibility-regression-auditorSkill
- adr-writerSkill
- agent-memory-designerSkill
- agent-trajectory-evaluatorSkill
- alerting-rules-tunerSkill
- analysis-memo-writerSkill
- annotation-disagreement-ledgerSkill
- annotation-task-spec-builderSkill
- api-deprecation-plannerSkill
- api-error-contract-designerSkill
- architecture-diagram-generatorSkill
- asset-metadata-review-ledgerSkill
- auth-flow-reviewerSkill
- background-job-reliability-auditorSkill
- branch-rebaserSkill
- brand-voice-profilerSkill
- bundle-analyzerSkill
- cache-policy-designerSkill
- canary-release-plannerSkill
- caption-correction-ledgerSkill
- changelog-from-prsSkill
- chart-chooserSkill
- circular-dependency-breakerSkill
- claude-settings-auditorSkill
- code-example-verifierSkill
- cold-email-deliverability-auditorSkill
- cold-start-optimizerSkill
- commit-splitterSkill
- competitor-teardownSkill
- component-spec-writerSkill
- connection-pool-tunerSkill
- content-repurposerSkill
- contract-obligation-registerSkill
- contract-test-designerSkill
- conventional-commitsSkill
- cors-configuratorSkill
- coverage-gap-finderSkill
- crm-export-auditorSkill
- dashboard-designerSkill
- data-retention-auditorSkill
- dataset-first-lookSkill
- dead-code-finderSkill
- deadlock-diagnoserSkill
- decision-log-reconcilerSkill
- dependency-auditSkill
- dependency-upgrade-plannerSkill
- design-brief-writerSkill
- design-critique-checklistSkill
- design-token-extractorSkill
- devcontainer-designerSkill
- distributed-tracing-instrumenterSkill
- dockerfile-optimizerSkill
- email-sequence-drafterSkill
- embedding-index-tunerSkill
- embedding-set-inspectorSkill
- entity-match-candidate-ledgerSkill
- exit-ticket-builderSkill
- extract-moduleSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
1 rule. About 0 are common, 1 is distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 23
Pick the smallest config that clears the bar: Prefer the configuration with the fewest/smallest chunks that hits your recall target — …
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Chunking is the highest-leverage, most-overlooked knob in retrieval: if the right passage never lands in a single chunk, no reranker or bigger model recovers it. This skill replaces "512 tokens with 50 overlap, because that's what the tutorial said" with a measured choice — sweep candidate strategies over a fixed eval set and pick the one that actually retrieves the answers.
When to use this skill
- Standing up retrieval for a new corpus and you need a defensible chunking default.
- RAG answers miss content you can see exists in the source documents.
- Deciding chunk size, overlap, or strategy (token vs. sentence vs. recursive vs. semantic).
- Migrating embedding models and want to re-confirm chunking still holds up.
Instructions
- Build a retrieval eval set first. Collect 20–50 real questions and, for each, the passage(s) that contain the answer (the "gold" spans). Hand-label if needed — even 20 cases beat eyeballing. This set is the ground truth every configuration is scored against; freeze it.
- Define the candidate configurations. A small grid, not a search of everything: 2–3 strategies (e.g. recursive, sentence, semantic) × 2–3 sizes (e.g. 256 / 512 / 1024 tokens) × overlap (0 / 10–15%). Hold the embedding model and retriever fixed so chunking is the only variable.
- Run each configuration end to end. For each config: chunk the corpus (e.g. with Chonkie), embed the chunks with the fixed model, index them, and run the eval queries.
- Score retrieval, not generation. Report recall@k (does a gold passage appear in the top-k?) and a rank-aware metric like nDCG@k for k ∈ {5, 10, 20}. Generation quality is downstream noise here — measure whether the right chunk is retrieved at all.
- Pick the smallest config that clears the bar. Prefer the configuration with the fewest/smallest chunks that hits your recall target — smaller chunks mean lower embedding cost, lower storage, and tighter prompts. Report the full table so the trade-off is visible.
- Re-check after any upstream change. New embedding model, new document types, or a corpus that grew in a new direction all invalidate the result — re-run the sweep.
[!WARNING] Never tune chunking without a frozen eval set and a baseline number. "The answers look better" is how silent recall regressions ship. If no eval set exists, building one is your first deliverable.
[!TIP] Semantic chunking often wins on heterogeneous prose but costs embeddings at ingestion time; fixed-size recursive chunking is cheaper and frequently close. Let the numbers, not the brochure, decide.
Output
A ranked table of configurations with recall@k and nDCG@k, the recommended configuration with its rationale, and the eval set itself (so the decision is reproducible and re-runnable).
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

