embedding-set-inspector
Skill1 starsChanged 5 days agoMITView on GitHub
Diagnose the health of an embedding set before blaming the retriever — checking normalization, dimensionality, near-duplicates, degenerate vectors, and corpus/query distribution mismatch. Use when retrieval quality is poor, after a re-embed, or before shipping a new index.
In this repository163 other files it gives its agents.
- agents.mdAGENTS.md
- 3d-asset-handoff-sheetSkill
- accessibility-regression-auditorSkill
- adr-writerSkill
- agent-memory-designerSkill
- agent-trajectory-evaluatorSkill
- alerting-rules-tunerSkill
- analysis-memo-writerSkill
- annotation-disagreement-ledgerSkill
- annotation-task-spec-builderSkill
- api-deprecation-plannerSkill
- api-error-contract-designerSkill
- architecture-diagram-generatorSkill
- asset-metadata-review-ledgerSkill
- auth-flow-reviewerSkill
- background-job-reliability-auditorSkill
- branch-rebaserSkill
- brand-voice-profilerSkill
- bundle-analyzerSkill
- cache-policy-designerSkill
- canary-release-plannerSkill
- caption-correction-ledgerSkill
- changelog-from-prsSkill
- chart-chooserSkill
- chunking-strategy-optimizerSkill
- circular-dependency-breakerSkill
- claude-settings-auditorSkill
- code-example-verifierSkill
- cold-email-deliverability-auditorSkill
- cold-start-optimizerSkill
- commit-splitterSkill
- competitor-teardownSkill
- component-spec-writerSkill
- connection-pool-tunerSkill
- content-repurposerSkill
- contract-obligation-registerSkill
- contract-test-designerSkill
- conventional-commitsSkill
- cors-configuratorSkill
- coverage-gap-finderSkill
- crm-export-auditorSkill
- dashboard-designerSkill
- data-retention-auditorSkill
- dataset-first-lookSkill
- dead-code-finderSkill
- deadlock-diagnoserSkill
- decision-log-reconcilerSkill
- dependency-auditSkill
- dependency-upgrade-plannerSkill
- design-brief-writerSkill
- design-critique-checklistSkill
- design-token-extractorSkill
- devcontainer-designerSkill
- distributed-tracing-instrumenterSkill
- dockerfile-optimizerSkill
- email-sequence-drafterSkill
- embedding-index-tunerSkill
- entity-match-candidate-ledgerSkill
- exit-ticket-builderSkill
- extract-moduleSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 3 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 13
After re-embedding a corpus (new model, new chunking) and before promoting the index.
- Distinctive: line 19
Confirm the basics: Verify every vector has the expected dimensionality and that vectors are normalized if your distance metric assumes it …
- Distinctive: line 20
Check for asymmetry handling: If the model supports input types (document vs.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
When retrieval is poor, teams reach for a bigger model or a reranker before checking whether the embeddings themselves are sound. This skill inspects an embedding set for the failure modes that quietly wreck recall, so you fix the cause instead of layering patches on top.
When to use this skill
- Retrieval recall is low and you want to rule out the embeddings before tuning the retriever.
- After re-embedding a corpus (new model, new chunking) and before promoting the index.
- A subset of documents is "invisible" to search no matter the query.
- Validating a freshly built index in CI before it ships.
Instructions
- Confirm the basics. Verify every vector has the expected dimensionality and that vectors are normalized if your distance metric assumes it (cosine vs. dot product vs. L2 mismatch is a classic silent bug). Flag any zero, NaN, or near-zero-norm vectors — usually empty or failed-to-embed chunks.
- Check for asymmetry handling. If the model supports input types (document vs. query), confirm documents were embedded as documents and queries as queries. Mixing them degrades retrieval and is easy to get wrong.
- Profile the distribution. Summarize pairwise similarity: if almost everything is highly similar to everything else, the embeddings are not discriminating (often over-large chunks or a domain mismatch). If clusters are extreme, check for duplicated or boilerplate content dominating the space.
- Find near-duplicates. Detect chunks whose embeddings are near-identical — repeated headers/footers, navigation, or licence text — which crowd out real answers in the top-k. Recommend dedup or metadata filtering.
- Test query/document alignment. Embed a handful of the eval queries and confirm their nearest neighbours are plausible. A systematic mismatch (queries land far from all documents) points to a model or input-type problem, not a tuning problem.
- Report and recommend. Summarize findings as
severity | issue | affected count | fix, ordered by impact on retrieval.
[!NOTE] Embeddings from different models are not comparable. Never mix vectors from two models in one index, and re-embed the whole corpus when you switch — see Choosing Embeddings in 2026.
[!WARNING] A normalization or distance-metric mismatch can make retrieval look "sort of working" while quietly tanking recall. Check it first — it is the single most common embedding bug.
Output
A health report: dimensionality/normalization status, count of degenerate vectors, near-duplicate clusters, distribution summary, query-alignment spot checks, and a prioritized list of fixes.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

