benchmark
Skill24 starsChanged 19 days agoMITView on GitHub
Benchmark your RAG pipeline's performance
95 other projects use a version of this.
Adopted by
Most recently taken up by
- akshaypimprikar/pragmatheir own version, first seen Nov 11, 2026
- Rojios/everything-claude-codetheir own version, first seen Nov 8, 2026
- accso/ai-context-mcptheir own version, first seen Nov 8, 2026
- timurgaleev/vibestacktheir own version, first seen Nov 8, 2026
- pankaj-mahaur/Agent-Daemontheir own version, first seen Nov 6, 2026
- datafusion-contrib/datafusion-distributedtheir own version, first seen Nov 2, 2026
Used with
Projects with it also use ios-clean, ios-fix and 10 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
Saqlain-811Skillios-clean
Remove the DebugBridge SPM package and all #if DEBUG wiring from an iOS app. (gstack)
Also in 5 projects that use this
Saqlain-811Skillios-fix
Autonomous iOS bug fixer. (gstack)
Also in 5 projects that use this
seek888Skillpair-agent
Pair a remote AI agent with your browser. One command generates a setup key and prints instructions the other agent can follow to connect. Works with OpenClaw, Hermes, Codex, Cursor, or any agent that can make HTTP requests. The remote agent gets its own tab with scoped access (read+write by default, admin on request). Use when asked to "pair agent", "connect agent", "share browser", "remote browser", "let another agent use my browser", or "give browser access". (gstack) Voice triggers (speech-to-text aliases): "pair agent", "connect agent", "share my browser", "remote browser access".
Also in 11 projects that use this
ShreyRaviSkilllanding-report
Read-only queue dashboard for workspace-aware ship. Shows which VERSION slots are currently claimed by open PRs, which sibling Conductor workspaces have WIP work likely to ship soon, and what slot /ship would pick next. No mutations — just a snapshot. Use when asked to "landing report", "what's in the queue", "show me open PRs", or "which version do I claim next". (gstack)
Also in 8 projects that use this
timurgaleevSkilldocument-generate
Generate missing docs for a feature, module or project as tutorials, how-tos, reference and explanation.
Also in 7 projects that use this
seek888Skillopen-gstack-browser
Launch GStack Browser — AI-controlled Chromium with the sidebar extension baked in. Opens a visible browser window where you can watch every action in real time. The sidebar shows a live activity feed and chat. Anti-bot stealth built in. Use when asked to "open gstack browser", "launch browser", "connect chrome", "open chrome", "real browser", "launch chrome", "side panel", or "control my browser". Voice triggers (speech-to-text aliases): "show me the browser".
Also in 8 projects that use this
- +1 more from Saqlain-811
Other versions95 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 95
- garrytan/gstack★ 142ksame name
- dragonflydb/dragonfly★ 32ksame name
- xtermjs/xterm.js★ 21ksame name
- udecode/plate★ 17ksame name
- Z3Prover/z3★ 13ksame name
- androidx/androidx★ 6.1ksame name
- samchon/typia★ 6ksame name
- PaddlePaddle/FastDeploy★ 3.7ksame name
- zalando/skipper★ 3.3ksame name
- samchon/nestia★ 2.2ksame name
85 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository23 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- architectureSkill
- audit-contentSkill
- break-itSkill
- buildSkill
- challengeSkill
- code-reviewSkill
- compareSkill
- debug-ragSkill
- evaluateSkill
- explainSkill
- fixSkill
- glossarySkill
- journalSkill
- lessonSkill
- paper-reviewSkill
- quizSkill
- recapSkill
- roadmapSkill
- sandboxSkill
- startSkill
- triageSkill
The file
We split this skill into the rules it gives an agent
8 rules. 0 are common, 5 are distinctive, and 3 are very specific to this project.
0
Common
Most of the 81 edited versions we read keep these.
5
Distinctive
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 17
Look for a learner profile at progress/learner-profile.md and for code in src/ and projects/.
- Very specific: line 18
If pipeline code exists in projects/, great — proceed.
- Very specific: line 19
If no pipeline or RAG code exists anywhere in projects/ or src/, guide them warmly:
- Distinctive: line 22
Stop here — do not continue to Step 2.
- Distinctive: line 23
If a pipeline is found, catalog the pipeline components: loader, chunker, embedder, vector store, retriever, generator.
- Distinctive: line 27
Ask the learner what they want to measure, or suggest a comprehensive benchmark covering:
- Distinctive: line 55
Configure warm-up runs: Run 5-10 queries before measuring to avoid cold-start effects.
- Distinctive: line 63
Run the full query set through the pipeline
“Common” is measured: kept by at least 6 in 10 of the 81 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Benchmark: Measure Your RAG Pipeline's Performance
Scope: This skill focuses on operational performance (latency, throughput, token costs, memory). For answer quality metrics (faithfulness, relevancy, correctness), use
/evaluate.
Set up and run comprehensive benchmarks on the learner's RAG pipeline to identify bottlenecks and establish performance baselines.
Step 1: Identify the Pipeline
Welcome! Let's measure how your RAG pipeline performs under the hood.
First, check whether the learner has existing work to benchmark:
- Look for a learner profile at
progress/learner-profile.mdand for code insrc/andprojects/. - If pipeline code exists in
projects/, great — proceed. If multiple pipelines exist, ask which to benchmark. If no pipeline or RAG code exists anywhere in
projects/orsrc/, guide them warmly:"Looks like you don't have a pipeline to benchmark yet — no worries! Let's get you set up first. Run
/buildto create your first RAG pipeline, and once it's running you can come back here to see exactly where the time and tokens go. It won't take long!"Stop here — do not continue to Step 2.
- If a pipeline is found, catalog the pipeline components: loader, chunker, embedder, vector store, retriever, generator.
Step 2: Define Benchmark Scope
Ask the learner what they want to measure, or suggest a comprehensive benchmark covering:
Latency Metrics
- End-to-end latency: Total time from query to answer
- Retrieval latency: Time to embed the query and fetch results from the vector store
- Generation latency: Time for the LLM to produce the answer
- Embedding latency: Time to embed a single query or a batch of documents
Throughput Metrics
- Queries per second: How many queries can the pipeline handle?
- Indexing throughput: How fast can documents be chunked, embedded, and stored?
Quality Metrics
- Retrieval accuracy: Precision@k, Recall@k, MRR (Mean Reciprocal Rank)
- Answer quality: Faithfulness, relevancy (via RAGAS or custom evaluators)
Resource Metrics
- Token usage: Tokens consumed per query (embedding + generation)
- Memory footprint: RAM usage of the vector store and pipeline components
- Cost estimate: Approximate cost per query for paid APIs
Step 3: Prepare the Benchmark Suite
Help the learner set up the benchmarking infrastructure:
- Create a test query set: 20-50 queries of varying complexity, representative of real usage.
- Establish ground truth (for quality metrics): Expected answers and relevant source documents.
- Set up timing instrumentation: Wrap each pipeline stage with timing code.
- Configure warm-up runs: Run 5-10 queries before measuring to avoid cold-start effects.
Provide reusable benchmark code that the learner can save and run again later.
Step 4: Run the Benchmarks
Execute the benchmarks and collect data:
- Run the full query set through the pipeline
- Record all metrics for each query
- Compute statistics: mean, median, p95, p99 for latency; averages for quality
Display a progress indicator as benchmarks run.
Step 5: Present Results
Display results in clear tables:
Performance Benchmark Report
============================
Pipeline: [name]
Queries: [N]
Date: [today]
Latency (ms):
Stage Mean Median P95 P99
Retrieval [...] [...] [...] [...]
Generation [...] [...] [...] [...]
End-to-end [...] [...] [...] [...]
Quality:
Precision@5: [score]
Recall@5: [score]
MRR: [score]
Faithfulness: [score]
Resources:
Avg tokens/query: [N]
Est. cost/query: $[X]Step 6: Identify Bottlenecks
Analyze the results and highlight:
- Which stage is the biggest latency contributor?
- Are there any outlier queries that are much slower?
- Where is quality weakest?
- Is cost per query sustainable at target scale?
Step 7: Optimization Suggestions
Provide prioritized recommendations:
- Quick wins: Changes that take < 30 minutes and yield measurable improvement
- Medium effort: Changes that require refactoring but have significant impact
- Architectural changes: Bigger changes for when the pipeline needs to scale
Examples: add caching, batch embeddings, switch to a faster embedding model, add a re-ranker, optimize chunk size.
Step 8: Save the Report
Save the benchmark report to projects/[pipeline-name]/eval/benchmarks/report-[date].md. Encourage the learner to re-run after optimizations to measure improvement.
Suggest 2-3 relevant next steps using slash commands:
/evaluate— measure answer quality metrics alongside your performance benchmarks/debug-rag— diagnose and fix the bottlenecks identified in your benchmark/architecture— redesign your pipeline architecture if fundamental changes are needed
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

