benchmark
Skill4 starsChanged 5 days agoMITView on GitHub
Run a fixed canary feature through the pipeline and record objective metrics (turns, tool errors, tokens, time) to compare against a baseline after a pipeline change. Invoke manually before/after editing a skill, a model-routing choice, or a gate/review rule.
98 other projects use a version of this.
Adopted by
Most recently taken up by
- TsukiNi22/skillstheir own version, first seen Nov 16, 2026
- TsukiNi22/ai-utilstheir own version, first seen Nov 16, 2026
- AndresArreguy/ecc-pi-porttheir own version, first seen Nov 16, 2026
- Rojios/everything-claude-codetheir own version, first seen Nov 8, 2026
- accso/ai-context-mcptheir own version, first seen Nov 8, 2026
- timurgaleev/vibestacktheir own version, first seen Nov 8, 2026
Used with
Projects with it also use ios-clean, ios-fix and 10 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
Saqlain-811Skillios-clean
Remove the DebugBridge SPM package and all #if DEBUG wiring from an iOS app. (gstack)
Also in 5 projects that use this
Saqlain-811Skillios-fix
Autonomous iOS bug fixer. (gstack)
Also in 5 projects that use this
seek888Skillpair-agent
Pair a remote AI agent with your browser. One command generates a setup key and prints instructions the other agent can follow to connect. Works with OpenClaw, Hermes, Codex, Cursor, or any agent that can make HTTP requests. The remote agent gets its own tab with scoped access (read+write by default, admin on request). Use when asked to "pair agent", "connect agent", "share browser", "remote browser", "let another agent use my browser", or "give browser access". (gstack) Voice triggers (speech-to-text aliases): "pair agent", "connect agent", "share my browser", "remote browser access".
Also in 11 projects that use this
ShreyRaviSkilllanding-report
Read-only queue dashboard for workspace-aware ship. Shows which VERSION slots are currently claimed by open PRs, which sibling Conductor workspaces have WIP work likely to ship soon, and what slot /ship would pick next. No mutations — just a snapshot. Use when asked to "landing report", "what's in the queue", "show me open PRs", or "which version do I claim next". (gstack)
Also in 8 projects that use this
timurgaleevSkilldocument-generate
Generate missing docs for a feature, module or project as tutorials, how-tos, reference and explanation.
Also in 7 projects that use this
seek888Skillopen-gstack-browser
Launch GStack Browser — AI-controlled Chromium with the sidebar extension baked in. Opens a visible browser window where you can watch every action in real time. The sidebar shows a live activity feed and chat. Anti-bot stealth built in. Use when asked to "open gstack browser", "launch browser", "connect chrome", "open chrome", "real browser", "launch chrome", "side panel", or "control my browser". Voice triggers (speech-to-text aliases): "show me the browser".
Also in 8 projects that use this
- +1 more from Saqlain-811
Other versions98 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 98
- garrytan/gstack★ 142ksame name
- dragonflydb/dragonfly★ 32ksame name
- xtermjs/xterm.js★ 21ksame name
- udecode/plate★ 17ksame name
- Z3Prover/z3★ 13ksame name
- androidx/androidx★ 6.1ksame name
- samchon/typia★ 6ksame name
- PaddlePaddle/FastDeploy★ 3.7ksame name
- zalando/skipper★ 3.3ksame name
- samchon/nestia★ 2.2ksame name
88 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository16 other files it gives its agents.
- bugfixSkill
- designSkill
- deterministic-pr-gatesSkill
- featureSkill
- gatesSkill
- parallel-reviewSkill
- pipeline-reviewSkill
- planSkill
- pr-followupSkill
- releaseSkill
- reviewSkill
- specSkill
- statusSkill
- sync-workflowSkill
- testSkill
- trim-contextSkill
The file
We split this skill into the rules it gives an agent
3 rules. 0 are common, 1 is distinctive, and 2 are very specific to this project.
0
Common
Most of the 84 edited versions we read keep these.
1
Distinctive
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 42
Read .claude/context/benchmark-log.md's most recent prior entry (any label).
- Distinctive: line 47
Do not draw a conclusion beyond what the numbers show — one run is a data point, not a trend.
- Very specific: line 50
Delete the branch, local and remote if pushed.
“Common” is measured: kept by at least 6 in 10 of the 84 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Benchmark Agent
You are the Benchmark Agent for an iOS app project using this pipeline. Your job is to run a small, fixed "canary" feature through the pipeline and record objective metrics, so a later change to the pipeline itself (a different model routing, a reworded gate, a new command instruction) can be compared against a known baseline instead of judged by feel.
This exists for the same reason a practitioner benchmark comparing two local model sizes on one fixed task (turns, tool errors, tokens, time, accuracy) beats trusting a public leaderboard for your actual workload: your own pipeline, on your own repo, is the only thing you actually need to validate. This command is deliberately small — it captures facts about one run, it does not replace judgment about whether those facts indicate a regression.
Trigger
Invoked manually, ideally right before and right after a change to .claude/skills/*/SKILL.md, a model: frontmatter change, or a gate/review rule change — e.g. /benchmark baseline before the change, /benchmark after-rule-change after it.
The canary feature (fixed — do not vary this between runs)
A disposable, single-purpose fixture, small enough to run in minutes and touching enough layers to exercise the real pipeline mechanics:
Add
Benchmark/TipCalculator.swift, a Domain Service with one pure function:calculateTip(billAmount: Decimal, percentage: Decimal) -> Decimal. AddBenchmark/TipCalculatorViewModel.swift, an@ObservableViewModel withbillAmount: Decimal,percentage: Decimal, and a computedtipAmount: Decimalthat calls the service. No SwiftData, no persistence, no View, no repository — this fixture deliberately skips those layers to stay minimal. Write tests for both.
This is intentionally trivial: the point is a stable, repeatable probe, not a real feature. Never merge it to develop.
Process
1. Start the clock
Record the current time before invoking /spec on the canary feature above.
2. Run the pipeline
/spec → /plan → /feature → /gates, on a branch named benchmark/<label>-YYYY-MM-DD off develop. Use the exact canary feature text above as the /spec input, unmodified. Let each stage run as it normally would — do not skip steps to save time, the point is measuring the real pipeline.
3. Stop the clock
Record the time when /gates reports its final summary (whether it passed on the first attempt or needed fixes — note either outcome).
4. Capture metrics
python3 scripts/capture_pipeline_metrics.py benchmark/<label>-YYYY-MM-DD --label <label> --started "<step-1 timestamp>" --ended "<step-3 timestamp>"This appends a structured entry to .claude/context/benchmark-log.md: commit count, commit subjects (to visually confirm RED/GREEN order), wall-clock duration, and the /gates pass/fail summary you paste in when prompted.
5. Compare against the log
Read .claude/context/benchmark-log.md's most recent prior entry (any label). Flag to the user if this run shows:
/gatesneeded a fix-and-rerun where the baseline passed clean (or vice versa)- Wall-clock more than ~50% higher than the baseline entry
- More commits than the fixture should need (2 tasks × RED+GREEN = 4 commits is the expected baseline shape)
Do not draw a conclusion beyond what the numbers show — one run is a data point, not a trend. If you have 3+ entries for the same label, note whether it's trending or noisy.
6. Clean up
Delete the branch, local and remote if pushed. The canary must never open a real PR against develop unless you're specifically diagnosing something that only fires post-PR (e.g. /review) — default is local-only through /gates.
Done when
.claude/context/benchmark-log.md has a new entry, the benchmark branch is deleted, and any regression against the prior entry is reported to the user.
Limitations (be honest about these)
- This does not measure token cost directly — Claude Code doesn't expose a per-session token count to a script. If you want that number, read it from the session's own usage display and add it to the log entry's notes field by hand.
- One canary run is a noisy sample (model variance, unrelated environment factors). Treat a single outlier as a prompt to re-run, not as a confirmed regression.
- This benchmarks the pipeline's mechanics (commits, gates, timing) on a fixed trivial fixture — it says nothing about whether the pipeline still produces correct code on a real, complex feature. It's a canary, not a replacement for real-feature testing.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

