perf-budget
Skill0 starsChanged 4 months agoMITView on GitHub
Catch hot-path and external/model-call latency regressions before they reach production — compare p95 per operation against configured budgets and a stored baseline, fail the PR on regression, and offer to ratchet the baseline down on improvement
6 other projects use a version of this.
Adopted by
Most recently taken up by
- Univers42/harness42their own version, first seen Nov 1, 2026
- vindreshsingh/engineering-skillstheir own version, first seen Oct 20, 2026
- mattbutlerengineering/mattbutlerengineeringtheir own version, first seen Oct 13, 2026
- Univers42/claude-deal-with-the-deviltheir own version, first seen Oct 4, 2026
- AlpayC/claude-skill-settheir own version, first seen Sep 14, 2026
- CaduS0uza/qa-supremetheir own version, first seen Sep 10, 2026
Other versions6 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 6
- Univers42/claude-deal-with-the-devil★ 4same name
- Univers42/harness42★ 4same name
- mattbutlerengineering/mattbutlerengineering★ 3same name
- CaduS0uza/qa-supreme★ 1same name
- AlpayC/claude-skill-set★ 0same name
- vindreshsingh/engineering-skills★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository29 other files it gives its agents.
- adopt-existing-projectSkill
- adrSkill
- branch-conflict-checkSkill
- ci-watchSkill
- compound-learningsSkill
- config-auditSkill
- cost-checkSkill
- coverage-ratchetSkill
- doc-syncSkill
- eval-harnessSkill
- flaky-triageSkill
- followup-trackingSkill
- incident-captureSkill
- kit-doctorSkill
- kit-updateSkill
- logging-initSkill
- migration-checkSkill
- new-project-bootstrapSkill
- nightly-auditSkill
- observability-checkSkill
- parallel-workSkill
- pr-babysitterSkill
- pre-prSkill
- prompt-regressionSkill
- releaseSkill
- session-handoffSkill
- sync-healthSkill
- using-the-kitSkill
- writing-kit-skillsSkill
The file
We split this skill into the rules it gives an agent
7 rules. 0 are common, 4 are distinctive, and 3 are very specific to this project.
0
Common
Most of the 5 edited versions we read keep these.
4
Distinctive
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 42
Read the per-path budget map from capabilities.perf.budgets.
- Very specific: line 55
Read the baseline file path from capabilities.perf.baseline_file (default: .perf-baseline.json).
- Distinctive: line 63
If the baseline file does not exist, treat every path's baseline as the budget value — the first run will create it.
- Very specific: line 66
Read the regression tolerance from capabilities.perf.tolerance_pct (default: 10).
- Distinctive: line 130
When one or more paths are strictly faster than their baseline, offer to lower the baseline to the current p95.
- Distinctive: line 149
If any path failed either check, the gate FAILS.
- Distinctive: line 157
Do not silently raise the baseline.
“Common” is measured: kept by at least 6 in 10 of the 5 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Performance Budget
The performance analogue of the cost guardrail. Where the cost gate prevents
overspending money, this gate prevents overspending time — catching regressions
in hot paths and external/model calls during review, before they affect users.
It enforces two thresholds per operation: an absolute budget (the p95 must
not exceed N ms) and a baseline tolerance (the p95 must not regress beyond a
stored per-path baseline by more than tolerance_pct). When a path improves,
it offers to ratchet the baseline down — one-way, like the coverage ratchet. It
measures and reports; it does not tune the hot path itself — fixing a
regression is the author's task, just as the coverage ratchet does not write
tests.
Profile-driven. Commands and thresholds below read from .claude/kit.yaml.
Steps tagged # kit.yaml → <key> use the value at that key if set; an empty
string or absent key means skip the step and mark it N/A. This skill is invoked
from pre-pr as a capability-gated step.
Workflow
1. Check the capability gate
Open .claude/kit.yaml and read capabilities.perf.enabled. If it is false
or absent, skip all remaining steps and report N/A. This skill ships
disabled by default — it is most valuable on a deployed service with real
traffic and an invocation_log table or a reproducible benchmark suite. Enable
it once those preconditions exist rather than running it against a library or a
project whose latency data does not yet exist.
2. Read budgets and baseline
Read the per-path budget map from capabilities.perf.budgets. It is a flat
YAML map of operation_name: p95_ms, for example:
# kit.yaml → capabilities.perf.budgets
capabilities:
perf:
budgets:
llm.chat_completion: 2000
db.task_list: 50
db.task_write: 30Read the baseline file path from capabilities.perf.baseline_file (default:
.perf-baseline.json). The file is a JSON object mapping operation name to the
last accepted p95 ms, for example:
{ "llm.chat_completion": 1900, "db.task_list": 44, "db.task_write": 27 }If the baseline file does not exist, treat every path's baseline as the budget value — the first run will create it.
Read the regression tolerance from capabilities.perf.tolerance_pct (default:
10). A path that regresses by more than this percentage above its stored
baseline triggers a failure even if it is still under its absolute budget.
3. Gather latencies
The source of latency data depends on capabilities.perf.source:
"benchmark" — run the configured benchmark command and parse its output:
# kit.yaml → capabilities.perf.benchmark_cmd
uv run pytest tests/benchmarks/ --benchmark-json=.benchmark-results.json -qParse .benchmark-results.json. For each benchmark, extract the name,
stats.median (p50, in seconds), and stats.ops or use stats.q95 / build
a p95 from the sorted sample if available. Convert to milliseconds.
Map benchmark names to the operation keys in capabilities.perf.budgets — the
benchmark name field must match a budget key exactly, or be prefixed with
bench_ that strips cleanly (e.g. bench_db.task_list → db.task_list).
Skip benchmarks that have no matching budget key.
"log" — query the spend/telemetry table for recent latency observations:
-- kit.yaml → capabilities.llm.spend_table (default: invocation_log)
SELECT
operation,
latency_ms
FROM invocation_log
WHERE timestamp >= datetime('now', '-7 days')
AND latency_ms IS NOT NULL
ORDER BY operation, latency_ms;Group rows by operation. Compute p50 and p95 from the sorted sample using
nearest-rank: p50 = row at index ceil(0.50 * n), p95 = row at index
ceil(0.95 * n). Require at least 20 observations per path to consider the
estimate reliable; paths with fewer observations are shown in the output but
marked (sparse — N obs) and do not block the gate.
4. Compare against budgets and baseline
For each budgeted path, apply both checks:
| Check | Condition | Result |
| Absolute budget | p95 > budget_ms | FAIL |
| Baseline regression | p95 > baseline_ms × (1 + tolerance_pct / 100) | FAIL |
| Baseline improvement | p95 < baseline_ms | PASS — offer to ratchet |
| Within tolerance | otherwise | PASS |
A path fails if either check is violated. Paths not present in
capabilities.perf.budgets are ignored.
Strictness (gates.strictness in kit.yaml, see docs/PROFILE.md):
at prototype, failures are reported as ADVISORY and do not block the PR.
At production, budgeted paths with sparse data (<20 observations) are a FAIL
rather than an annotation — a production gate that can't measure its path is
not a gate — and a missing baseline file blocks instead of bootstrapping.
5. Ratchet the baseline down on improvement
When one or more paths are strictly faster than their baseline, offer to lower the baseline to the current p95. This is a one-way gate: baselines only decrease (tighter) — never increase automatically. Ratcheting locks in the improvement so future regressions back to the old number are caught.
To accept the ratchet for an improved path, write the updated JSON to the baseline file and stage it:
# update .perf-baseline.json with the new per-path p95 values, then:
git add .perf-baseline.jsonConfirm with the author before committing — it is a deliberate policy tightening,
not a mechanical fix. A baseline change commit message should read:
perf: ratchet baseline — <path> <old>→<new> ms.
6. Fail the gate on regression
If any path failed either check, the gate FAILS. The author must either:
- Investigate and fix the regression (preferred), or
- Explicitly acknowledge that the workload changed and the new baseline is
correct — in which case update the baseline file with a commit message
explaining why and update
capabilities.perf.budgetsif the absolute budget also needs to move.
Do not silently raise the baseline. Do not comment out a budgeted path without a note explaining the removal.
Notes
- This skill is invoked from pre-pr as step 17 (capability-gated). Add it
to the pre-pr checklist output as:
- [ ] Perf budget: <results / N/A>. - This skill is most valuable on a service with real traffic (source: "log")
or a project with a stable benchmark suite (source: "benchmark"). On a
library without either, enable it only after both preconditions exist.
It ships disabled by default (
capabilities.perf.enabled: false). - The runtime companion to this gate is
templates/perf_budget.py— aPerfBudgetcontext manager / decorator that emitsperf_budget_exceededwarnings into the same observability pipeline as fallback alerts. Wire it around hot paths so theinvocation_logtable accumulates the latency data this gate reads. - p95 is the right aggregation for a latency gate: it filters single-sample noise while surfacing tail behavior that affects real users. Do not gate on mean — means hide the tail.
Output
## Performance Budget
Source: <benchmark | log — last 7 days>
| Path | p50 ms | p95 ms | Budget ms | Baseline ms | Δ baseline | Verdict |
|---|---|---|---|---|---|---|
| llm.chat_completion | 1 120 | 1 840 | 2 000 | 1 900 | -60 ms (-3.2 %) | PASS |
| db.task_list | 42 | 91 | 50 | 80 | +11 ms (+13.8 %) | FAIL — budget exceeded |
| db.task_write | 18 | 29 | 30 | 31 | -2 ms (-6.5 %) | PASS — improved |
Overall verdict: FAIL — 1 path exceeded budget
Ratchet action: <none | offered to lower baseline — db.task_write 31 ms → 29 ms>Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

