NomicFoundation·NomicFoundation/hardhat/.claude/skills/benchmark-run-sizing/SKILL.md

benchmark-run-sizing

Skill8.5k starsChanged 47 days agoLicence not identifiedView on GitHub

Analyzes the Hardhat regression benchmark history (the hardhat-benchmark-results repo's data.js) and recommends how many times each benchmark should run. It reads every commit's individual run times, computes each benchmark's per-commit coefficient of variation, removes outliers, takes the 95th-percentile CV, and sizes run counts for target noise levels (sigma = 3%, 1.5%, 1%, i.e. regression alert limits of 10% / 5% / 3%). Use when asked to right-size benchmark run counts, justify or revisit run counts, assess benchmark variance/noise, or decide whether to add or remove runs after changing the regression alert threshold.