agentleFS
jimyagjimyag/dotfiles/skills/benchmark-checklist/SKILL.md

benchmark-checklist

Skill11 starsChanged 27 days agoMITView on GitHub

在运行基准、比较性能方案或报告实测吞吐、延迟和资源收益时使用;检查测量有效性与结果可信度,不用于仅凭源码查找潜在性能问题。

42 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use ketchup-video, principle-explain-the-number and 10 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • tristanphvnSkill

    ketchup-video

    Catch the user up like /keel:ketchup, with a short silent video of slides for what happened and the waiting calls as text under it. Use for /keel:ketchup-video.

    Also in 14 projects that use this

  • ianclearySkill

    principle-explain-the-number

    Apply before you trust, report, or act on a number you measured: a speedup, a regression, a throughput, a latency, or an eval result. Find what limits it, and rule out that it measured something other than the work you think.

    Also in 28 projects that use this

  • ianclearySkill

    poteto-help

    Guides users through pstack setup, /poteto-mode, and picking the skill, playbook, or principle for a task. Type /poteto-help with a question.

    Also in 14 projects that use this

  • instructureSkill

    ketchup

    Rebuild context after /clear by analyzing uncommitted changes across Canvas LMS and all plugin repositories

    Also in 25 projects that use this

  • drafaelSkill

    in-your-own-words

    Restate the user's goals and the problem they are trying to solve to check understanding. Manual invocation only; use when the user explicitly invokes in-your-own-words or requests this understanding check. Do not solve the task or rewrite the assistant's last answer.

    Also in 24 projects that use this

  • jaydubya818Skill

    principle-attack-the-premise

    Apply when repeated fixes fail and a shared assumption may be wrong.

    Also in 27 projects that use this

The file

Files
SKILL.md

性能测量校验

让性能结论对应实际完成且结果正确的工作。使用项目现有基准入口,不为一次测量引入新框架。

固定声明和测量边界

  • 写清要判断的声明、单位和工作负载,包括数据量、并发、初始状态,以及冷缓存或热缓存。先读测量脚本,确认计时包含和排除了什么。
  • 对照使用相同环境、版本约束和数据,双方按实际部署方式构建和调优。无法消除的配置差异属于结论的条件。
  • 先检查机器负载和压测端余量。采样器、追踪器会改变耗时,定位瓶颈与报告数字分开运行。

检查测量是否成立

  1. 确认工作发生在计时区间内:异步操作已经等待,惰性结果已被消费,后台写入在需要持久化的声明下已完成。用实际完成的请求、行数或字节数核对,不能只统计发起次数。
  2. 同时检查结果正确性、错误数、拒绝数、超时和重试。空结果、提前返回或快速报错不算优化;相同请求量但完成工作量不同的结果不能直接比较。
  3. 找到主要瓶颈,并检查生成负载的一端是否先饱和。无法解释收益机制或变化异常时,先排除缓存、空操作、调试构建和计时范围错误,再给结论。
  4. 检查数量级:吞吐不能凭空超过资源带宽;仅优化占总耗时一小部分的代码,不能解释远超该占比的端到端耗时下降。
  5. 对方案比较交替运行 A/B,记录多次运行的中位数和波动范围。次数以足够区分差异与噪声为准;差异落在波动内时报告没有可测差异。用户只要粗估时可单次运行,但仍核对工作量和错误,并注明单次结果。
  6. 微基准结论注明适用范围;要声称用户体验或服务性能改善,还需测量真实数据量与并发下的端到端路径。

持续优化时

先确认测量能区分已知快慢场景,再固定负载、统计口径和回归门槛。每轮只验证一个假设,记录改动、前后数字、正确性检查和保留或撤销的理由。只有超过噪声且行为保持正确的改动才算收益;测量口径改变后重新建立基线。停止条件使用用户目标、预算或收益已不足以覆盖成本,不自动提交或推送。

交付

给出更快、更慢、没有可测差异或证据不足,并附单位、工作负载、次数、波动范围、错误与完成工作量、瓶颈证据及复跑命令。缺少正确性或工作量证据时不判定优胜者。只完成静态分析时写明未测量。

已有输出足够时只分析,不为补齐清单擅自压测生产服务;需要运行时遵循任务授权、项目验证入口和资源上限。

统一约束

  • 证据与验收:遵循 ../_shared/common-acceptance.md,本 skill 只补充其中没有的要求。

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.