agentleFS
OpenPerfAgentOpenPerfAgent/who-ate-my-flops/skills/setup-baseline/SKILL.md

setup-baseline

Skill35 starsChanged 43 days agoApache-2.0View on GitHub

Build the starting point both diagnose and optimize measure everything else against: a worktree, a workspace, the computation graph and execution schedule, and the base commit measured for speed, memory and correctness and profiled in full. It arms nothing, so the caller decides what happens next.

The file

We split this skill into the rules it gives an agent

6 rules. About 0 are common, 1 is distinctive, and 5 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 5

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Create a separate worktree and make every change there.

    line 12
  2. Very specific:

    Use <worktree>/workspace-who-ate-my-flops/ as <ws>.

    line 14
  3. Very specific:

    Keep process records in records/, including any planning notes.

    line 24
  4. Very specific:

    Leave contract.md, latest-report.md, and benchmark.csv at the workspace root.

    line 25
  5. Very specific:

    Put helper scripts in tools/ and run artifacts in runs/.

    line 26
  6. Very specific:

    Use the extract-computation-graph skill to build the job's computation graph and execution schedule inside the workspace: …

    line 28

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

setup-baseline

In: contract.md, as init settled it. Out: a workspace whose base row is recorded in records/candidates.jsonl and benchmark.csv, and whose base commit carries a trace and a diagnosis.

  1. Create a separate worktree and make every change there. Leave the original checkout alone.
  2. Use <worktree>/workspace-who-ate-my-flops/ as <ws>. Add /workspace-who-ate-my-flops/ to the local exclude file returned by git rev-parse --git-path info/exclude from the worktree, preserving existing entries. Create the workspace with python <plugin root>/tools/workspace.py init <ws> <contract>. The plugin root is the directory two levels above this SKILL.md, and Claude Code also exposes it as ${CLAUDE_PLUGIN_ROOT}. Every tool this plugin ships resolves from there, because you are standing in the target's worktree and never in the plugin. Read workspace.py first: it defines the file formats everything downstream uses, and the workspace is the common ground the skills coordinate through. Keep process records in records/, including any planning notes. Leave contract.md, latest-report.md, and benchmark.csv at the workspace root. Put helper scripts in tools/ and run artifacts in runs/. Avoid adding ad hoc files to the workspace root.
  3. Use the extract-computation-graph skill to build the job's computation graph and execution schedule inside the workspace: records/computation_graph.json for what is computed, and records/execution_schedule.json for when and where it is computed and what moves between devices. Both get read against the profile results later, to locate the performance problems and find fixes for them.
  4. Record the original code's speed, memory and correctness using the scripts defined in run of contract.md, and diagnose it. The baseline covers all four, and all four get written down:

    • Diagnosis: run the torch-profile skill on the unmodified code, in full. Its traces go under commits/<base sha>/trace/ and its report to commits/<base sha>/report.md. This is the one diagnosis of the user's own code: every later profile only says whether a change removed something from it, and the handover points back to it. finish refuses a campaign without it.
    • Speed: by default measure the warm-up cost, which is paid once, and the steady step time, which is paid many times over.
    • Memory: measure CPU and GPU memory, and watch how each moves across the warm-up phase and the steady-step phase.
    • Correctness: a. Read the ## Correctness Instrumentation Plan section of contract.md for where the checkpoints go and which recorder it names. b. Use the recorder the repo already has. In imp_genai that is research/core/probe, the same code vendored under its own name. Install one only when the repo has none, and then install parity (https://github.com/OpenPerfAgent/parity): import parity, recorded when PARITY=1 with PARITY_OUT=<file>, compared with parity compare. c. Read its README first, then put the planned calls in. d. Smoke run it 3 times, and work out the tolerance its numbers actually hold to. parity derive does this, and it needs three runs of unmodified code.

    Record the result with workspace.py add --verdict base, and carry the tolerance forward: every later comparison is read against it.

The questions ended with init, but setup can still fail in a way worth handing back: a runner that does not run, a node that is busy, a tolerance that will not hold. Say what failed and stop, rather than working around it.

Arm nothing here. What keeps a campaign turning for hours belongs to the caller, and a diagnosis has no loop to keep turning.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.