agentleFS
Walsworth-GroupWalsworth-Group/physkit/skills/new-campaign/SKILL.md

new-campaign

Skill7 starsChanged 3 months agoMITView on GitHub

Interview and scaffold a launch-ready autoresearch campaign: goal, frozen metric, archetype, project and campaign tree. Use to start or set up a campaign; not for running one.

The file

Show its rules and text

We split this skill into the rules it gives an agent

22 rules. About 0 are common, 9 are distinctive, and 13 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈9

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 13

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Do not use this to run an already-scaffolded loop, to verify a finding (verify), or to write the final report (report, campaign mode).

    line 43
  2. Very specific:

    framing artifact: If one exists, or you ran frame-problem for this goal in-session, read it and

    line 71
  3. Distinctive:

    Read these so your questions and fills are grounded:

    line 85
  4. Very specific:

    An ancestor holds PROJECT.md: use that project (it carries the lineage in PROJECT.md §Lineage).

    line 108
  5. Very specific:

    All of this background lives in the project overview, ../../PROJECT.md — the governing physics (with the target-of-study section), the …

    line 273
  6. Distinctive:

    If experiments are small, a few MB, say so explicitly and note that no plan is needed.

    line 340
  7. Distinctive:

    Ask which environment the user wants, and default to proposing a fresh virtual environment (a new venv or conda env).

    line 382
  8. Distinctive:

    User's own ideas (always ask): Ask the user for (a) any ideas or directions they would like you to try, and (b) anything that should not be …

    line 396
  9. Very specific:

    Before generating anything, write a concise campaign spec: the archetype(s) that apply, the FROZEN metric with its signed-off plausible …

    line 419
  10. Very specific:

    Resolve every [[OPTIONAL — … ]] block: delete it if it does not apply, which keeps the slim core slim.

    line 456
  11. Very specific:

    Resolve every [[KEEP — … ]] tag: keep what it marks - the heading, key, column or line its consumer reads, spelled exactly - and delete …

    line 462
  12. Distinctive:

    Every kit release is tagged, so this pin recovers the exact skills, reference docs, and templates the campaign was generated under — the …

    line 492
  13. Distinctive:

    Add no directions of your own.

    line 536
  14. Very specific:

    Verify the layer is wired: before handoff: .claude/settings.json registering guard_promotion, .codex/hooks.json registering the synchronous …

    line 538
  15. Distinctive:

    Ask the user or draw on the topic-4 literature.

    line 572
  16. Very specific:

    Simulation validation (conditional on topic 4): If the campaign has a simulation aspect, author docs/SIMULATION_VALIDATION.md for it, a …

    line 617
  17. Very specific:

    Seed the user's directions: Put the user's ideas to try (topic 12) in the opening JOURNAL — ranked under ## What to try next and why, …

    line 626
  18. Very specific:

    Leave one line under ## Where we stand in results/JOURNAL.md: Setup audit: baseline 000 = … ± … (N seeds).

    line 718
  19. Distinctive:

    Run both gates on the generated tree and fix anything they flag:

    line 730
  20. Very specific:

    If Codex is the first-launch harness (or the alternate in a switchable campaign), tell the user to open a fresh task with that absolute …

    line 775
  21. Very specific:

    Before the first real experiment, run the prompt-surface audit (docs/reference/prompt-surface-audit.md): capture each role's context as …

    line 821
  22. Distinctive:

    Never overwrite existing files without confirmation.

    line 829

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

new-campaign — interview, then scaffold a campaign

Turn a fuzzy research goal into a running, rigorous, documented campaign. The deliverable is scaffolding that encodes the campaign's intent so the loop-running agent cannot drift from it: a FROZEN metric, an experiment loop, levers, guardrails, logging, verification, and reporting.

Every problem is its own; the templates are a starting point, not a form. They encode what has worked across many campaigns: take them as inspiration and shape them to this problem. Reword, restructure, drop a section that does not serve it, or change the procedure where a different one would work better. Only three things are fixed:

  • What code reads, marked [[KEEP — …]] in the templates (the JSON configs are machine config throughout: change values, not keys). Keep the marked element and delete the tag. The kit's machinery (run_campaign.py, the gates, the verification presets) is never edited per campaign; a method that needs it to behave differently is proposed to the user, not built.
  • The frozen integrity core: the honesty and verification rules and the FROZEN metric.
  • Nothing new that binds workers enters without the user's yes. You know no more about this problem than the workers who follow you, and a rule, constraint, or procedure written into the scaffold steers every one of them and cannot be argued with from inside a spawn. Propose each by name (Step 2); what you only suspect goes to the JOURNAL marked [setup inference] (Step 3).

Stance for the interview (CORE Stage 0, applied to setup):

  • Fill from sources first; ask only when unsure. Never ask what you can check. Before asking anything, harvest every answer you can definitively obtain from what is already available: the conversation so far, any documents or paper manuscripts the user says describe their setup, data and config files in the target directory, and anything you can detect directly (Step 0.5). Ask the user only about genuine gaps.
  • When a value came from a source but you are unsure it applies, ask a short question naming the source — for example, "should I use the experimental parameters described in Smith et al. for this problem?" — rather than asking the user to supply it from scratch.
  • State assumptions explicitly. Present multiple interpretations when the request is ambiguous. Push back when a simpler scope exists. Co-design the metric rather than inventing it. Narrow the purpose: a campaign explores one idea or one slice, not a whole project.

When to invoke

  • The user wants to start / set up / scaffold a new campaign or optimization loop.
  • An existing project needs the autoresearch structure (campaign.md + metric + scoreboard) added.

Do not use this to run an already-scaffolded loop, to verify a finding (verify), or to write the final report (report, campaign mode). To fork a campaign whose findings a bug or correction invalidated — a clean break with an archived source and a re-verified carry-forward — use fork-campaign, not this skill.

  • If the idea itself is unvetted, offer to run assess-idea first. It is advisory, covering feasibility and worth, and never blocks scaffolding, but a quick GO / DE-RISK-FIRST / NO-GO can save a campaign built on a low-ceiling problem. Off by default; propose it when the user has not already pressure-tested the direction.

Inputs

  • The target directory where the campaign should be created (ask if not given; default: the current working directory). Never overwrite existing files without explicit confirmation.
  • Optionally, a one-line goal the user already typed. Use it to seed the interview, not to skip it.
  • Optionally, a framing artifact from frame-problem (a FRAMING.md). If one exists, compose it as described below rather than re-interviewing.

Two ways to gather the inputs

(a) Live interview (default): Step 1 below, topic by topic. (b) Draft-document mode, for a user who prefers editing a document to answering in the terminal: emit a pre-filled draft campaign.md (from its template), filling every slot you can detect or harvest, marking what needs the user as [PROPOSED] or an explicit prompt. The user edits the draft directly — the document they craft IS the campaign definition, not a form transcribed into one — then you finalize it at sign-off: resolve every [PROPOSED] tag and prompt with them, and the validator refuses any that survive. Either way the result is the same signed-off campaign.md (Step 2). Offer the draft when the interview would be long, or when the user is on a slow channel.

Compose frame-problem — don't re-interview the physics

frame-problem owns physics scoping (inventory, verification plan, decomposition) and writes a durable framing artifact. If one exists, or you ran frame-problem for this goal in-session, read it and confirm rather than re-ask: carry its physics inventory, a-priori bounds, verification plan, and step decomposition straight into the spec, and fill only the campaign-only gaps (metric co-design, levers/frozen, data, compute, archetype). If no framing artifact exists and the problem has real physics, run frame-problem first, or fold its three outputs into the interview, rather than duplicating the work. Within a session this is shared context; across sessions the persisted artifact carries it. Do not interview twice.

Step 0 — read the kit before interviewing

PhysKit resource links in this skill are relative to the directory containing this SKILL.md, not to the campaign working directory. Resolve them to absolute paths before reads, copies, or commands. If any linked package resource is absent, this is a skills-only projection rather than the full PhysKit package: stop and ask the user to install the full plugin; never fall back silently to a source checkout.

Read these so your questions and fills are grounded:

  • ../../docs/CORE.md — the always-on spine and the general skills it routes to (frame-problem, verify, simulation-design, correctness-suite, report, …) that this campaign composes.
  • ../../templates/ARCHETYPE_CHEATSHEET.md — the campaign archetypes (A–I) and their patterns.
  • ../../templates/PLACEHOLDERS.md — the fill convention and canonical placeholder list.
  • Skim ../../templates/campaign/ — what you will fill. campaign.md is the whole signed-off contract now: goal, metric, levers, §Verification, §Known landmines, and the append-only §Decision record tail. For archetype E, also read ../../docs/reference/hardware-in-the-loop.md (§0 is an example assembled loop) and ../../docs/reference/instrument-contract.md (the lab-side protocol the kit's scripts/hardware/ machinery speaks).

Step 0.1 — establish the project home (every campaign lives in one)

Every campaign lives inside a project: the home for one physical system investigated over many campaigns and ad-hoc sessions, so shared data and docs live once and are referenced by relative path. This is unconditional; there is no bare-campaign mode. A one-off costs three near-empty documents for it, since the physics, conventions, and instrument knowledge lives once in the project overview rather than multiplying. In exchange it grows into a sibling campaign, a fork, or a follow-up session with no reorganization and no path rewrites.

  • An ancestor holds PROJECT.md: use that project (it carries the lineage in PROJECT.md §Lineage). The campaign lives at <project>/campaigns/<NN>_<slug>/; pick <NN> as the next zero-padded index after the last lineage row.
  • Otherwise, run new-project first to create the home, then come back here. Carry everything you have already gathered: new-project fills its tree from the same answers, so never re-interview. When new-project invoked you, the flow is one-way: do not call back into it. Tell the user the home is being created; do not ask permission for it.
  • A pre-existing bare campaign (imported or hand-built) is lifted into a project rather than left as it is: create the home around it, move it to campaigns/01_<slug>/, hoist its physics, conventions, and instrument docs into the project overview, and append its lineage row.

Consequences, unconditional, with no campaign-local variant:

  1. Physics/conventions/instrument are referenced from the project overview (../../PROJECT.md), never copied.
  2. Data is ../../data.
  3. The campaign appends its lineage row (PROJECT.md §Lineage) and resolves campaign.md §Relationship to sibling campaigns: keep and fill if coupled to a sibling, delete if independent. Coupling decides that one, not project membership.
  4. The campaign CLAUDE.md project-context block is always kept.
  5. A campaign's managed instructions require the project's managed CLAUDE.md/AGENTS.md pair (new-project writes it). If the pair is missing or differs, stop and have the operator choose the authoritative side, then reconcile it explicitly (scripts/sync_agent_instructions.py --scope project --from claude|agents --write, run from the campaign); never overwrite an instruction file by inference.

Step 0.5 — harvest sources and detect the environment (before asking anything)

Collect what already answers the interview, so you only ask about real gaps:

  • Read everything the user pointed to: setup docs, paper manuscripts, prior notes, plus any data and config files already in the target directory. Extract concrete values (geometry, parameters, units, data shapes) and note each value's source, so you can cite it and ask "use the value from <source>?" if its applicability is uncertain. For papers, prefer the arXiv version and convert any PDF with a scientific-PDF-to-markdown tool to save tokens.
  • Detect, do not ask, what you can check: the compute (CPU cores, RAM, and GPU, e.g. by running nvidia-smi), the free disk space on the volume the campaign will live on, the available Python/toolchain, and the data files present and their shapes.

Carry these harvested facts into the interview.

Step 0.6 — establish the source-authority ledger (when more than one source describes the system)

Replication and model-improvement campaigns routinely ship several descriptions of the same system that disagree: the paper text, a figure, leftover or post-publication code, a released trained-model or data artifact, and the user's recollection. Building the baseline on the wrong one is the most expensive setup mistake available. A faithful reproduction of the wrong spec looks correct and only unravels hours into the loop, and a "match the published number" gate can then drive the agent to reverse-engineer the original's bug. Before scaffolding, build a short ledger:

  • Enumerate every source that describes a load-bearing fact: the architecture/method, the training/processing recipe, normalization and units, hyperparameters, the data convention.
  • Rank them by authority. A shipped artifact you can parse or run (trained weights, saved model, released data) outranks the paper text or figure, which outranks modified or leftover code, which outranks human recollection. The artifact is ground truth; everything else is a description of it.
  • Resolve each load-bearing fact to the highest-authority source available by reading the artifact. Parse the saved model's topology, probe the data's actual statistics, run the released weights on your eval set, rather than inferring from a lower-authority proxy. The campaign already pins data conventions empirically (Step 1, topic 5); pin the method and recipe with the same rigor. Do not defer it to "exp 000's job" when an artifact on disk settles it in seconds.
  • Tag each fact's provenance: PINNED (from an authoritative artifact or empirical probe) · REPORTED (in text) · INFERRED (from an untrusted proxy) · RECALLED (from memory). Carry the tags into the spec.
  • Quarantine, do not absorb, untrusted or contradictory sources. State explicitly that the leftover or modified code is not the spec, and why, so the loop agent does not re-anchor on it.
  • A load-bearing fact you can only mark INFERRED or RECALLED is a genuine gap. Ask the user in Step 1, naming the source and the conflict — for example, "the figure shows no dropout but the leftover code has it; the saved model would settle it, do you have the trained weights?" — rather than silently baking it in.

Seed the resolved ledger into campaign.md §Decision record (Step 3) so the provenance survives into the loop.

Step 1 — the interview (topic by topic)

Go one topic at a time and confirm understanding before moving on. For each topic, fill what the sources already answer and ask only the gaps. Use the AskUserQuestion tool for categorical forks, and free-form dialogue for the metric and physics narrative. Capture every answer; you will need it to fill the templates.

Always ask these, even if you think you could infer them: the internet/literature level (topic 11), the user's own ideas to try or avoid (topic 12), and which harness will launch first (topic 9). The time budget (topic 8) and environment (topic 9) are also effectively always-ask because you cannot measure them.

Always determine, for every project, whether there is a simulation component: a forward model, device-performance simulation, synthetic or simulated training data, an evaluator or simulator, or a surrogate. Most campaigns have one, so assume yes unless this is a clear exception (direct experimental control, or training or fitting purely on real experimental data). If there is one, apply the V&V doctrine (docs/SIMULATION_VALIDATION.md) and wire its gates into the core docs (Step 3).

  1. Goal & the core decision. What is the campaign trying to achieve, and what decision does its result drive? Force a narrow scope. If the goal needs several coupled pieces, flag it for the multi-stage plan (topic 11).
  2. Archetype(s) — which apply (none, one, or several). Decide which of the archetypes apply to this campaign; it may be zero, one, or several. A campaign can be both forward-model and generative-design; a discovery loop (G) can be scored through a forward model; active design (H) can be used to discriminate models (I). Inherit and merge the metric, lever, FROZEN, and failure-mode guidance from every archetype you deem relevant; do not force a single best fit. State which apply and why, and let the user correct it. If several apply in sequence rather than at once, that is a multi-stage campaign (topic 11). Archetypes G (discovery-loop), H (experiment-design), and I (model-selection) each have a dedicated operational skill.
  3. The metric — build it together (the most important topic). Co-design the single FROZEN primary: its name, direction (higher or lower is better), precise definition and units, the FROZEN scorer it lives in, and its plausible range (below). Then the baseline (exp 000) value and method, any theoretical ceilings, the guardrail (necessary but not sufficient), and diagnostics. Pin down what feeds the metric and what is FROZEN.

    Measure the baseline and the floor here; never freeze a published number. The prior work's delivered pipeline or number this campaign will freeze and build on is the load-bearing unknown the whole campaign inherits: the one campaign measured end to end spent most of its loop auditing its own ledger because its contract carried a floor taken from a paper that the data did not support. Before anything is frozen, the setup audit (Step 3.7) reproduces the baseline from its config, measures the floor from replicates run in this tree, and checks that the scorer orders a synthetic pair the way the mission says it should — each as a contract check in the suite. If a number genuinely cannot be measured here, say so plainly in the campaign spec as an open risk rather than silently trusting it.

    Propose the plausible range and get it signed off. Once the metric is agreed, propose the range a correct pipeline cannot leave. A good baseline is the perfect score at one end and a TRIVIAL predictor's score (all zeros, the mean, a constant) on the evaluation data at the other — e.g. RMSE in [0, RMSE of all-zeros vs the ground-truth test images]; suggest something else if it suits the problem better (an RMSE floor at the measurement noise, below which a score points at leakage), with its basis. Compute the trivial end when the data is in hand, never guess it (if it is not yet staged, compute it in Step 3 before the scorer freezes, and re-present). Present it for the user's sign-off, saying what it does: a result outside it is flagged for extra scrutiny as a possible implementation error. It is a cheap bug check, not a prediction of how well anything will do, so keep it to what no correct pipeline produces. Capture it in both forms: the prose with its basis ({{METRIC_PLAUSIBLE_RANGE}}, {{APRIORI_BOUND_NOTES}}) and the literal the FROZEN scorer enforces ({{PLAUSIBILITY_BOUNDS}}, e.g. (0.0, 0.84), or None if the user declines one).

    Then ask the question that decides what the comparison checks mean. It is campaign-wide, written into configs/loop.json campaign, and has no safe default:

    • Does re-running one experiment reproduce the metric exactly? → {{NOISE_MODEL}}. This is a question about the pipeline, not the physics, and it is answered by running one thing twice rather than from intuition: an analytic solve with no RNG is deterministic; anything with sampling, stochastic optimization, or hardware noise is stochastic. Then set the resolution below which a change is not a change: {{METRIC_TOLERANCE}} for a deterministic campaign, or {{NOISE_FLOOR}}, the replicate spread, for a stochastic one. Either may be null if it is not yet known; the checks artifact then reports that check unavailable and names the knob, which is honest, where a guessed number is not. Write null, never a TODO string: both are JSON numbers, and prose there breaks the parse.

    Getting it wrong is silent and expensive: a stochastic campaign mislabelled deterministic promotes noise, and a deterministic one mislabelled stochastic is held to a spread test it can never satisfy.

    Finally, set {{METRIC_VERSION}} / {{DATA_VERSION}} (default v1 / v1). The runner stamps them onto every scoreboard row and result, and they are bumped whenever the FROZEN scorer or the evaluation data changes — a metric_version bump is a metric epoch (roles/REFERENCE.md §Metric epochs). Say in one sentence that campaign.metric_revision ships cautious: a revision the loop wants raises ESCALATE.md and waits; the user moves that dial, never the loop.

    The primary, guardrail, and diagnostics, plus the experiment-type run numerics (e.g. epochs and best epoch for ML; iterations, wall-clock, and n_free_params for fits), plus a tier column (Scout/Confirmation/Extended) and the always-present metric_version / data_version comparability tags, become the columns of results/scoreboard.tsv, the campaign's ledger, which is separate from the JOURNAL, the campaign's working memory.

  4. Physics / system knowledge. Capture, from sources wherever possible, everything needed to model the system:

    • Physical objects & geometry: dimensions, arrangement or layout, material(s), and quantities. For an ASI fit from NV-magnetometry data, for example: the lattice geometry and spacing, island dimensions, the material, the sample orientation relative to the measured NV axis, the bias-field orientation, and the pixel size and field of view.
    • Experimental / measurement geometry: what is measured, by what instrument, in what frame, at what standoff and resolution.
    • Governing physics: the foundational equations, simulation methods and approximations, the common techniques for this problem, and the system's typical behavior through the lens of this problem (degeneracies, symmetries, regimes — for instance that magnetization-from-field is non-unique, or how an ASI traverses a hysteresis loop). Source this from user information, provided literature, or, if enabled, a targeted search.
    • Conventions: units, signs, and ordering (→ ../../PROJECT.md §Standing constraints), and known artifacts or recurring bugs to guard against. Capture the simulation methods and approximations here too; the always-determine simulation check above decides whether the V&V plan in Step 3 applies.

    All of this background lives in the project overview, ../../PROJECT.md — the governing physics (with the target-of-study section), the conventions, the instrument, and the durable decisions, one section each. Reference it; never copy any part of it into the campaign. new-project seeded it, so fill any thin slot here, and if this campaign surfaced new physics or conventions, update it in place: it is a living doc shared by every campaign. Its §Standing constraints is the operative section — the campaign driver delivers it into every spawned prompt, every cycle — so put the hard, run-constraining facts there (inventory tags, a-priori bound, sign/unit/frame conventions, run-constraining rig parameters) and keep it lean; depth goes in the sections below it. It includes the physics inventory — every relevant effect tagged accounted-for / negligible-by-scale / out-of-scope / deferred / unknown, with a reason, from the frame-problem artifact — and the predictive framework: the mechanism or model the loop predicts with, at a complexity matched to the problem, plus the discriminating thought-experiments it licenses.

    The campaign templates already link the overview by its literal relative path (../../PROJECT.md; one directory deeper, ../../../…) — there is no path placeholder to fill. Record anything you had to ask, so the next agent does not have to. If a frame-problem artifact exists, confirm its inventory rather than rebuilding it. 4b. Verification — capture the application-specific checks now. Decide how a result will be known to be right before the loop runs: the analytic limits, invariants, convergence, independent-method parity, and above all the field-standard checks, known artifacts, and gotchas for this problem, which is the part generic doctrine cannot supply. Mark verification versus validation (held-out data; calibration is not validation). These fill campaign.md §Verification — a few lines of campaign-specific checks, which verify reads and correctness-suite encodes as regression gates; the generic procedure and the V&V&UQ method live in the kit (verify, correctness-suite, the plugin's docs/SIMULATION_VALIDATION.md for sim-heavy campaigns) and are never restated per campaign. Inherit these from the frame-problem verification plan if one exists.

  5. Data / inputs. Source; fixed experimental data versus generated or simulated; splits to hold fixed; formats, shapes, paths. For a fixed-data campaign the dataset protocol collapses to a "verify exists, READ-ONLY" note.
  6. Levers vs FROZEN. What the agent may change (the orthogonal lever families) versus what is FROZEN (scoring math, core physics, geometry, data) versus READ-ONLY versus MUTABLE code.
  7. Experimental / hardware constraints. Fairness rules; for a hardware-in-the-loop campaign, the hardware safety limits as hard constraints, irreversibility, and what actions need confirmation.
  8. Compute resources, time budget, and disk budget. Detect the available hardware rather than asking for it (CPU cores, RAM, GPU via nvidia-smi). If it is relevant to the problem, such as a GPU for ML, record it explicitly in the scaffolding — the README Environment section and a note in campaign.md — so future agents do not overlook available resources. Ask the user only for what you cannot measure, chiefly the time budget, per-run and overall-loop. Use this to fill the run caps archetype-specifically: epochs for ML, replication depth for a cheap closed-form fit, expensive or irreversible runs for hardware.

    Tiered scout/confirmation caps only make sense when the problem supports a cheaper partial evaluation (fewer epochs, a partial dataset) and the full evaluation is genuinely long (over 10 min, and especially over 60 min). Otherwise fill a single-tier budget. When tiering applies, consider three tiers: a very short smoke check (does the code run at all?), a short scout (is the idea promising?), and a longer confirmation/validation run. If you cannot tell whether the evaluation suits partial runs, ask; otherwise propose the scheme.

    Size the disk cost per experiment too, not just the time. A loop that fills its volume overnight corrupts the save-set it was mid-write on, and the failure surfaces as an unrelated-looking gate error hours later. Derive the estimate rather than asking for it: sum what one experiment leaves behind from the artifact plan (topic 5 and roles/worker.md §The save-set). That is the save-set (result JSON, figure, and the scripts/ snapshot copy: usually a few hundred KB) plus the bulk artifacts that dominate it (trained weights and checkpoints, generated datasets, computed operators, raw field or trace arrays). Compute the bulk terms from real shapes where you can (n_points × n_channels × 8 bytes, a model's parameter count, the size of an existing example file), not from an impression. Then state the three numbers to the user: MB per experiment · projected total for the planned experiment count · free space detected.

    • Record the per-unit figure in configs/loop.json (unit_cost.typical_disk_mb). The driver refuses to launch, and halts at the loop top, below about three experiments of headroom.
    • If the projection is a material fraction of free space (say over 25%), stop and design the retention plan with the user before scaffolding: which bulk artifacts are kept, and which are dropped and regenerated on demand from the snapshot and seed. Fill it into roles/worker.md ({{RETENTION_PLAN}}, in roles/REFERENCE.md §Retention; the reasoning goes to campaign.md §Decision record's retention entry). The permanent record — scoreboard row, result JSON, figure, snapshot, narrative, verdicts — is never part of that trade: it is small by construction, and dropping any of it un-saves the result. Anything not deterministically regenerable (hardware measurements, external services, unrecorded seeds) is likewise never droppable.
    • If experiments are small, a few MB, say so explicitly and note that no plan is needed. A stated "not a constraint here" is what stops the next agent from re-litigating it.

    From the same unit-cost assessment, derive the supervisor loop knobs (configs/loop.json, which the driver reads; PLACEHOLDERS §Loop configuration): the unit's wall-clock and disk cost, and the iteration timeout (greater than the Extended cap). Also record campaign.reproducible (whether one unit can be independently re-executed from its config alone: seeded, not a one-shot measurement, not hardware-in-the-loop) — a fact a verifier reads when it chooses how to refute, not a trigger. There is no verification threshold to set: the preset (8b) decides each result's one check, and the runner records it in each results/checks_<NN>.json. There is no cadence or queue to set: the worker chooses each unit from the JOURNAL. The harness-specific worker/verifier model and effort selections are topic 9. Propose the values with their reasoning; the user confirms them with the spec. 8b. Autonomy and verification — one step, two choices. Both are the user's decisions about how far to trust the loop on THIS problem, so ask them together and record both in configs/loop.json.

    • Metric revision (campaign.metric_revision, {{METRIC_REVISION}}): "If the loop finds the frozen metric itself is wrong, may it revise it on its own — declaring, verifying, and re-baselining a metric epoch without stopping (autonomous) — or must it stop and ask (cautious, the default)?"
    • Verification preset (verify.preset, {{VERIFICATION_PRESET}}): "How hard is this problem, and how much of the budget should go to checking? Choose by how likely a plausible-looking result is wrong here." Then present the five presets verbatim from the plugin's docs/reference/verification-presets.md — the Summary and Choose it when lines of none · minimal (the default) · light · heavy · intensive. Do not paraphrase them: that file is the single description, and the driver prints the same table at preflight. Record the choice, and one line on why, in the campaign spec.
  9. Active harness, role models/effort, environment & version control. Ask which harness will launch this campaign first: Claude Code, Codex, or switchable. If switchable, also ask which one launches first. This choice does not prune the dual-compatible scaffold; it sets {{AGENT_BACKEND}}, decides which role settings to ask for, chooses the Step 4 preflight, and tailors the handoff. A later switch uses --backend after stopping the current supervisor.

    For each harness the user plans to use, ask for the worker and verifier model and effort independently (the verifier pair also runs the end-of-loop claim audit). Explain that blank means inherit that harness's configured default, which is the least maintenance but is not a reproducible pin. Claude uses model + claude_effort; Claude effort may be low, medium, high, xhigh, or max, subject to the selected model and Claude Code version. Codex uses codex_model + codex_reasoning_effort; Codex effort may be minimal, low, medium, high, or xhigh. The direct Codex verifier follows the worker's Codex pair; the established Claude nested verifier follows the verifier block's Claude pair. Fill the unused harness's fields with empty strings when the user chose only one. Seed sensible model defaults for the selected harness, but let the user confirm them because model tiers shift with every release; leave effort blank unless the user chooses it.

    Ask which environment the user wants, and default to proposing a fresh virtual environment (a new venv or conda env). Record the choice and setup commands in the README Environment / Quick-start. Offer git, do not impose it: recommend it for shared, large, or long-lived projects, but make clear that the per-experiment snapshot is the reproducibility baseline either way. A small solo campaign can skip git without losing reproducibility (docs/reference/version-control.md). Record the choice; never make git a precondition for starting.

  10. Existing code & tools. What runnable code or data already exists to reuse, and which libraries to prefer (mature and free over from-scratch).
  11. Internet / literature level (always ask; 3 options). This sets ongoing literature use during the loop: none (no ongoing searches; remove the literature-search rule), only when needed (search when stuck, the default), or extensive (proactively search at setup to seed ../../PROJECT.md/LITERATURE.md, and periodically thereafter, not only when stuck). Independently, if the user provided any starting literature, summarize it in LITERATURE.md and keep the file, even when ongoing search is none. For needed and extensive, prefer arXiv and convert PDFs efficiently.
  12. User's own ideas (always ask). Ask the user for (a) any ideas or directions they would like you to try, and (b) anything that should not be tried: out of scope, known dead ends, forbidden approaches. Record the to-try ideas in the opening JOURNAL — ranked under ## What to try next and why, attributed to the user, with the first as the NEXT: line — and fold the do-not-try items into campaign.md §FROZEN / out of scope. User ideas are inputs, not a ceiling; you remain encouraged to propose well-founded novel ideas of your own (CORE: novelty and rigor are compatible).
  13. Multi-stage plan — propose it proactively. Many complex tasks are best done as several sequential loops rather than one: validating and improving on a simpler subcase first usually yields a more powerful, better-validated final result. Design a staged plan and propose it. Forward-model-before-decoder is one case. Others include optimizing on idealized or simpler physics first, to set a baseline and a solid procedure while the physics is simple, before the real system; or inserting a stage that produces an experimental protocol for the user to carry out, such as a gauss-meter measurement, whose results feed the next stage. Example — permanent-magnet array from measured surface fields: (1) optimize the arrangement with idealized magnets, which establishes a baseline and a solid optimizer while the physics is simple; (2) design a measurement protocol the user runs to characterize their real magnets' imperfections; (3) optimize the arrangement of the imperfect magnets, building on (1) and the data from (2). Each stage is a sibling campaign: its own campaigns/<NN>_<slug>/ with its own campaign.md, ledger, and lineage row (Step 3, Sibling campaigns & multi-stage). Scaffold stage 1 now; confirm the staging with the user and record the later stages as planned successors rather than building them up front.

Step 2 — write the campaign spec and get sign-off (anti-drift checkpoint)

Before generating anything, write a concise campaign spec: the archetype(s) that apply, the FROZEN metric with its signed-off plausible range, baseline, guardrail and diagnostics, levers, FROZEN/READ-ONLY/MUTABLE, constraints, caps and whether tiering applies, the data plan, the detected compute, first-launch harness and role model/effort settings, chosen environment, the multi-stage plan if any, and the values you will use for the key placeholders. Cite the source for any value pulled from a document, paper, or dataset, with its provenance tag (PINNED / REPORTED / INFERRED / RECALLED) from the Step 0.6 ledger, and call out every load-bearing fact still at INFERRED or RECALLED as an open question for the user to resolve before the baseline run. The same rule binds every steering statement the spec proposes - a landmine, an artifact to guard against, a range, a direction: each carries its source or is marked as your own inference, and an inference never enters campaign.md (Step 3). End the spec with the adaptations: how this scaffold departs from the templates - sections dropped or restructured, procedure changed - one line each with its reason; and, asked one by one, every rule or constraint on workers the templates do not already carry. Show the spec to the user and get explicit sign-off or corrections. This is the checkpoint that keeps the build aligned with the original intent.

Step 3 — generate the campaign tree

Copy the linked PhysKit package directory ../../templates/campaign/ into the target directory and fill it from the spec, obeying PLACEHOLDERS.md. The copy is what creates the tree: take the whole directory, not a hand-picked subset. The bullets below say what to fill; they are not the file list. Files the copy brings that no bullet dwells on survive the fill unless the signed-off adaptations drop them. Five are checked by the Step 4 validator: roles/worker.md (the worker's contract - every instruction it acts on; roles/verifier.md and roles/capstone.md are the other two spawned roles' contracts, roles/REFERENCE.md the reference the worker's contract points at and no spawn reads whole), configs/_template.json (the per-experiment config template), scripts/frozen_core.py (the FROZEN-core file list and content hash that run_campaign.py stamps and guard_promotion.py recomputes at promotion), tests/correctness_checks.py (the standing suite), and README.md (the cold-reader front door; its §Setup status points at preflight). configs/RECOMMENDED.json (the current-best pointer) is not validator-checked, but the promotion rule writes to it, so keep it too.

  • Fill the four role files (roles/worker.md, roles/verifier.md, roles/capstone.md, roles/REFERENCE.md) from the spec like every other template. The loop's per-unit procedure lives in the worker's contract, not in campaign.md; the runner's own prompt is one opening line, so a rule that is not in the role file does not reach the agent.
  • Resolve every {{PLACEHOLDER}} clearly, explicitly, and concisely, specific enough to act on without the interview that produced it. Where the user deferred a value, write an explicit TODO noting the deferral, never a silent guess.
  • Resolve every [[OPTIONAL — … ]] block: delete it if it does not apply, which keeps the slim core slim; keep with a note a universal section that is merely not yet wired, marked (DEFERRED). Include the hardware-safety block only for archetype E. The ceiling blocks are one set: ceilings appear in the Metric, Decision/promotion, and Before-accepting sections, so keep or delete them together. For a fixed-data campaign, delete campaign.md §Dataset / inputs entirely, leaving in its place the READ-ONLY note its [[OPTIONAL]] marker quotes.
  • Resolve every [[KEEP — … ]] tag: keep what it marks - the heading, key, column or line its consumer reads, spelled exactly - and delete the tag. The text around it is yours to shape.
  • Apply the signed-off adaptations. Rewording to fit the problem needs no sign-off; a dropped section, a changed procedure, or a new rule the user did not see in Step 2 does not enter the scaffold.
  • Setup state is mechanical, not a ledger. README.md §Setup status ships as a static pointer: --preflight is the authoritative answer to "is this runnable?" (it refuses to launch and names each missing item with its fix — and Step 4 requires it green before handoff), and the downstream gates enforce the rest — never author a to-do table restating what the code computes live. Setup tasks that preflight cannot know about and this session cannot finish — a calibration to pin, a file only the user can stage — go in at the top of the opening results/JOURNAL.md ## What to try next and why, before the user's ideas to try, so the loop's working list starts with them.
  • Resolve every [PROPOSED] tag: present each speculative discipline to the user to keep or cut; drop the tag on kept ones, and delete cut ones.
  • Strip the .tmpl suffix on write (campaign.md.tmpl → campaign.md, and so on).
  • Always-on context. Render CLAUDE.md.tmpl once, write CLAUDE.md, then byte-copy that rendered file to exact-case regular AGENTS.md in the campaign root. Claude Code and Codex auto-load their respective identical twin; never edit only one. A plugin's docs/ are not auto-loaded, so the always-loaded rules live here per campaign. Every line of it is re-injected every turn, so it carries only what the running loop acts on — the integrity core, the research reflexes, the hard gates, and the routing table: no kit-design rationale, no scaffolding-time instructions (the fresh-session launch is Step 5's handoff message to the user), and no commentary on how the file was produced. The reasoning behind its rules is the plugin's docs/CORE.md and the skills it routes to — referenced at the stamped version, never inlined: reference material must not ride in every worker's context on every turn.
  • Stamp the kit version — the campaign's doctrine and tooling provenance. Fill {{PHYSKIT_VERSION}} (it appears in campaign.md's header and atop results/logbook.md) as vX.Y.Z (<ref>, YYYY-MM-DD): read the version from the active harness manifest under the resolved PhysKit package root (.codex-plugin/plugin.json on Codex, .claude-plugin/plugin.json on Claude; require equality when both are present). Obtain <ref> with git -C "<resolved PhysKit package root>" rev-parse --short HEAD when that root is a Git checkout (omit it otherwise), and add the scaffold date. Every kit release is tagged, so this pin recovers the exact skills, reference docs, and templates the campaign was generated under — the whole kit, not just a doctrine excerpt. It is a record: never update it to a newer version, which does not retroactively govern a run that already happened (invalidated results fork instead — fork-campaign).
  • The enforcement layer. Generate .claude/settings.json and .codex/hooks.json (from their .tmpl files) and scripts/loop_heartbeat.py, the Stop hook guarding attended sessions in a live campaign, plus scripts/supervisor_lock.py, the process-held cross-platform lock that permits only one live --supervise owner, scripts/guard_promotion_codex.py, the Codex apply_patch perimeter adapter, scripts/codex_verifier.py, the Codex-only direct-refuter policy, and scripts/sync_agent_instructions.py, the read-only resume gate and explicit twin reconciler. That hook refuses an end-of-turn that is not the human's results/STOP or a recorded results/stop_request.json, and its block message routes toward launching the driver; it stands down for driver-spawned supervisor-mode workers, whose per-unit end-of-turn is earned, and for any attended session while the driver's results/.driver_alive sentinel is fresh. Also generate scripts/guard_promotion.py, the PreToolUse hook that denies a mislabelled promotion write — a configs/RECOMMENDED.json update or a scoreboard edit introducing a CONFIRMED/NEW BEST token that does not carry the promoted result's label exactly as its results/checks_<NN>.json holds it, or that promotes a result a check refuted. Fill {{PYTHON_CMD}} with the python invocation detected in Step 0.5 (py -3 on Windows, python3 on macOS/Linux) so both harness hook commands run on this machine.

    Codex project hooks are trust-gated by the host. At handoff, tell the user to open a fresh Codex task from the campaign root, run /hooks, and review/trust the exact .codex/hooks.json definition before relying on it. The validator checks the file, matcher, command, and synchronous mode; it cannot prove host trust. This adapter covers canonical apply_patch; do not claim that specialized mutation paths which opt out of standard hooks are guarded. A protected promotion must be its own single-file patch: atomic co-mutation could otherwise invalidate the verdict, correctness state, or frozen core after the guard inspected their pre-patch state.

    Generate configs/loop.json: its campaign block from topic 3 (noise model, the resolution knob, the version tags), its loop knobs from topic 8, and its initial backend plus role model/effort settings from topic 9. Fill configs/_template.json, the do-not-run per-experiment config every unit is copied from, with its input paths, lever fields, and run and cost fields from topics 5–8. Fill configs/RECOMMENDED.json, the current-best pointer, mirroring exp 000 until the first promotion. Create the artifact-gate directory results/verdicts/.

    The user's topic-12 to-try ideas go into the opening results/JOURNAL.md §What to try next and why — ranked, attributed to the user, and phrased as questions, never as a candidate named as the expected answer: a setup-written first instinct is the one thing in the tree that carries no evidence, and a campaign has run twenty units on one. Write "does <lever> account for <observation>, and what would show it?", not "<lever> is probably it". A later JOURNAL rewrite re-states them from the evidence then on the ledger, never by quoting this opening one. Add no directions of your own.

    Verify the layer is wired before handoff: .claude/settings.json registering guard_promotion, .codex/hooks.json registering the synchronous canonical apply_patch adapter, both gate directories present, and loop.json parsing. A tree without them runs with no gates at all, and a tree assembled by hand rather than by this skill is the case where that happens; campaign-status asserts all of it.

  • Campaign code — filled here, implemented in Step 3.5. Fill scripts/metric.py (the FROZEN scorer and --selftest) and scripts/experiment.py (the campaign's code: the five hooks and the seams) with the campaign's names, paths, and metric wiring during the template fill; the real implementations — the scorer's math and the three experiment hooks — are Step 3.5's job, in this session, never deferred to the loop. scripts/run_campaign.py is the kit's machinery (gates, checks, snapshot, driver): it is rendered, never implemented per campaign, and replaced wholesale on upgrade. Leave scripts/frozen_core.py's CORE_FILES_DEFAULT alone, since editing it per campaign forks the one authority; if this campaign's frozen physics spans more files, such as a shared simulator or a forward model from topic 6, declare them in configs/frozen_core.json instead. tests/correctness_checks.py, the correctness suite, ships as skip-stubs. From campaign.md §Verification, seed at least the analytic-limit and invariant checks for the frozen forward model; see the correctness-suite skill. Declare each check's row (@check(row=...): invariants — cheap output properties, every unit; machinery — fixed-input checks of the code; gates — expensive per-result gates): the runner schedules by row under the verification preset and meters what fires every unit. The four contract checks the template ships (baseline_reproduces, noise_floor_measured, metric_discriminates, synthetic_recovery) are the setup audit's (Step 3.7). It is green-gated: run_campaign records its status before every unit, and the promotion guard reads it.
  • Author the goal, verification, and engineering content. The campaign's orienting answers live in campaign.md now: §Mission carries the goal, the decision it serves, and what "done" looks like ({{DECISION_SERVED}} / {{DONE_CRITERIA}}) - written so it holds under any preset: the label a result earns is the preset's, so "done" never asks for a verification ("a verified primitive") or a pass condition it does not define ("survives the panel"); §FROZEN / out of scope carries {{OUT_OF_SCOPE}}. There is no expectations section in the contract: the hypotheses the campaign tests live in the opening JOURNAL's §Live hypotheses, as questions, where the next worker re-judges them from evidence. The goal-linked task decomposition from frame-problem goes into results/JOURNAL.md — the NEXT pointer and §What to try next and why, with the reasoning — which is the campaign's only forward-looking record; there is no separate orientation file to keep in sync with it. When the field has them, fill the project doc's optional "Where this sits in the field" section (../../PROJECT.md): the gap or novelty, adjacent work, the threshold ladder and named precedents for the key quantity, and what not to claim. Ask the user or draw on the topic-4 literature; skip it if the quantity has no canonical thresholds, and never invent them. Fill campaign.md §Verification with the application-specific checks from topic 4b, the gate verify and correctness-suite consume. Every §Verification line and every §Known landmines row names its source (the user, a derivation, a reference, a prior campaign's ledger id or verdict); the landmine table's fourth column is that source, and the validator refuses a row without one. What you merely suspect - a pitfall you would guess at, a direction you expect to fail - goes into the opening JOURNAL (§Open questions or §What to try next and why) marked [setup inference], where the worker who tests it can drop it. It never enters the contract: a contract line cannot be argued with from inside a spawn, and a guessed landmine has steered campaigns away from paths nobody had tested. The contract is the problem's facts and the campaign's constraints, not a strategy - the workers have their own. Seed campaign.md §Decision record with the Step 0.6 source-authority ledger: one ED per load-bearing fact resolved from a source, giving its value, the authoritative artifact it is PINNED to, the provenance tag, and the lower-authority sources quarantined and why. An empty section is correct only if there was a single source.
  • Reference docs (see topic 4). The physics, conventions, instrument, and durable-decision background is the project overview ../../PROJECT.md, so the campaign tree carries no copy of it, and the templates already link it by literal relative path. Update it in place if this campaign added physics or conventions — run-constraining facts into §Standing constraints, depth below. Record the detected compute and the chosen environment and setup commands in the README (Environment / Quick-start), note available compute in campaign.md §Targets, and register any new docs in the README.
  • Project wiring — always, from Step 0.1:
    1. Reference the canonical data as ../../data; never copy it into the campaign.
    2. Append the lineage row — ../../PROJECT.md §Lineage Campaigns table: index <NN>, campaign name, date, the approach tag, the FROZEN primary metric, what changed from the predecessor (or "— (first)"), headline result (leave pending until exp 000), verified discoveries (none yet), and status active.
    3. Resolve campaign.md §Relationship to sibling campaigns: keep it and fill {{SIBLING_FROZEN_MUTABLE}} and {{CONSUMED_ARTIFACTS}} if this campaign is coupled to a sibling (what it FREEZES versus MUTATES; which promoted ../../<dir>/<artifact> it consumes); delete it if independent. Coupling decides this block, not project membership, so the first campaign in a fresh project deletes it.
    4. Keep the campaign CLAUDE.md project-context block.
    5. A verified finding that is a fact about the system, rather than a config that scored well, is later recorded in ../../PROJECT.md §Updates from discoveries, citing its verdict artifact by path — the pointer that survives fork-and-archive. Note this in the handoff so the loop agent knows the path.
  • Literature (per the topic-11 level). The ongoing-search rule and the LITERATURE.md file are decided separately:
    • Ongoing-search rule in LITERATURE.md §Literature search rule: for none, resolve the [[OPTIONAL — literature search enabled]] block as delete; for only-when-needed and extensive, keep it and fill {{LITERATURE_LEVEL}} with the chosen intensity (search-when-stuck versus proactive-and-periodic).
    • LITERATURE.md itself: keep it and seed it with summaries of any literature the user provided (prefer arXiv; convert PDFs; cache), even when ongoing search is none. Delete the file only when ongoing search is none and the user provided no starting literature. When kept under none, treat it as a static reference: keep the summaries and references, and drop the roadmap and next-search sections.
  • Simulation validation (conditional on topic 4). If the campaign has a simulation aspect, author docs/SIMULATION_VALIDATION.md for it, a tailored V&V&UQ plan from ../../docs/SIMULATION_VALIDATION.md, and wire its gates into the core docs: a verify-before-validate gate, under which the simulator must pass code and solution verification before any simulator-based result is trusted, plus the sim-to-experiment guardrail in campaign.md §Mind the simulation-to-experiment gap. Record the simulator's assumptions, domain of validity, and V&V status in ../../PROJECT.md (§Equations and methods; domain of validity is a standing constraint), and keep calibration is not validation by holding out independent validation data. Register it in the README.
  • Seed the user's directions. Put the user's ideas to try (topic 12) in the opening JOURNAL — ranked under ## What to try next and why, attributed to the user, the first also the NEXT: line — and fold the do-not-try items into campaign.md's FROZEN and out-of-scope list.
  • Sibling campaigns & multi-stage (project model, §5). A second track, approach, or stage is always a sibling campaign: its own campaigns/<NN>_<slug>/ with its own campaign.md and ledger, coupled via campaign.md §Relationship to sibling campaigns and the promoted project-level artifacts (../../<dir>/<artifact>_<ver>, each with a §Lineage promotion line). Scaffold each by re-running this skill in the project, with a new lineage row per sibling.
  • The co-designed metric IS the campaign's first metric epoch, and it needs no record of its own: the runner binds the scorer's hash on the first experiment, and from then on a scorer change with no new epoch (a metric_version bump plus an entry in configs/loop.json campaign.metric_epochs) refuses to spend. campaign.metric_revision ships cautious (a metric revision the loop wants raises ESCALATE.md and waits); say so in one sentence at handoff — the user moves the dial, never the loop.
  • Seed results/scoreboard.tsv with its header. Leave results/logbook.md (append-only full record) and results/JOURNAL.md (the campaign's working memory — the NEXT: line and its sections) with their intros, the first NEXT pointer naming exp 000, and §What to try next and why holding the user's ideas under the rule above. The logbook, scoreboard, and diagnostics/ logs are append-only and never read in full, which is what keeps the record complete without bloating the loop's context. If the user wants hard separation, the logbook and diagnostics can be redirected to a path outside the campaign tree; the default is in-tree for reproducibility.

Step 3.5 — implement the campaign code (finish setup; do not hand off stubs)

This session holds the whole interview — the metric definition, the data paths and shapes, the conventions, the topic-10 code to reuse. No later agent will ever be better positioned to write this code, so write it now; deferring it "for the loop" hands the work to a colder context and guarantees the fresh session a red preflight. Sub-agents may draft; this session verifies.

  1. Implement scripts/metric.py for real — the topic-3 co-designed FROZEN scorer, not a NotImplementedError stub, with the signed-off PLAUSIBILITY_BOUNDS (its trivial end computed now if topic 3 could not) — and run python scripts/metric.py --selftest until green. The selftest exists to catch sign and convention bugs before they can corrupt the campaign; an unimplemented scorer defers exactly those bugs. The scorer is FROZEN from here on: locked against quiet change — a later revision is a declared metric epoch, never an in-place edit.
  2. Implement the three campaign hooks in scripts/experiment.py — validate_method (the lever contract), run_experiment (the experiment dispatcher), load_target (the target/data loader) — wiring the topic-5 data paths and the topic-10 existing code. Implement save_figure() so it actually writes the file, and fill the seams the campaign needs (campaign_scoreboard_fields for its scoreboard columns, campaign_result_fields for the evidence behind the number, baseline_config for exp 000, campaign_checks for checks only it can make; ZERO_EVALUATION_MODES if a unit can record no score - a decision-record entry, a procedure - whose configs then name a _deliverable; see the seam file's header). Helpers and models go in experiment.py or sibling modules under scripts/; the runner is never edited.
  3. What this session genuinely cannot finish — a data file only the user can stage, an instrument to connect, a credential — is left as its named preflight failure, written as an opening results/JOURNAL.md NEXT item, and handed over by name in Step 5. Never silently, and never as an unimplemented hook: code is always this session's to write.
  4. Hardware campaigns (archetype E): wire the hooks to the shipped machinery, never re-implement it. Write configs/instrument.json per the linked ../../docs/reference/instrument-contract.md (experiment types, required_keys vs defaults, range tables, cost model, serializer, the two roots) — every cap a constraint, never a lever. Then: validate_method calls the emission gate (../../scripts/hardware/instrument.py, gate(); reject-don't-fill, teaching-text reasons); run_experiment submits through the declared transport and waits on the sibling markers, with ingest — the declared reader plus the quarantine stamp — at its tail; load_target stays the metric's reference loader, never the ingest path. Validate the channel with the resolved ../../scripts/hardware/dispatch.py preflight command alongside Step 4's gates — a mis-named order directory is ignored silently by the lab side, and preflight catches the naming while the readiness probe (an emit of a throwaway unit, then wait; the contract's silent-wedge mitigation) proves the channel end to end before the campaign starts.
  5. Run experiment 000 now — python scripts/run_campaign.py --config configs/exp000_baseline.json, under the same gates every unit runs under. The setup session is the best-placed agent that will ever see this campaign, and the floor is machinery, not direction: a fresh worker that has to establish it spends its first units on plumbing. Check the gate criteria from topic 3 (the expected floor or range; the sanity checks that prove the chain is sound - a known input maps to a known output, the metric collapses on a null input, units come out right); a floor you cannot trust makes every later delta meaningless, so STOP and debug before handing off. Then --record it (the analysis names the floor and what the chain validated), fill campaign.md §Decision record ED-000 (the baseline floor: methods, command, criteria met, the floor), and write the opening results/JOURNAL.md from the result: §Where we stand carries the floor with its ledger id; §Live hypotheses and §What to try next and why carry the user's ideas as questions (below). Preflight then shows the exp000 row, and the loop's first worker chooses a real unit.

Step 3.7 — the setup audit (before anything is frozen)

The cheapest verification a campaign can buy is the one that happens before the first unit: a contract error — a floor taken from a paper, a baseline nobody re-ran, a metric that does not measure the mission — is paid for on every unit afterwards, and no per-unit verification preset reaches it. Fill the four contract checks the suite template ships as real checks, and run the suite until they are green:

  • baseline_reproduces — exp 000 re-scored from its config equals the recorded baseline within the measured noise;
  • noise_floor_measured — the floor or plausibility threshold in campaign.md derives from replicates run in this tree (three or more seeds), not from a published number; the check asserts the config value against the measured spread;
  • metric_discriminates — a pair of synthetic inputs the mission says should order one way; the FROZEN scorer orders them so (the judgement "the metric measures the mission" made executable);
  • synthetic_recovery — for a fitter, decoder, or model-improvement campaign: data generated from known parameters under the campaign's noise model fits back within tolerance; any other archetype returns skip: not a fitter. Leave one line under ## Where we stand in results/JOURNAL.md: Setup audit: baseline 000 = … ± … (N seeds); floor …; metric discriminates; recovery ± …; contract checks green. The validator (Step 4) refuses a suite that lacks the four names unless tests/CORRECTNESS_NA.md records why; preflight prints their status with the rest of the suite.

Then the steering audit: read campaign.md above §Decision record once more, as the worker will - every landmine row, every §Verification line, the plausible range and its basis, every "held fixed" - and every line you added to any file the templates did not carry; for each ask where it came from. Anything you cannot source, or the user did not sign off, is deleted or moved to the JOURNAL marked [setup inference]. A setup agent knows no more than the workers that follow it; what it writes into the contract binds them all the same.

Step 4 — validate (two deterministic gates)

Run both gates on the generated tree and fix anything they flag:

  1. The bundled validator:
python "<resolved path to this skill>/scripts/validate_campaign.py" <target_dir>

Resolve the executable path from the linked scripts/validate_campaign.py; the shell's current directory is not the resource base. It fails if any literal {{…}}, dangling [[OPTIONAL, stray [PROPOSED], or [[KEEP tag remains, or if a required file is missing. 2. The driver's own launch check, from the campaign root, with the Step 0.5 python and the first-launch harness selected in topic 9:

<python> scripts/run_campaign.py --preflight --backend <active harness: codex|claude>

This is the same check --supervise runs at launch, so passing it here makes scaffold-complete and launch-ready the same bar by construction — any future preflight check automatically becomes a scaffolding obligation. Fix every failure (each names its fix); the only failures allowed to survive to handoff are the Step 3.5.4 kind this session genuinely cannot resolve, each named in the handoff. For a switchable campaign, run both backend preflights when both CLIs are available. If the alternate CLI is unavailable, record that its preflight remains unverified rather than claiming both launch paths passed.

A campaign with an unresolved token, a failing selftest, or an unexplained red preflight is not done.

Step 5 — hand off (and start a fresh session)

Tell the user what was created — including that setup is finished: the scorer and hooks are implemented, --selftest passes, and preflight is green (or name each handed-over failure and its fix). Name the destination up front: the loop runs in supervisor mode, launched and watched through the run-campaign skill. The fresh session's job is physkit:run-campaign ($physkit:… in Codex, /physkit:… in Claude), never executing the loop by hand.

The critical handoff rule: do not start the loop in this setup session. The campaign's managed instruction twin and enforcement hooks only take effect in a task/session that starts in the campaign folder. This setup session was already open when they were written, so it has loaded neither. If Claude is the first-launch harness (or the alternate in a switchable campaign), hand over this established launch command:

cd "<absolute path to campaign dir>"
claude --dangerously-skip-permissions

Give the absolute path, never one relative to this setup session's cwd. The fresh session's shell has no memory of where this session was running: a relative or partial path that resolved correctly here will point at the wrong place, or nowhere, in the new terminal. Resolve the campaign dir to its full path before handing over the command. (add --plugin-dir "<KIT_DIR>" if running the kit per-session rather than installed).

If Codex is the first-launch harness (or the alternate in a switchable campaign), tell the user to open a fresh task with that absolute campaign directory as its root, confirm AGENTS.md is active, and run /hooks to review/trust .codex/hooks.json before the first launch.

In that fresh session, the sequence is:

  1. Confirm the session, clear any handed-over items: confirm the active instruction twin is loaded (/context and CLAUDE.md in Claude; AGENTS.md in Codex); if Step 4 handed over preflight failures this session could not resolve (staged data, an instrument, a credential), clear them — they are named in the handoff and as the opening JOURNAL NEXT items, and run-campaign's preflight will name them again until they are fixed.
  2. Invoke the harness's physkit:run-campaign skill. That session becomes the loop's cockpit: it runs the driver's launch-settings confirmation (a concierge/role mismatch is informational), runs the driver's preflight and translates any failure into its fix, launches --supervise in the background, narrates progress from the driver log, and relays the user's steering into results/inbox/. Experiment 000 already ran in Step 3.5 (its floor is the exp000 ledger row and campaign.md §Decision record ED-000); the loop's first worker chooses a real unit. Fallback, if no skill is available: hand over <python> scripts/run_campaign.py --preflight --backend <harness>, then --supervise --backend <harness>, writing out the actual python command detected in Step 0.5, e.g. py -3, and never the literal placeholder.

The driver runs its preflight (agent CLI, loop config, metric self-test; each failure names its fix, and --preflight runs it standalone), spawns a fresh worker agent per unit with no unit assigned (the frozen spine auto-loads; the driver compiles campaign.md, the role file, the standing constraints, the JOURNAL and the config template into the prompt — the brief, sized at preflight and at every spawn), and enforces the artifact gates. The worker chooses the one experiment the campaign needs now from the JOURNAL. Failures split by kind. An operational failure — a worker that dies, hangs, or leaves its unit unvalidated — is retried, then counted, and the loop continues. The driver hard-halts only on integrity conditions (results/ESCALATE.md, a corrupt results/driver_state.json) or systemic failure (several units failing in a row), and stops for the user when a worker writes **NEXT:** STOP - <why> (the claim audit runs first). Loop knobs live in configs/loop.json, which is user-owned; roles/worker.md §Long solves, escalation, and your exit explains the contract.

Attended work means watching and steering through the cockpit, not executing by hand. A by-hand --config run is for debugging a single experiment; it is not a mode for running the loop. The interactive-executor pattern is not used: it produced unbounded context, docs that were never re-read, premature stops, and verification decay — the failure record that motivated the driver.

  • The campaign lives at <project>/campaigns/<NN>_<slug>/ and its opening lineage row is status: active. Hand over the absolute path to the campaign dir, not the project root. Remind the loop agent that the shared data and promoted artifacts are ../../data and the purpose dirs (../../scripts, ../../models, …), referenced and never copied, with the ancestor ../../CLAUDE.md carrying the project rules; that ad-hoc follow-up work belongs in ../../sessions/<date>_<slug>/ rather than in this tree; and that a verified system-level finding is later recorded in ../../PROJECT.md §Updates from discoveries, citing its verdict.
  • If the loop runs agents beyond the main session — Task subagents, or SDK-spawned generation and critic roles — those contexts inherit nothing from CLAUDE.md; their prompts replace the doctrine stack. Before the first real experiment, run the prompt-surface audit (docs/reference/prompt-surface-audit.md): capture each role's context as rendered, including dynamically generated blocks, and verify every relied-upon rule is physically present. Re-run it after any prompt change.

Notes

  • This skill scaffolds; it does not run the loop. After hand-off, the loop-running agent executes campaign.md and its role file.
  • Never overwrite existing files without confirmation. If the target already holds a campaign, add a sibling campaign in the project (a new campaigns/<NN>_<slug>/ and lineage row), or stop and ask.
  • Imported, relocated, or hand-built trees silently lose the enforcement layer. A tree not generated by this skill — copied in, assembled from frozen inputs, or moved — usually lacks .claude/settings.json, scripts/guard_promotion.py, scripts/loop_heartbeat.py, the gate directory (results/verdicts/), and configs/loop.json. The promotion gate, the loop gate, and the driver are then all absent, and the doctrine prose in CLAUDE.md is decorative. Re-run the Step 3 enforcement-layer wiring for any such tree, and lift it into a project home per Step 0.1; campaign-status asserts the whole set.
  • The metric and core physics you scaffold are FROZEN by construction — locked against quiet change; a later revision is a declared metric epoch (roles/REFERENCE.md §Metric epochs), never an in-place edit. Say so to the user, and make the --selftest real before exp 000 so a convention or sign slip cannot corrupt the whole campaign.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.