Exp Batch: confirm necessity → clarify config → preflight → run → land metrics

Small but complete: one branch, done right, plottable. Do not expand scope sideways into other branches; do not shrink rigor to save time.

Apply goal-first: run what supports the learning question, Pareto/ablation coverage, agreed scientific claim or a live decision. Stop when that purpose and required coverage are satisfied; incidental findings do not silently expand the batch. A scientifically needed matrix is not wasteful merely because it has many cells.

1. Confirm the branch is still necessary

• Check project memory, the campaign plan, and existing results: has a probe already pruned this branch? Is the metric already measured under an equivalent configuration? If yes, say so and stop (or record why the earlier pruning does not apply).
• Restate in 2-3 sentences: what this batch adds to the final table/figure, which cells it fills, and its acceptance metric. No plan document.
• For a new algorithm branch, use exp-workflow to propose and agree the evidence route: an existing result, a proxy or a direct exploratory/confirmatory matrix. Costly full-model runs need code-readiness evidence, not a mandatory prior proxy or positive result; do not make the researcher invent the study.

2. Grill only unresolved config decisions

• Facts (paths, device availability, cache locations, prior configs) are yours to inspect. Ask the user only for choices that change the scientific meaning: hyperparameters not pinned by protocol, seed count, which split, callback-vs-reload coverage, formal vs exploratory status.
• Keep the batch's metric contract identical to sibling runs: same splits, same metric definitions, same surfaces. A batch measured differently cannot share a figure with the others — flag any deviation explicitly in the record.
• Ask one round of structured questions; a second round is allowed only when an answer opened a genuinely new scientific choice. If config decisions explode, the work is actually a campaign — propose exp-campaign.

3. Preflight: environment and resources

A run that starts without this pass is the run that fights someone else for the default device. It takes seconds, needs no approval, and happens before every launch — probe or formal, long or short. Skipping ceremony is the point (no scaffolding, no defensive code, no extra approval rounds); this one look is not ceremony.

• Environment: interpreter/environment activation, working directory, the dependency versions that are actually importable now, input paths present, output and checkpoint directories writable.
• Know the platform first: use the device platform the project's memory, conventions, or the session already names (CUDA / Ascend / MUSA / PPU-class / …). If it is genuinely unknown, read it off the environment (installed toolkit, device files, what the project's own launcher sets) or ask once; then work in that platform's own tool and variable vocabulary — wrong-platform errors are self-inflicted and cost runs.
• Resources as they are, not as they usually are: read the live device state with whatever this box uses — nvidia-smi, Ascend's npu-smi, MUSA's mthreads-smi, or the platform's equivalent — for utilization, memory, and which process owns each device (other people's jobs and this project's own parallel branches both count). Decide from that read and pass the device explicitly through the platform's visible-device variable (CUDA_VISIBLE_DEVICES, ASCEND_RT_VISIBLE_DEVICES, MUSA_VISIBLE_DEVICES, …); never let a run fall through to the default device. Disk for checkpoints/logs, and RAM/CPU when the job is not device-bound, come from the same look.
• Collisions: leftover jobs from earlier attempts ("did the last one really die?") and sibling branches are the usual hidden claim on a card; owner, name, and elapsed time in the process list settle it.
• Cost: estimate the whole batch—shared loading, per-configuration preparation/compilation/scoring and repeats—not just one short run. A small dataset can still multiply costly loading or inefficient evaluation across many configurations.
• Then run under the project's rules on who launches what. If the free resources do not fit the configuration, do not start and do not silently shrink it: state what is short, and pick between waiting, another device, or an agreed config change.
• Batch/model readiness: before a batch through the real-model evaluation path—including small-dataset exploratory matrices—or any costly run, follow exp-workflow: inspect with pytorch-perf-tuning, preserve semantics, check numerics and a tiny real-model path, and assess aggregate cost. Reuse valid evidence rather than retuning every cell. Cheap local one-off probes need only their relevant checks; a stage label is not an efficiency exemption or a reason for deep tuning.

4. Implement in this session, run under project conventions

• Implementation happens here, in the same session, after the researcher issues the go-ahead. No fresh-session handoff (that is exp-campaign's model); the researcher drives execution timing, and formal runs follow the project's rules on who launches them — the researcher runs them or grants explicit permission.
• Module placement: put each piece in the module it belongs to; keep experiment-specific code inside the experiment's own directory with its predictable layout, so the experiment is findable by name. Do not create a shared abstraction for one consumer; do split into separate modules when sibling branches would otherwise contend on one file.
• Loose coupling: consume stable shared libraries through their existing interfaces. Never modify shared internals to fit this batch, never add batch-specific branches to shared code. If an interface genuinely lacks something, stop and escalate — that is a campaign-level decision.
• Ponytail research code: the minimum code that runs the batch correctly. No defensive programming: no speculative validation, no "just in case" branches, no swallowed errors — on error, exit with the real error. When a run fails, first suspect the environment (device contention, full disk, stale cache, conflicting job) before the code; only change code when the evidence says the code is wrong.
• Watch for the known failure modes of this project class: OOM (yours or the machine's), NaN, checkpoint serialization, degenerate metric collapse (e.g., all-same-answer), and eviction — a neighbour's job or the kernel taking the device mid-run. On failure: preserve the failing artifact/log, isolate minimally, keep a failure-status row without inventing valid metrics or filling missing values with zero.
• Separate "environment" from "code" before acting: exit code, the log's last lines, and a fresh device read with the platform's own tool (is my process still alive? who owns the device now?) tell an evicted job apart from a real bug. An evicted run keeps its evidence, gets a free device chosen from the live read, and is requeued with the deviation recorded; only change code when the evidence says the code is wrong. A config changed to fit the resources must be flagged — it can break the metric contract with sibling runs.
• These shapes are the pattern, not the whole space. Unforeseen failures are yours to handle on the spot: decide, preserve the evidence, and say what you did. Do not stall waiting for a rule that does not exist, and never let a failure pass as a result.
• Negative results are results: record them with the same schema as successes. Distinguish process completion, data/numerical QC and scientific support; exit code 0 does not establish the latter two. Preserve retries as separate attempts, not independent scientific repeats or replacements for failed evidence.

5. Wait on long runs with later (delegated execution)

Long runs (device hours) must not hold the turn and must not be polled in a loop. The unattended pattern:

• Delegation is explicit. "Run it and check the results" delegates execution. If the researcher has not delegated, offer it before launching: "this needs ~Xh; I can run it in the background and resume automatically with the later tool — delegate to me?" State that each turn will end with a later schedule until the batch lands.
• Launch in the background per project conventions (nohup/tmux/launcher), then call the later tool: estimate the duration, add ~10% margin (5h run → schedule 5.5h), and write the prompt as a complete instruction to your future self — what to check, what to decide, what to run next.
• End the turn after scheduling — the later call is the LAST action of the turn: no idling, no spinning, no token-burning polls while waiting. The scheduled prompt arrives as a user message and resumes the session with no human present. Chain wake-ups across runs: each wake-up checks, decides, launches the next run, schedules the next later.
• A timer is an estimate, not a completion signal. On wake-up, verify the run actually finished and succeeded before recording metrics. If it is still running, re-estimate the remainder from observed progress (log/epoch counters, current throughput — contention can stretch a planned 5h run to 7h), never from the original plan, and schedule another later for that remainder plus margin. Repeat until the run actually finishes; there is no wake-up budget.
• If the host has no later tool, degrade: tell the researcher the expected duration and when to return, or follow the project's own waiting convention.
• A background-task completion wake is not a substitute here: training and batch jobs need the custom next-step prompt and duration margin that later carries. Use background tasks for completion-driven shell or read-only investigation, not for these scheduled experiment chains.

6. Land the metrics where figures can use them

• Every run lands in the shared results table with: id, config, split, surface, metrics, status, evidence path. Include every epoch/point recorded, not only the chosen one; retain sample/config identities, pairing and the independent unit needed for analysis. Use the existing schema/notes rather than a second registry.
• Interpret support, valid negatives and inconclusive outcomes with evidence interpretation. Bind conclusions to actual input/code/metric versions; return execution/QC defects to repair, weak controls to design, and unresolved mechanisms to discussion instead of automatically extending the batch. Data figures can use research-evidence over this same results table.
• Before finishing, check that results support the learning question or agreed comparison with consistent axes/metrics. Add measurements when response trends, Pareto/ablation coverage or formal claims need them; required scientific curve coverage is legitimate. Avoid additions whose only purpose is visual symmetry.
• Batch report is mandatory — including for partial, blocked, or aborted work. Write <batch-or-stage>.report.md beside the campaign plan (default docs/experiments/<slug>.stage<N>.report.md; a standalone branch goes in the project's experiment area) with: 完成情况总表 (per planned run/cell: 完成 / 部分 / 搁置 / 未动 + evidence path), the exact configs and commands that ran, what was not run and why, metric-contract deviations, failures and how they were handled (eviction / OOM / NaN), and what the campaign should reconsider. Never fake a result row, never silently skip a planned run.
• When this batch changes route status or key understanding, use save-mem to update the existing route record/profile with the report as evidence; batch completion is not proof of the hypothesis or completion of parallel routes. Report concisely in chat. The planning session is not woken by this batch — the researcher brings the report back, so the file is what gets reviewed.