validating-merges

Tracking is a separate completion gate

Before selecting a batch, read docs/reference/validator-tracking.md and run its preflight. Discover the actual tools exposed by this client; do not infer a private writer from an ordinary Codex connection, silently activate another profile, or register an external client as a Genesis session.

For EACH row: persist valid measurement evidence immediately, classify every concern, file/reuse its safely redacted public issue (private only for the concrete disclosure harm in CLAUDE.md), read the task back, and add structured tracking. Free-text findings[].disposition, chat, a run log, or a verification row does not prove routing. Receipts carry the exact live task excerpt plus your rationale for how it covers this concern. Verify meaning as well as existence; quote matching does not decide semantic relevance for you. Each private receipt states the actual harm and why redaction/public-task/private-evidence cannot avoid it.

Both writers retain the measurement verdict when routing is pending and leave the row OPEN. CLI exit 4 means measurement recorded or previewed, tracking pending; re-read before retrying. MCP exposes recorded, tracking_complete, tracking and would_close separately. A preview stores nothing. Never call the batch handled until the selected-row checkpoint reports complete tracking.

Checkpoint after each row, before moving batches, and before a completion report. Honor a user pause immediately: stop selecting new work, preserve the current measurement, run a selected-row checkpoint --pause, and report outstanding targets/receipts. Incomplete routing is not permission to refuse a pause. A missing private writer affects genuinely private items only; continue safe public filing. The historical audit reads OPEN and CLOSED rows, including gap-carrying legacy records. It reports debt for manual classification and never rewrites closed evidence. Search existing issues first; historical missing structured links are not proof that no issue exists.

Running under Codex

This section applies to the separately qualified interactive Codex validator. It qualifies the shell-ledger and deploy directions below; the shared evidence, risk, control and finding-routing rules still apply. Read .agents/skills/genesis-external-client/SKILL.md and docs/reference/codex-validator-boundary.md before using the tools. The ordinary Codex external profile retains its health/recall floor and cannot write memories or verification records merely because this skill is discoverable.

The dedicated launcher's MCP server keys are genesis_validator_health and genesis_validator_memory. Codex tool names use mcp__<server-key>__<tool-name>:

pr_verification_close takes evidence_json and an independent pr, with optional park, note, verdict expectation and dry_run. Verdicts are derived from the same strict document policy described below. Use that tool, never the shell pr_verification.py close writer. A server-forced preview writes no obligation or attempt note and discharges nothing. Actual recording stays locked until separately authorized calibration and operator opt-in; no request can clear it. After an uncertain real response, re-read stored evidence/count before retrying.

This conversation is not a Genesis session. Do not register its transcript, create a charter, use the session ledger or pass its identifiers to session tools. Append progress/evidence/uncertainty to ~/.genesis/output/validator-runs/<date>.md. Private follow-up creation uses a stable request_key and strategy=user_input_needed, without schedule, pin or session provenance. Updates cover eligible private records across clients; they cannot promote or dispatch them. Memory writes retain the existing collection/supersedes interface and partial-failure limits. Do not write memory to manufacture a verification result when that memory store is under test.

One evidence claim per declared claim applies to Codex exactly as to any validator (see the claims[] rule under Evidence). MEASURED in the Codex validator calibration, 2026-11-05: 14 of 14 documents carried one claim against 3 to 14 in the blind ground truth, and 6 of them derived pass-mechanical on rows whose ground truth had gaps.

Never deploy, change a service, push, merge or fix code. The deploy recipes and Host-Deploy Gate below describe the Claude operator's responsibilities. For Codex, a row waiting on deployment is NOT-YET-DONE: leave it open and note the required deploy in a safely described public task and checkpoint (private only for unavoidable disclosure harm). Do not invent deployment evidence or mark cannot-verify just because this client cannot deploy.

Public issues and issue/PR timeline comments go through scripts/file_tracker_issue.py; comments use --comment-on <number> with a --body-file. Use body files and the canonical shell restrictions, never raw gh or API writes. Public-by-default applies to security-related and install-local findings too. Redact sensitive supporting evidence; keep the whole task private only under CLAUDE.md's concrete disclosure-harm test. A successful scan is not permission to publish private content. Validators do not request development reviews or trigger paid agents; the filer is not a review-budget gate.

A native guard denial stops that action, not the run. Do not retry it through another command spelling, subprocess, tool or client. Record the claim it served as a not_yet_done gap whose command is the denied step and whose reason is "blocked by the validator guard: <one line>", so the row stays open for a client that can run it; use NOT_VERIFIABLE_HERE only where no client on this install could. List the denial (command, reason) in the run log and carry on with the other claims and rows. Put any question for the owner in plain text in your report: the validator profile has no question tool, and its guard denied the native one in the 2026-11-06 calibration. Do not write stores under test or run migrations on live data. Discovery of this skill does not qualify or activate the launcher, and it does not authorize PR6.

Write commands so that the most common false denials never fire; once a command is denied, the paragraph above governs. The credentials guard expands a * it cannot prove is literal, quoted or not, so a raw sqlite3 or python -c query with count(*) run from the checkout is read as touching the credentials file. Prefer python -m genesis db query "<sql>", which is exempt; otherwise write count(1), name columns and use explicit paths. A command whose prose names the credentials file trips the same guard. Separately, the box-wide pytest lock is a refusal, not a guard denial: another session holds it. Wait on it with python3 scripts/pytest_lock_wait.py (run in the background if you have other rows to work), and never stop the run or kill the holder because of it.

Load Gate

Three session types touch a PR and they are not phases of one life:

If the job is "make the change", you are a build session. If it is "get the PRs merged", you are a closing session. If it is "did the merged thing actually work", you are here.

What you own

pr_verifications (issue #1718 half B). The repo-pulse worker opens one row per merged PR; a documentation-only diff is auto-closed by a deterministic path rule; everything else stays OPEN until you record evidence. Read the backlog with

python3 scripts/repo_pulse_worker.py --verification-backlog

and record what you concluded with

python3 scripts/pr_verification.py print-schema > ~/.genesis/output/prv-<N>.json
#   ... fill it in ...
python3 scripts/pr_verification.py close --pr <N> \
    --evidence-file ~/.genesis/output/prv-<N>.json [--note "..."] [--dry-run] \
    [--verdict <what you expect>] [--park]

Read back what was decided, and on what evidence, with

python3 scripts/repo_pulse_worker.py --verification-log [--pr <N>]

A closed row can still owe an event-gated recheck. The backlog's Rechecks due section lists dates on or before today (UTC) and every named event for the validator to judge. It prints the stored --index and a recording command bound to the database it read. After running the named read-only measurement, write {"repo": "owner/name", "result": "done" | "failed", "note": "what it showed"} to a result file outside the repository and record it with:

python3 scripts/pr_verification.py recheck --pr <N> --index <I> \
    --result-file ~/.genesis/output/prv-<N>-recheck.json [--dry-run] [--db-path <DB>]

done finishes the recheck; failed appends the result and leaves it due. Bring a failed measurement to the owner as fail-intent; recording files nothing and never reopens the row. raced means nothing was appended by this attempt: re-read before retrying. A malformed stored entry/history is listed once for inspection with --verification-log, retained, and refused as an append target.

Two mechanics worth knowing before your first run. --dry-run resolves the target and renders the record without writing — use it, because a closed row cannot be amended through this tool. And genesis_db_path() is repo-root relative, so running the closer from a linked worktree points it at that worktree's non-existent data/genesis.db: run it from the main checkout or pass --db-path. The message says so, but the diagnosis is not the one you would guess.

The backlog is PER INSTALL — each box's worker fills its own ledger against its own deploy state, so two installs hold different open sets and neither one's count says anything about the other.

The five verdicts

Never pass/fail. Every PR gets exactly one of these (standing ruling) — and you do not choose it: the tool DERIVES it from your evidence document, so the verdict always says what the evidence says:

A gap is anything you declared in gaps (typed by kind — see Evidence below), plus what the document implies: no negative control, any claim at NOT_VERIFIABLE_HERE, any claim at INFERRED, and a named SUSPECTED FAILURE (INFERRED) for each claim that failed only at INFERRED. A failure is gated by tier exactly as a pass is: only an ESTABLISHED one goes to the user as fail-intent. They are named in the closed row's reason, so a row that did not close mechanical says why — and a suspected failure, which never closes a row, is written at the head of the open row's note, so the backlog shows it rather than your note alone.

Your judgment enters in exactly one place. --park turns a document that derives pass-with-measured-gaps into cannot-verify: something was established, but nothing MATERIAL was — the measured claim is trivial next to what could not be reached. It is refused on anything else, and above all on a failure: a failure is never parked. --verdict is optional and worth passing: it states what you EXPECT, and if the document derives something else the tool refuses with "document derives X" — which catches a document that does not say what you think.

A FAIL-intent never triggers an automatic rollback (standing ruling). It goes to the user as a conversation. The row stays open because the obligation is not discharged until the fix lands — and you are not the one who fixes it.

A non-closing outcome requires --note, and the note is the whole point of the row staying open: it is what stops the next validator re-deriving why this could not be finished. It surfaces inline in the backlog as ATTEMPTED 2x cannot-verify — <note>, so a parked row announces itself. Parked rows also sort LAST in the backlog, behind never-attempted work (a fail-intent row sorts FIRST: it is a finding, and no cap may hide it) — parked rows are typically the oldest, and an oldest-first reader would otherwise let them starve the work you can actually do.

A closing outcome refuses --note rather than silently discarding it — a closed row's prose belongs in the evidence document's findings, which does not change the verdict. (gaps would: every entry is a gap.) not-yet-done is the one open outcome that needs no --note: its NOT YET DONE gaps already say what is left and the command that does it, and they lead the row's note; a note you add goes after them. --park is refused on it — undone work is not a reason it cannot be done.

FILE AN ISSUE, or RECORD A CANNOT-VERIFY?

These feel like one question and are two: what did the verification find (the verdict) versus is there work anyone could pick up (the route). Only the second decides filing.

The discriminator is a question, not a label — name the ACTOR and the RESOURCE.

• Can you name a real actor and a real resource? "a box without the graph client installed", "another install", "after the next deploy runs", "a clean venv". → Reachable. This is ordinary work: FILE it. A repo-owned defect is a GitHub issue, including state local to one box. Use a private follow-up only when useful public wording necessarily causes concrete disclosure harm. Scrub anything install-identifying first — an issue carries technical detail only.
• Does naming the actor require inventing a hypothetical? Hardware nobody has and nobody has reason to get; an event that may never occur. → Genuinely unverifiable. Do NOT file. An unactionable issue burns whoever picks it up, and the repo already routes "cannot be picked up" to tabled rather than to issues. Record it on the row — mark the claim verdict unverified, tier NOT_VERIFIABLE_HERE, and pass --note "<the precondition>"; a document with nothing else established derives cannot-verify, and one with only trivial measurements can be --parked. That leaves the obligation open and annotated, so the next validator on any install sees ATTEMPTED … cannot-verify — <your note> in the backlog and re-checks it once the precondition is reachable, instead of re-deriving your conclusion from scratch.

The abuse this marking invites, and the test that stops it. "I did not get to it" is never CANNOT-VERIFY. Ask: could I have done this with the access and the time I actually had? If yes, it is NOT-YET-DONE — do it, or record each step as a not_yet_done gap with its command, which derives not-yet-done: the row stays open with the command at the head of its note, where the next validator starts. Measured instance, the first validator session's own first PR: a sub-claim was labelled unverifiable when it was reachable on that very box with a throwaway venv; it had been skipped for cost. If CANNOT-VERIFY becomes the routine escape, closed degrades to "we looked at it" and the word stops meaning anything. Keep it RARE.

The same test applies to a gap. NOT-YET-DONE is never a gap that a closing verdict can carry: do not write a reachable-but-skipped step (a test file you did not run, a probe skipped for cost) as a method gap and close. Record it as a not_yet_done gap, which keeps the row open, or run it. The tool trusts the kind you choose, so the discipline is still yours. Instance, one backlog run: a keyword scan of the 158 gaps on its 85 gap-carrying closed rows flagged 20 as test files or probes that were skipped, not hand-checked one by one.

A closed row keeps event-gated rechecks. "The code is right, but the event it handles has not happened yet" (the first revert, an engine old enough to prune, a skipped backup directory) is a real gap that production will eventually test. Apply the actor-and-resource test above first. An event that can be named and will happen in ordinary running ("after the next deploy", "on 11-03", "on the first REVERT_CODE") is reachable later. Record a typed event_gated gap naming its trigger and one read-only recheck command. Closing stores it on the same row, and Rechecks due surfaces it for later measurement without reopening the row. A repo defect still gets an issue. An event that may never occur is the cannot-verify case above and is not filed.

Security-related work is public by default too. Withhold the whole item only when a useful redacted public description necessarily enables concrete exploit, cost or harm, or reveals protected information; record that specific reason and why public wording with private evidence cannot avoid it.

Before you judge anything: verify on the right code

Merged is not deployed. Confirm the box you are testing has actually pulled and deployed the merge — from the running version, the installed artifact, or the logs — before forming any verdict. A validator running against a tree that predates the merge will measure a defect the branch already fixed and report it with complete confidence.

And do NOT establish deployment by ancestry. MEASURED: gh pr view --json mergeCommit on a STACKED PR names a commit reachable only from that PR's own base branch, because baseRefName was a feature branch rather than the default; the change reached the default branch inside the squash of its parent, under a different PR number. git merge-base --is-ancestor <mergeCommit> HEAD therefore returns a false "not deployed" while the code is live and behaviourally correct. Establish it by content (the change is present in the deployed artifact) or by BEHAVIOUR (the live thing was observed doing it), and check baseRefName before reading anything about DEPLOYMENT into mergeCommit. The document's merge_commit is still that mergeCommit (close checks it); the parent squash you actually exercised goes in deploy.detail. Ancestry is a fast positive path and never a negative verdict.

Report which copy of the code you actually exercised. A verdict that does not say is not a verdict.

The E2E: line is a LEAD, not a limit

A PR body may declare its post-merge verification. That declaration is the author's hint from when they knew most — read it first. It is not the boundary of the job: assume an E2E is owed and hunt for the real effect. A bare E2E: none on a code PR is a reason to look harder, not a release.

Two things follow, both measured on the first validator session's pilots:

• Roughly half the backlog carries no declaration at all, so deriving the verification from the diff is the common mode, not the exception.
• The undeclared surfaces are where the findings are. A PR that fixes a RENDERER does not repair an already-rendered artifact; a PR that widens an uninstall path cannot be verified without running it. Neither was declared.

Triage by risk, and sample — do not chase

The backlog outruns any per-PR cadence. Order by consequence: gates and enforcement hooks, then runtime and migrations, then everything else. Deploy main on a cadence and validate against that, rather than deploying per PR.

Age is an ASSET, not a cost — this inverts the obvious intuition and it is measured. An old row whose behaviour is EVENT-GATED has had weeks for production to run the experiment: one three-week-old PR yielded 19 real production events, giving n=19 for both of its claims from a single query — the strongest evidence of three pilots and the cheapest to obtain. A one-day-old scheduled job gave n=37 runs. A hook PR had no production samples at all and needed synthetic probes. So oldest merge first — the backlog's order within each group, after failed rows lead and never-attempted rows are put ahead of parked ones — is right for a second reason: old rows are not a wall, they are the tier where the experiment already ran.

Negative controls are not optional here

MEASURED: negative controls caught two would-be FALSE FINDINGS in three pilot PRs — a recurring journal warning that turned out to be environmental (every sibling unit with the same hardening directive emitted it; the one without them emitted none), and an apparently-dead error branch that was simply reached by a different syntax than the probe used.

For every claim you verify, run the arm that must come out the OTHER way. A guard that blocks the case under test would also block everything; only the control tells those apart. A verification with no control is a story.

The PR's own tests are the author's evidence, not yours. Re-running them confirms what the author already checked, and it cannot find a case they did not think of. Every claim needs at least one probe or code-path read that the shipped tests do not already perform: the other branch of a two-sided rule, the other call sites of a changed message, the path the test fixture skips. Before deriving a verdict, also search the issues that cite the PR, open and closed (gh issue list --state all --search "#<number>", and its title words); a later report of the same behaviour is evidence, and a fail it names must be measured or refuted, never ignored. MEASURED in the 2026-11-06 Codex calibration: both false passes (2 of 14 rows) came from this. One PR's refusal rule was tested only on its release half, and the defect was in the serve half; another's post-save warning was tested on one stop path, and a second path lacked it. Open issues named both defects, and neither was read.

Hard limits

• Never merge. That is the closing session's job and the user's approval.
• Never write to the stores UNDER TEST, and never run a migration against a live database. Read live state freely; do not mutate it to force a code path. The one store you DO write is your own obligation ledger — that is what discharging a row means, and it is not an exception to this rule but the subject of a different one.

• An E2E is a TEST, not an operation. Apart from your own ledger (above), it does not write real data: no live database rows, no real config, no messages sent, nothing pushed or delivered to another install. Use a scratch copy, a scratch home (GENESIS_HOME, HOME), a throwaway clone, a mock payload, or production evidence that already exists (logs, journals, rows the system wrote on its own). A PR that "cannot be verified without running it" (an uninstall path, a restore) runs against a scratch install, not the live one. A write to real data is acceptable only when nothing else can establish the claim, and then:

  • Deliberate it first. Name exactly what will be written, where, and why no test double reaches the claim, and get the user's yes for that write. A PR body's E2E: line that says "do X live" (Devin's included) is the author's lead, not that approval.
  • No user present, no write. A dispatched validator cannot get that yes, so it records the unreached part as a scope limit instead.
  • Clean up after. Remove what you wrote, or say why it must stay, and record which it was in the evidence document.
  • Otherwise, record the unreached part as a scope limit. A gap that is named beats a side effect nobody tracked.

  Deploying merged main (the Host-Deploy Gate below) is not an E2E write. It is the precondition for testing at all.

• Probes can write without looking like writes. Some hooks write the live database as a side effect of matching. For example, the procedure advisor bumps procedural_memory.surfaced_count when it matches (unless the database is fenced). Before you pipe a payload to a hook, check whether it writes, and point every path it writes through at a scratch location: GENESIS_DB_PATH for the counter, and HOME or GENESIS_SESSIONS_DIR for the advisor's fire log (GENESIS_HOME alone does not move that log). Read the hook for the variables it actually uses.
• A delegated check gets your own check before it closes a row. When a subagent or another session wrote the evidence document, re-derive at least one load-bearing claim yourself from ground truth before the real close. A closed row is permanent record, and one agent's report is not enough for that. If your re-check fails or errors, the row does not close until the check runs clean.
• Do not collide with a deploy. Never restart services or run an E2E while another session might be deploying. Where the repo provides a deploy lock wrapper, run under it; where it does not yet, check for in-flight deploys before touching a service.
• Separate INSTALL findings from REPO findings. A failure may be specific to one box's configuration rather than a defect in the repo. Say which before filing anything, and file a repo issue only for the repo half.
• Triage a question before asking it. A validating session uses the closing session's four buckets: answer it from evidence, follow a standing rule, decide and log a low-stakes reversible call for batch review, or ask the owner a genuine decision (closing-session, "Working the queue", step 5). The triage never moves what this skill already gives the owner: a fail-intent goes to them as a conversation (the verdicts table above), and a native guard denial means stop and ask (under Codex: stop that action and carry on, as "Running under Codex" says).

Evidence: what the record has to carry

The evidence document is validated by a strict model (genesis.session_awareness.pr_evidence): every prose field must be a non-blank string, an unknown key is refused rather than dropped, and the refusal names the exact field path. Its shape was derived from pilot validations rather than designed, because each pilot broke a field the obvious version would have had:

• repo + pr — the ledger row's KEY, both halves. The document names the row it verifies, and the row must be OPEN for exactly that pair; a closed row cannot be amended, so a mis-bind would be permanent. --pr must ALSO equal the document's pr — a deliberate second copy, because a typo inside the document would otherwise close the wrong row with nothing to catch it.
• deploy.method + established + detail — because ancestry lies (above). HOW deployment was checked travels with the row, and established is the structured outcome the verdict gates on: false derives cannot-verify whatever the claims say, since nothing measured on a tree that does not carry the merge says anything about the merge. A failure measured there is kept as a named gap, never taken to the user as fail-intent.
• claims[] with a per-claim TIER (MEASURED / INFERRED / READ / NOT_VERIFIABLE_HERE) — because a declaration DECOMPOSES and the row is binary. One PR's two declared claims resolved into four sub-claims across three tiers; a single verdict would have been true of some of it and false of the rest. An INFERRED claim is one you composed from two measured facts — label it, never promote it. Each claim's verdict is pass, fail or unverified, and unverified goes with NOT_VERIFIABLE_HERE and only with it: the model refuses either without the other, so a claim nobody could check is never written down as a pass or as a failure. List every claim the PR makes, one entry each: every behaviour its body promises, its E2E: line, and each behaviour the merged diff changes (behaviours, not files). Each gets its own control. Never drop one and never fold it into a neighbour, because pass-mechanical asserts that nothing was left out. A claim this install cannot reach is NOT_VERIFIABLE_HERE. A claim you could have run but did not is a not_yet_done gap with its command, which keeps the row open.
• e2e[] — explicit coverage of the PR's declared E2E work. Each entry has step (the exact declaration text without the E2E: prefix) and exactly one zero-based claim or gap index into claims[] or gaps[], for example: {"step": "restart and measure the timer", "claim": 0}. A gap mapping uses {"step": "check the owner device", "gap": 0} and the gap keeps its existing kind, verdict and recheck behavior. Never omit unreachable steps. CLI and MCP close read the PR body alongside its merge SHA and refuse a closing result, including preview, if any visible plan declaration lacks a mapping or a mapping names text no longer declared. Every plan line is one opaque step: mapping a compound line asserts coverage of its whole text. Decompose its behaviors into claims and use multiple mappings when needed. Code checks the structure; it does not judge whether your measurement covers the prose. E2E: none — <reason> needs no mapping. The field defaults to [] so older evidence still parses; it cannot close over a declared plan without coverage. An incomplete open attempt can still be recorded without a complete mapping.
• controls[] — a reader cannot distinguish a non-vacuous verification from a vacuous one without them. A document with none can still close, but never as pass-mechanical: "no negative control" becomes a named gap.
• merge_commit — the PR's OWN merge commit, 7+ hex characters. close checks it against GitHub (gh pr view --json mergeCommit,body) and refuses a mismatch with exit 3, on --dry-run too; if gh cannot resolve it, the close is refused (exit 2) rather than waved through. MEASURED 2026-10-27: 5 of 15 documents in one group named main HEAD or the deployed commit instead. On a stacked PR this is still GitHub's mergeCommit (on the parent branch); record the parent squash you actually exercised in deploy.detail.
• gaps[], each with a kind — what this document leaves unestablished, and the kind decides what it does. MEASURED 2026-10-27 over one evening's 99 closes: of 158 untyped gaps, 35 were reachable work that had not been done, 44 waited on an event, 57 were method limits and 22 belonged to the owner — and the untyped list closed every one of them alike.
  • method (detail) and owner_only (detail) — close as pass-with-measured-gaps with the gap named.
  • event_gated (detail, trigger: a real YYYY-MM-DD or a named event, recheck: one read-only command or SQL that re-measures it) — closes with the gap named and stores a recheck on the closed row. The backlog lists it under Rechecks due until a result marks it done. The close is refused before the rechecks column exists, including previews, so the measurement cannot disappear during an upgrade.
  • not_yet_done (detail, command) — keeps the row OPEN as not-yet-done. The old untyped scope_limits is refused by name, never mapped across: choosing each entry's kind is the judgment the untyped list let a close skip.
• findings[], each with a DISPOSITION — an undispositioned finding is a drop wearing a record, and the validator refuses one.

Nothing closes a row unless something was established — at least one claim passing at MEASURED or READ. INFERRED does not count: an inferred claim never enters permanent record in the grammar of a fact, and a discharged obligation is permanent record. A document that establishes nothing derives cannot-verify: the row stays open, your note travels with it, and the next validator inherits what you found.

Why the verdict is derived rather than chosen, because the obvious version was built first and failed instructively. It took the verdict from the validator and policed it with a growing list of refusal rules, and twice, one fix apart, the rules contradicted each other: a document whose claims were all INFERRED, and later one with a single measured claim and no control, were each refused under ALL FOUR verdicts, with the refusals pointing at each other. The only way out was to relabel a tier — the tool paying you to falsify the one field it exists to protect. A derived verdict cannot do that: every valid document has exactly one, by construction. If you ever find a document the tool will not accept at all, that is a defect in the tool — file it; do not edit the evidence to fit.

Devin's install-scoped PRs

Devin's install and hybrid PRs carry E2E: lines written for Genesis to run. The validator is their natural consumer — those are the rows where the declaration is most likely to be both present and actionable.

Load genesis-development (the shared core) first if this session has not: it holds the review-chain, evidence and private-data rules every session uses.

Deploys: detach update.sh from the session

DEPLOYS ARE THE EXCEPTION (to the core's run_in_background rule), and this is the one to get right. scripts/update.sh must be DETACHED FROM THE SESSION ENTIRELY — not foregrounded, and not backgrounded. (bootstrap.sh and host-setup.sh are long too but are NOT in this group; see the warning below before reaching for either.)

• Foreground is impossible: MEASURED 2026-09-16, a bare update.sh ran 1022s, 1.7× the 600000ms hard ceiling.
• run_in_background is NOT the fix either, despite having no 120s ceiling: it is tied to the SESSION's lifetime, and deploys launched that way were killed mid-run on 2026-07-22 (leaving genesis-server DOWN during bootstrap) and 2026-09-16 (during the pre-update backup). The precise kill mechanism is UNRESOLVED — three candidates are eliminated in CC memory deploy_detach_not_background, and the successor hypothesis there is hedged, not established. Detachment removes the coupling to this session, which is what every remaining candidate runs through. It is not immunity in general: an OOM kill reaches a systemd unit just as readily.

XDG_RUNTIME_DIR="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}" \
DBUS_SESSION_BUS_ADDRESS="${DBUS_SESSION_BUS_ADDRESS:-unix:path=/run/user/$(id -u)/bus}" \
systemd-run --user --collect --unit genesis-deploy-manual \
  --working-directory=$HOME/genesis \
  --setenv=PATH="$PATH" --setenv=SSH_AUTH_SOCK --setenv=GENESIS_SYNC_PRIVATE_PATTERNS \
  /bin/bash -c 'mkdir -p ~/tmp; exec ./scripts/update.sh > ~/tmp/deploy-$(date +%Y%m%d-%H%M).log 2>&1'

substituting the checkout root you actually ran from for $HOME/genesis if it differs — scripts/install.sh sets REPO_DIR from its own location, so noncanonical checkouts are supported, and update.sh derives its root the same way, so the canonical path is an example, not a constant. (Keep the command itself pasteable — a literal <placeholder> in the value is a shell syntax error.)

Two preconditions, checked before detaching, not after:

• Linger. A --user unit dies with the user manager when your last login session ends unless linger is enabled. loginctl show-user $(id -u) -p Linger must print Linger=yes; if it does not, run loginctl enable-linger $(id -u) first (one-time, needs sudo). Without it the "detached" deploy survives the CC session only as long as you stay logged in — which is the same coupling, one level up.
• Noninteractive sudo. update.sh runs bootstrap.sh on every run that applies a change (its "Run bootstrap" step), and bootstrap calls sudo. The no-sudo claim covers only update.sh's own text, not its transitive calls. If sudo prompts anywhere on this install, the detached deploy aborts mid-bootstrap. Require NOPASSWD-style sudo: sudo's per-terminal timestamps mean refreshing the credential in your terminal does not authorize the no-TTY calls inside the unit, and a warm cache expires mid-deploy anyway. Check it in the same no-TTY service context the deploy will get, with the bus variables set: systemd-run --user --wait --pipe --quiet --collect -- sudo -n true must exit 0 (MEASURED 2026-10-01: --scope --wait is refused outright, "--wait may not be combined with --scope"). If it fails, run the deploy in a terminal instead.

mkdir -p ~/tmp is part of the command, not tidiness. ~/tmp is not guaranteed — install.sh and bootstrap.sh create it only when /tmp is small — and the shell opens the redirect before exec'ing update.sh, so a missing directory kills the unit instantly, before the deploy's own mkdir -p "$HOME/tmp" near the top of update.sh can run. With --collect that leaves nothing behind to diagnose, which is the same invisible failure the verify step below exists for.

The bus variables come first, and they are not decoration. A CC session often has no XDG_RUNTIME_DIR, and without it the systemd-run --user CLIENT cannot reach the user manager: MEASURED 2026-09-16, a scrubbed environment gives Failed to connect to bus: No medium found and nothing starts at all. --setenv configures the prospective unit and cannot help the client connect, so it is no substitute. update.sh exports exactly these two near its top for the same reason, and cc/invoker.py records the same failure as measured on a live install. This one is easy to get wrong by testing in a session that happens to have the variables — which is how it reached review here.

A transient service inherits the user manager's environment, not your shell's. Only what you name with --setenv crosses over, so a deploy depending on an exported variable silently changes behaviour: SSH_AUTH_SOCK (update.sh's git fetch of a private SSH remote fails without the agent) and GENESIS_SYNC_PRIVATE_PATTERNS (bootstrap.sh branches on it) are forwarded above. Naming an unset variable is harmless — measured, the unit still starts.

Never apply this to bootstrap.sh or host-setup.sh. Both need a channel a detached service does not have. bootstrap.sh runs under set -euo pipefail and calls bare sudo (the journald and sshd drop-in steps' sudo mkdir), so wherever sudo prompts it aborts partway through configuring the machine — and "sudo is passwordless here" is a fact about one box, not about the recipe. host-setup.sh is worse: interactive, run on the bare host VM, and its "Delete and recreate? [Y/n]" prompt treats EOF as the default Y, so detaching it stops and renames the existing container. update.sh is the only script in this family with no sudo calls of its own — it does invoke bootstrap.sh (its "Run bootstrap" step), which does call sudo, hence the noninteractive-sudo precondition above — and it remains the only one covered. Run the other two in a real terminal; the advisory no longer fires on them rather than offer a recipe that breaks them.

Wrap the command you actually ran — update.sh and bootstrap.sh are not interchangeable and take their own arguments. If you build that wrapper programmatically, quote the whole inner script as one unit (shlex.quote) rather than interpolating an already-quoted command into bash -c '…': MEASURED, the naive form turns --msg 'a b' into --msg a and executes a $(…) the user quoted as data.

--setenv=PATH is load-bearing: a --user unit otherwise gets systemd's default PATH, which omits ~/.local/bin, so bootstrap takes its uv not found — installing… branch rather than the branch an interactive run takes. The shipped code sidesteps this by passing the whole environment (updates.py::_apply_direct uses env=os.environ.copy()).

Two things that look like detachment and are not. systemd-run --user --scope keeps the CALLER'S session id (MEASURED) — cgroup-isolated, still session-held; _apply_direct gets away with it only because its parent is genesis-server rather than a CC session. And setsid is not detachment: it runs the program in a new session but does not background it or stop the caller waiting on it — MEASURED, timeout 1 setsid bash -c 'sleep 4; echo > f' exits 124 having written no file. What detaches is the default transient SERVICE mode (no --scope): MEASURED 2026-10-01 on systemd 255, a service child's parent is the user manager and it runs in a new process session, while a --scope child's parent is the calling shell. --unit does not detach anything — an auto-named service lands in the same place — it only gives the unit a fixed name for the verify step below.

Then VERIFY it took — with the same two bus variables set (a scrubbed session otherwise gets a bus error, which says nothing about the launch), systemctl --user is-active genesis-deploy-manual must print active. systemd-run does NOT inherit the tool's cwd, so a bare relative path resolves under $HOME, bash exits instantly, --collect reaps the unit, and a failed launch is indistinguishable from a successful one. (It also reports inactive once the deploy has FINISHED, so check promptly and read the log for the outcome.) genesis-server going inactive mid-run is EXPECTED. When the server is up, POST /api/genesis/updates/apply with {"supervised": false} does the same job and passes the environment for you.

A mid-run kill is never a no-op, but what it does depends on the signal and on the phase. Three windows, in order. From where trap '_on_signal INT' INT arms the rollback trap — just BEFORE _write_state "merging", not after the merge — an interrupt runs _on_signal → _do_rollback. Between trap '_on_signal_prestop …' and there, _on_signal_prestop restarts the services it stopped and explicitly does not roll back. Earlier still — during the pre-update backup (the backup.sh call) — no INT/TERM trap is installed yet, so a SIGTERM there runs no handler at all. That last point matters for diagnosis: "no trap ran" does NOT identify a kill as SIGKILL, because the untrapped window produces the same evidence. Note too that _do_rollback disarms INT/TERM as it starts, so a second signal lands mid-rollback with default disposition, and the rollback has failure exits that report ROLLBACK INCOMPLETE. Either way: verify state (server active, no mid-rebase, pin, both CC versions) before re-running; it is idempotent. (A non-blocking PreToolUse advisory hook, scripts/hooks/deploy_timeout_advisory.py, nudges toward this whenever a command can run update.sh, and stays quiet on reads of it. It deliberately does not check whether you already detached: that needs systemd-run's option grammar, an open set, so a detached launch still gets the note.)

Why this paragraph is worded so defensively: PR #1221 shipped both this policy and that hook on 2026-07-22 prescribing run_in_background, from a CC memory written two days earlier. An incident that same day refuted it. The correction landed only in a second CC memory on 07-23, both public surfaces kept the original, and a third deploy was killed by this advice 56 days later. A remedy is only as good as the case it was actually executed against.

An earlier draft of this paragraph also claimed #1221 shipped a second, already-stale assertion about signal handling. That was false, and worth recording as the more useful lesson: it was inferred from the CC memory #1221 was written from, rather than read out of the commit, which shipped the correct claim. A note is evidence about what someone believed when they wrote it — never about what the code says now, and never about what some other artifact contains. Open the commit.

Host-Deploy Gate (merged ≠ deployed)

A merged PR that touches host-deployed paths is NOT done at merge. The guardian and the host VM only pick up changes when scripts/update.sh runs — merging and walking away leaves the host running stale code indefinitely (observed live: a host guardian sat 3 PRs behind for a week because every session assumed deploy "happens somehow").

Trigger paths (match = this gate applies): src/genesis/guardian/, scripts/guardian-gateway.sh, scripts/install_guardian.sh, scripts/host-setup.sh, scripts/update.sh, scripts/lib/cc_version.sh.

After merging such a PR, in the same session:

1. Run scripts/update.sh from ~/genesis, DETACHED from the session with the systemd-run --user recipe under "DEPLOYS ARE THE EXCEPTION" above, never in the foreground or via run_in_background (it redeploys the guardian when guardian-relevant paths changed and heals host/container CC + Node pin drift — including on a no-delta run).
2. Verify the deploy landed: gateway version op reports the expected deployed_commit / CC version; guardian tick healthy in its journal.
3. State the deploy + verification result explicitly in the wrap-up. If the deploy cannot happen this session, record it — never leave deploy as an implicit assumption. A purely LOCAL blocker (host unreachable tonight) is a follow_up_create row. A blocker exposing a REPO-level gap (a missing reconcile mechanism, no self-heal path) needs BOTH: an issue for the mechanism fix, AND a local row for the fact that THIS install is still undeployed — another contributor can close the issue without ever touching this host, which is exactly the stale-host failure above.

The reverse direction is equally binding: host VMs are deploy targets, never edit-in-place dev environments. An emergency hand-edit on a host gets a same-day PR that lands the same change at source — a host divergence that outlives its incident is a bug.