Updating everything

There is no single command, and that is the whole problem. "Update" names six independent surfaces on two machines from three repositories, each with its own version line. Every one of them can be stale while the others are current, and none of them complain.

The failure this skill exists to prevent, from 2026-08-15: mecha 0.1.4 was tagged, pushed, and published to crates.io, and the services were restarted — and mecha --version still said 0.1.3 afterwards, because nothing had run cargo install. Four surfaces were stale at once that day: the installed binaries, mecha-triggers (left out of the restart), the benchmark's musl binary (eight days old, so a benchmark would have measured old code and reported it as new), and the factory client.

The one rule

Verify the running thing, never the repo.

A tag proves a commit exists. A green CI run proves it compiles. git log proves someone wrote it. None of them prove that the bytes executing on this machine contain it. Ask the process what it is, not the source tree: mecha --version, ls -l ~/.cargo/bin/, /proc/<pid>/exe.

Three specific confusions worth naming, because each one has actually happened:

• A release is not an install. Tagging and publishing changes crates.io, not ~/.cargo/bin.
• A restart is not a reinstall. systemctl restart re-executes the same file on disk. Restarting before installing accomplishes nothing.
• A debug build is not the one that runs. Both mecha's MCP config and the benchmark point at release paths.

The six surfaces

Work them in this order — later ones depend on earlier ones.

1. Installed binaries (~/.cargo/bin)

From ~/Github/mecha:

cargo install --path mecha-cli  --locked --force   # mecha
cargo install --path mecha-mail --locked --force   # mecha-mail, mecha-google, mecha-outlook

And the graph, from ~/Github/mecha-graph — installed with mecha since 2026-08-16, because ~/.mecha/config.toml runs ~/.cargo/bin/mecha-graph-mcp, not a repo path.

This inverted on 2026-09-01, and the old rule is the trap now. Until then the public checkout was a generated artifact — git archive HEAD from ~/Github/personalized_knowledge_graph minus an exclusion list, through a gate that deletes the tree it refuses — and this file said in bold that ~/Github/mecha-graph was not a source. It is the source now. Work happens there; the private repo keeps only its notes — HANDOFF, RESEARCH_LOOP, OPERATIONS — and the roster tooling, and is no longer where code is written. The gold eval sets are not there either: they moved to ~/.mecha-graph/eval/ (0600), resolved by eval::gold_path_from, and private PR #5 (2026-09-02) removed eval/gold*.jsonl and scripts/nightly-mecha.sh from that checkout outright.

What made that safe was moving the gate rather than dropping it. The public repo carries .githooks/pre-push and .github/workflows/denylist.yml, both running the same roster from outside the repository (~/.mecha-graph/ denylist.txt, 0600, and the PUBLIC_DENYLIST secret), and both fail closed: a missing or empty roster refuses the push rather than waving it through. Install the hooks once per clone — git config core.hooksPath .githooks — because a clone without it is a clone with no gate.

The version drift is what forced it: the export existed to keep two repos level and had let them reach 0.1.3 against 0.1.4, which is the cost the split was charging. The last export ran on 2026-09-01 and is mecha-graph#2; the histories stay disjoint, and nothing was back-published.

Nothing runs from the private checkout any more (2026-09-04, the owner's ruling: no calls to pkg, everything on the public repo). Both crontab lines run from ~/Github/mecha-graph — 01:30 scripts/nightly.sh and 08:00 scripts/nightly-mecha.sh, both of which exist there — and every MCP consumer runs ~/.cargo/bin/mecha-graph-mcp, byte-identical to the public release build: mecha's own [[mcp]] name = "graph", Claude Code's user-scope server (now also named graph; the pkg entry that pointed at the private repo's Aug-16 pkg-mcp — holding the empty pre-rename ~/pkg/graph.db, so every kg_* call answered with nothing rather than an error — was removed), and Hermes's mcp_servers.pkg entry, repointed the same morning. A host app keeps its old server process until it restarts: Claude Code sessions and the Hermes dashboard started before the change are still holding the private binary, and restart to pick up the new one. The private repo keeps only its notes and the roster tooling (above), and nothing on this machine executes from it. This paragraph used to say the 08:00 line ran the private copy of nightly-mecha.sh "by design"; that copy has not existed since private PR #5 on 2026-09-02, and the 01:30 line was the last one moved.

The "land public first" rule for a change spanning the two repos has no code case left: its worked example — --min-sources on the public binary and nightly-mecha.sh as the private caller passing it — collapsed when the caller moved public, and nothing that runs now lives on the private side. What still spans the split is the roster (~/.mecha-graph/denylist.txt and the PUBLIC_DENYLIST secret, read by the public repo's .githooks/pre-push and denylist.yml) and the operator docs, neither of which gates a binary; the reflex that survives is that a public-side hook which reads a roster must be able to read it before a roster edit relies on it.

cargo install --path mecha-graph     --locked --force   # mecha-graph (CLI)
cargo install --path mecha-graph-mcp --locked --force   # mecha-graph-mcp (MCP server)

Both lines, every time — and the second one is the one that matters at runtime. They are separate crates that both link mecha-graph-core, so installing mecha-graph alone leaves the MCP server running whatever library code it was last built against, silently. Found 2026-08-24: a session developing in the graph repo had been running cargo install --path mecha-graph all evening and left mecha-graph-mcp an hour stale — across an arc whose central fix was in fact::normalize_predicate, which had been auto-registering any predicate it could not alias (49 of 83 predicates arrived unreviewed that way). The CLI stopped doing it; the MCP server, which reaches that code through kg_upsert → assert_fact, would have gone on minting predicate fragments through the one write path nobody watches. A stale MCP binary is worse than a stale CLI, because everything mecha does at runtime goes through it and nothing about it announces its version.

A probe literal must be one the range added, or it proves only that the file exists. strings ~/.cargo/bin/mecha | grep -c '<literal>' is the check CLAUDE.md prescribes because the version string cannot tell builds apart — but it discriminates only if the string was not in the previous build. Take it from the diff's added lines (git diff <installed>..<target> | grep '^+'), never from memory of the code: on 2026-09-05 a probe on cannot determine the working directory printed 8 before and 8 after a reinstall, and would have called a failed install current. And state a count only for a command you ran verbatim, or say present/absent — a peer will re-run it, and a wrong number makes a correct deploy look wrong.

Verify: mecha --version names the commit you built — mecha 0.1.24 (git <sha>), the sha git rev-parse --short=9 HEAD prints in the checkout you installed from (a version number alone cannot tell two builds apart) — and the graph server answers from the installed path:

echo '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}' \
  | ~/.cargo/bin/mecha-graph-mcp | python3 -c 'import json,sys; print(len(json.load(sys.stdin)["result"]["tools"]), "tools")'

And that probe only proves the file. Every host that spawned the server as its own child keeps the old process until it restarts — the long-running mecha units in §2 that reach setup::prepare* do (mecha-serve, mecha-slack, mecha-triggers, mecha-drain; the voice worker runs worker.py under the Pipecat venv and spawns no MCP child), and so do Claude Code sessions (user-scope server graph) and the Hermes dashboard, which no step here restarts. The timer-fired oneshots (mecha-mail-classify, mecha-ruminate, mecha-frontdoor) need nothing here, as §2 says: each firing execs a fresh mecha and a fresh server, and restarting one by hand does not refresh a process, it runs the sweep. What those units do need is ~/.cargo/bin on their Environment=PATH, because a bare command = "mecha-graph-mcp" resolves against the unit's PATH, and what a failed spawn does depends on the caller: everything that reaches setup::prepare* (mail classify, frontdoor, serve, slack, triggers, validate, learn — not reflect, which builds its provider from Config::load and spawns no MCP child) skips the server with one stderr line and carries on — the classifier or the front door would run a whole sweep with the kg_* tools absent and exit 0 — while the four direct-connect commands (distill, corroborate, gossip, vet) fail that step loudly with a non-zero exit, which in ruminate.sh (not set -e) shows up in the dated logfile while the script walks on. The silent case is the one to guard; the loud one is the good outcome. Every shipped unit that execs mecha, directly or through a wrapper, now carries that line (scripts/*.service, scripts/voice/*.service) — and the shipped file is not the installed one. ~/.config/systemd/user/*.service are copies, and a copy drifts: on 2026-09-06 four installed units (mecha-serve, mecha-frontdoor, mecha-mail-classify, mecha-ruminate) predated the PATH line their repo copies carried, and mecha serve — started by absolute path, so the unit looked healthy — had answered 502 on every graph page since the 2026-09-05 reboot because its bare mecha-graph spawn ran on the manager's PATH. When a unit's child says "not found", ask systemd what it will actually exec with — the merged view, drop-ins included, over the installed units (an installed unit with no scripts/ copy, like mecha-drain, is exactly the kind a repo-side loop cannot see) — and only then diff the file to find out why:

# a running unit: the PATH its process really holds (manager inheritance
# and EnvironmentFile= included, which `show -p Environment` does not see)
# every installed mecha unit, from the manager rather than a list kept by hand
for u in $(systemctl --user list-unit-files 'mecha-*.service' --no-legend | awk '{print $1}'); do
  pid=$(systemctl --user show -p MainPID --value "$u")
  printf '%s: ' "$u"
  if [ "$pid" != 0 ]; then
    tr '\0' '\n' < /proc/$pid/environ | grep '^PATH=' \
      || echo '(running with no PATH in its environment)'
  else
    # not running (a timer-fired oneshot between firings): the Environment=
    # assignments, drop-ins merged — inheritance from the manager is invisible here
    systemctl --user show "$u" -p Environment
  fi
done
# then, for a unit whose PATH is wrong or missing, find out why:
#   systemctl --user cat <unit>          # main file + drop-ins
#   diff scripts/<unit>.service ~/.config/systemd/user/<unit>.service
#   (mecha-serve ships from scripts/voice/, not scripts/)
# and the fix, when the repo copy is right and the installed one drifted:
#   cp scripts[/voice]/<unit>.service ~/.config/systemd/user/ && systemctl --user daemon-reload
#   EXCEPT mecha-slots.service: its repo copy ships `--account YOUR-MAIL-ACCOUNT`,
#   a placeholder, so it always diffs; copying it over would replace the live
#   polls account. Edit the installed unit's lines by hand instead.
#   then restart a long-running unit; a timer-fired oneshot picks it up at its next firing.
# not a unit, but installed the same way and drifting the same way: the
# daytime mail sweep's ExecCondition= runs a *copy* of scripts/model-idle.sh,
# and a stale copy fails toward running the sweep
#   diff scripts/model-idle.sh ~/.local/bin/mecha-model-idle
#   install -D -m 755 scripts/model-idle.sh ~/.local/bin/mecha-model-idle   # when the repo copy is right

mecha-serve carries the line twice on this machine — the installed unit and mecha-serve.service.d/path.conf — and the drop-in is the one to keep: it adds ~/.local/bin and holds the incident's record, and the later assignment wins, so cat showing both is the expected state, not drift.

A silent line is the trap the loop guards against: a unit with no PATH must print that, not let its label run into the next unit's value.

A file diff alone misreads in both directions: a unit correct only through a drop-in prints as drifted, and a unit byte-identical to the repo but overridden by a stale drop-in prints as clean. (mecha-voice-serve is disabled, inactive and deliberately left stale, so a file diff over scripts/voice/ reports it forever — another reason to start from the effective state.)

After every install of mecha-graph-mcp, either restart those hosts or accept that they run the previous build until their next start; nothing about a stale child announces itself, which is the same silence the removed pkg-mcp entry hid behind (the two-repo paragraph earlier in this step). Check with pgrep -af mecha-graph-mcp and compare each process's start time to the binary's mtime.

The charter is a wire format across machines, and the parser only ever tightens. A [line.sensor] table (2026-09-04) or a setpoint = 0 (refused since the readings phase the same day) authored on the updated machine makes the older binary on the other one run with no charter at all — one stderr line the TUI's alternate screen covers, not a refusal — until it is updated too. Update both machines before editing the charter from either; mecha doctor on the stale one reports it as the emergency it is.

1b. The web app's assets

mecha serve serves the UI from [web] assets in the live config, which points at ~/.mecha/web/dist — a stable home, deliberately not a repo or worktree path (production once served from a Claude session's worktree, which is the deferred-failure shape: fine until the session directory is cleaned). The dist is a build artifact, not in git, so a binary install does not update it:

cd <clean worktree>/web && npm ci && npm run build
rsync -a --delete dist/ ~/.mecha/web/dist/

When the same range changes the binary too, restart mecha-serve first and rsync last: step 1's install, then the restart (step 2, whose hold check is what "clear" means here), then this rsync. mecha serve reads assets per request, so a new page reaches the phone the moment it lands, and a new page on the old binary can fail silently in the unsafe direction. On 2026-09-30 #429's modal sent a mask that the old image_generate ignored, so for about two minutes a painted edit would have redone the whole picture. If a hold keeps serve up, keep the old dist live until it clears. An asset-only change needs no restart, so the order does not matter for one.

Before that rsync, ask what is deployed — the dist may be another lane's live test. git tag -l deployed-local in the main checkout names the commit whose build is on the box when a session deployed something other than main (re-point it whenever you deploy; delete it when main is deployed). And read the mismatch correctly even when the tag is missing: a served bundle hash that matches no tree you can see is far more likely to be another session's deployment than a leftover — on 2026-08-29 it was exactly that, filed as "stale", and the rsync that followed silently reverted a surface the owner was live-testing. If the dist is not what you are about to install, announce to the live sessions before replacing it.

For an asset-only change, restart nothing. With a binary change, the restart already came before the rsync, as above. Either way, verify the served page, not the directory: the 8443 door returning 200 with the new bundle hash.

web/public/ lands at dist root, and one of those files is load-bearing. voice-uplink-transform.js is the buffered uplink's tap (docs/VOICE-LINK-DESIGN.md §2.1, since #231 on 2026-09-14): a page that loads it declares the channel in its offer; a dist rsynced without it is a 404 the page turns into an RTP fallback — heard, but with #226's behaviour on a stall rather than #231's, and nothing in the repo to say why. Check it through the door the phone uses, the way the bundle is checked: curl -s -o /dev/null -w '%{http_code} %{content_type}\n' https://YOUR-HOST.YOUR-TAILNET.ts.net:8443/voice-uplink-transform.js → 200 text/javascript (the host is the tailnet name; it lives in docs/OPERATIONS.md, which is gitignored and present only on the operator's box — which is why it is a placeholder here rather than a value). Not the loopback origin: tailscale serve proxies :443 and :8443 to 127.0.0.1:63242, so probing 63242 with a hand-supplied Tailscale-User-Login tests the app and skips the door — and the 404 this check exists to catch is one the phone gets.

When a header probe echoes a value back (If-Modified-Since from a Last-Modified you just grepped), strip carriage returns first — | tr -d '\r' — or the CR rides into the outgoing header and the server answers 400, which reads exactly like a server bug and is your curl. Two sessions hit it independently within an hour on 2026-08-29. Since tower-http 0.7 the same probe can send If-None-Match with the ETag the 200 carried (same CR trap), and a 304 from either header echoes both validators back — a bare 304 with no ETag means the binary answering predates the bump.

2. The long-running services

These hold an open file handle on the old binary and must be restarted after step 1 — and not through a run in flight. Read ~/.mecha/holds first: each file is a run holding the model (hold.rs), and its pid says which unit it lives in:

for f in ~/.mecha/holds/*.hold; do [ -e "$f" ] || continue
  # The pid is in the name (`{pid}-{uuid}.hold`), as `hold::pid_of` reads
  # it: an unreadable hold is still a hold, never a stale one.
  name=$(basename "$f"); pid=${name%%-*}
  what=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["what"])' "$f" \
         2>/dev/null) || what='(unreadable, still a hold)'
  printf '%s  %s  ' "$name" "$what"
  tail -1 /proc/$pid/cgroup 2>/dev/null || echo "(pid $pid gone: a stale file)"
done

A hold whose cgroup is a unit you are about to restart is somebody's live run. On 2026-09-30 it was the owner's persona chat in mecha-serve. Wait for it to clear, or restart the other units now and that one after. A hold in a unit you are not restarting does not block; on 2026-09-30 that was mecha-ruminate's nightly validate. A new page waiting on this restart stays unpublished until it happens (step 1b).

The serve restart is gated in code, not by reading the list above. On 2026-10-01 at 13:09Z the count was printed beside an unconditional restart, read 1, and the restart went ahead anyway — it cut off a chat's picture 81 s into rendering. The loop above stays as the diagnostic: it names each run (what), which the gate does not read. Restart the rest, then serve only through the test.

What the holds directory can say is narrower than "a run is in flight in serve", so the test treats the gaps as held where it can see them:

• A pending model switch is held. While a switch waits, a serve turn yields its hold (hold_when_clear: the run is the side that yields), so for the whole of a model load no .hold file names it. The switch-*.json beside the holds is the sign.
• A turn on a non-local provider takes no hold at all. Follower::enter holds only through a loopback router, so a turn on a cloud provider or a fallback is invisible here. If serve has one configured, check the page or journalctl --user -u mecha-serve before restarting.

# Is a run held inside mecha-serve? Unknown is held: a holds directory that
# exists and cannot be read, or a live pid whose cgroup cannot be read.
serve_held() {
  [ -e ~/.mecha/holds ] && [ ! -r ~/.mecha/holds ] && return 0
  # A switch waiting: turns in flight have yielded their holds to it.
  for f in ~/.mecha/holds/switch-*.json; do [ -e "$f" ] && return 0; done
  for f in ~/.mecha/holds/*.hold; do [ -e "$f" ] || continue
    name=$(basename "$f"); pid=${name%%-*}
    [ -d /proc/$pid ] || continue          # gone: a stale file, not a run
    cg=$(tail -1 /proc/$pid/cgroup 2>/dev/null) || return 0
    case "$cg" in *mecha-serve.service*) return 0 ;; esac
  done
  return 1
}
systemctl --user restart mecha-slack.service mecha-triggers.service \
                         mecha-drain.service mecha-voice-worker.service
if serve_held; then
  echo "HELD: a run or a model switch in mecha-serve — the loop above names it; run this block again once it clears"
else
  systemctl --user restart mecha-serve.service
fi

Running the block again once the hold clears restarts the other four a second time, which costs nothing. A new page waits on the serve restart (step 1b), so do not rsync it while this says HELD.

mecha-serve and mecha-voice-worker were missing from this list until 2026-08-25, which is worth naming because the omission is the shape this skill exists to catch. mecha-serve is three surfaces in one process — the web app, the chat agent, and the mounted voice facade — so it holds a handle on the new mecha binary and serves [web] assets; step 1b mentions restarting it, but only in the branch a session reaches when it changed the assets, so a pure-Rust change would have installed a binary nothing re-executed. mecha-voice-worker is the other direction and easier to miss: it runs scripts/voice/worker.py from the repo working tree, so it is the one unit here that goes stale on a change that never touched Rust at all and never appears in cargo install output.

And "from the repo working tree" means from whatever branch the shared checkout is on — check that before the restart, not after. On 2026-09-03 ~/Github/mecha was on a merged feature branch (a session had branched there, as the HANDOFF banner says not to), so restarting the worker would have relaunched the pre-merge worker.py and reported success; the fix that was being deployed (#145) would not have been live. Three things follow, each learned that morning:

git -C ~/Github/mecha symbolic-ref --short HEAD   # expect: main
git -C ~/Github/mecha status --porcelain          # expect: empty

• A worktree-isolated session cannot fix this. Its guard refuses git aimed at the shared checkout, and relaying the owner's instruction to the session that works there does not lift that session's gate either — a peer's report of the owner's word is exactly the shape a permission gate exists to refuse. So the switch is the owner's, or the session whose own user says so in its own session. Plan for that before you start the restart list, not when the worker is the last unit left.
• The switch has a runnable recipe in docs/HANDOFF.md (§Machine state, dated, 2026-09-03), reviewed pass after pass until it graded as code: prove the fast-forward for both HEAD and refs/heads/main, prove scripts/start-router.sh, scripts/mmproj.sh, scripts/start-moe-mtp.sh and scripts/voice/parakeet_server.py unchanged across the move (the first is llama-local's ExecStart since 2026-09-27, through the drop-in llama-local.service.d/router.conf, and sources the second; the third is the rollback the unit's own ExecStart names; the fourth is what mecha-parakeet runs from this tree), read any dirty file before discarding it, land on origin/main in one hop, and only then restart the worker. Copy that block; do not improvise a switch main && pull, which is the version that failed silently on a dirty file and would have passed through a stale local main.
• Name the digest. The launch-script check used to compare git blob ids (git rev-parse <ref>:path, d76da36c… that day); it is git diff --quiet over the launch files now. A peer re-checking with sha256sum got a different number for the same bytes and nearly read it as "the file changed", which would have restarted llama-local for nothing. git diff --quiet A B -- path is the algorithm-free form.
• scripts/mtp-graft.py is a launch input too, through a file it built. The uncensored Qwen3.6 preset serves a graft whose name hashes that script (start-router.sh graft_path), so a move that changes it retires the graft: before restarting llama-local, run the router script with a stub server (LLAMA_SERVER=/bin/echo XDG_RUNTIME_DIR=<scratch>) and run the "to graft its head" line it prints, if any. Skipped, the restart serves that arm without MTP (~70 against ~100 tok/s) and says so only in the journal (docs/LLAMA-SERVER.md §Router mode).

Verify the worker the same way as every other unit: its own Uvicorn running on http://127.0.0.1:7860 line, from a journal window that opens at the restart (--since "$since UTC", with since taken just before systemctl restart), or the old process's line passes the check.

The chatterbox container is not in the list either, and it moves with the worker in one order: the worker first, then docker restart chatterbox. It serves scripts/voice/chatterbox_server.py from the same shared checkout, and a new server refuses a control its model drops while an old worker sends exaggeration / cfg_weight on every sentence, so a new server under an old worker is a whole-voice outage that reads as a TTS failure (#511). Restart it only when that file or a module it imports from there (audio_stretch.py) changed, with no call live: the read-only mount is imported once at start, so an edit to either reaches nothing until the restart; docker restart keeps the container's environment, so changing CHATTERBOX_MODEL means recreating it (the docker run line is in VOICE-RESEARCH.md, beside the serving wrapper).

mecha-breeze-adapter is not in the list either, and it is the TTS the worker speaks through by default (since 2026-10-03; docs/VOICE-BREEZE-DESIGN.md). It runs scripts/voice/breeze_server.py from the shared checkout, imported once at start, so restart it — after the worker, with no call live — when that file or audio_stretch.py changed. Never restart mecha-breeze-tts for a repo change: it runs a build outside the repo (~/models/breeze-qwentts), and a restart costs a model load and forgets every registered voice (the adapter teaches them again on the next sentence).

mecha-parakeet is deliberately not in the list. It runs scripts/voice/parakeet_server.py, so restart it when that file changes — and only then, because coming back costs a model load and voice is deaf until it finishes.

llama-ocr is not in the list either, and restarts itself. It is socket activated (docs/DOCUMENT-EXTRACTION-DESIGN.md §6): llama-ocr.socket holds :8085, the server starts on the first request and stops after ten idle minutes, so it picks up a new llama.cpp build or launcher on its next cold start. What does need a step is a change under scripts/llama/: the units and launcher run from ~/.local/bin and ~/.config/systemd/user, not the tree, so re-run scripts/llama/install.sh (it copies, reloads, and re-enables the socket). Verify with a request, not is-active — an idle server is correctly inactive: curl -s localhost:8085/health must print {"status":"ok"}, and journalctl --user -u llama-ocr.service show a fresh listening on http://127.0.0.1:18085.

The timer-driven units need nothing. mecha-frontdoor, mecha-ruminate and mecha-slots are .timer-fired and exec fresh on each firing, so they pick up a new binary by themselves. Knowing which list a unit is on is the difference between a restart that matters and cargo-culting six of them.

mecha-comfyui-idle-reset.timer is the one installed by copy. It runs ~/.local/bin/comfyui-idle-reset, which scripts/comfyui/install.sh copied, so a change to scripts/comfyui/ reaches nothing until that script is run again. Verify it with systemctl --user list-timers mecha-comfyui-idle-reset.timer, and its last decision with journalctl --user -u mecha-comfyui-idle-reset -n 5 (every tick says what it decided; a check that cannot run fails the unit, which mecha doctor reports). It restarts comfyui.service on its own, but only when ComfyUI is idle and holding memory; a deploy never needs to restart ComfyUI for it.

One exception in kind, not in restart: mecha-slots.service's second ExecStart is the poll lifecycle's timer line and must read factory-publish drain; mecha-mail bookings; factory-publish polls sweep; mecha-mail polls --account <your mail account>; mecha polls sweep — the bookings leg names no account on purpose (the unit's comment says why), and the polls leg names the one polls send from — a unit still on the two-verb line silently leaves every meeting poll at "invites 0/N" forever, with nothing in the repo to say why. And a pick card is a calendar_create_event draft: mail__calendar_create_event must be in [outbox] tools (it is, by the documented default) or mecha polls sweep refuses to stage one, per record, per tick. The third fact lives in ~/.mecha/config.toml, which no build reaches: factory__poll_create and factory__poll_meeting_create stay in [outbox] tools and must not be in publish_tools (ruling 6 of MEETING-POLL-UX-DESIGN.md — the card is a letter); a config that still lists them there reviews the invitation as a publication, refuses edit on the owner's own sentence, and hides the release from the writing miner. Done 2026-09-05; a fresh clone's config example is already right.

Verify — and take a startup line, never is-active, since a unit that crashes on its first request is active for a while first. journalctl --user -u mecha-slack.service --since "2 minutes ago" should show the reconnect line ("Connected to … N owner(s)"), mecha-triggers should log "N trigger(s), N enabled · ticking every minute", mecha-serve should print both its doors ("voice facade on http://127.0.0.1:8990" and "mecha serve on http://127.0.0.1:63242"), and mecha-voice-worker should reach "Uvicorn running on http://127.0.0.1:7860". A service that comes back but logs nothing is not evidence of success.

3. The graph MCP server — folded into step 1 since 2026-08-16

~/.mecha/config.toml runs ~/.cargo/bin/mecha-graph-mcp (server alias graph, prefix_tools = false, tools are bare kg_*), so the graph updates through the same cargo install ritual as everything else — step 1 covers it, and step 2's restart is what hands the running Slack connector the new server (its MCP children are spawned at connector start). What this surface still owns: a repo build is no longer an install here either — a cargo build --release in the graph repo changes nothing mecha can see, which is the same trap as mecha's own binaries wearing a different repo. And there is a seventh binary, which no cargo install reaches. This paragraph used to say the graph repo's own nightly "builds and runs from its repo tree and is not mecha's concern". It does not build, and it is: scripts/nightly.sh sets PKG="$REPO_DIR/target/release/mecha-graph" and executes whatever is sitting there. Its link --auto step runs the kNN linker, straight into the owner's graph, at 01:30.

Found 2026-08-26, hours after a session repaired a linker bug out of the live graph — 30 placeholder nodes merged, 121 payloads rewritten, 23 accepted facts re-pointed. That binary was dated Aug 25. Left alone, the nightly would have re-run the old linker and re-staged the same damage while every version string on the machine read current. So:

cd ~/Github/mecha-graph && cargo build --release
ls -l target/release/mecha-graph     # must be today

Do not assume cargo install --path refreshed it — where cargo puts an install's intermediate artifacts is not a promise about that path, so check the date rather than reason about it. This is the skill's own thesis failing inside the skill's own text: verify the running thing, never the repo — and a cron job's binary is a running thing that answers to no --version anybody types.

mecha itself needs no change when graph tools change — it discovers tools via tools/list.

4. The benchmark binary

bench/run.sh uses target-musl/release/mecha, a static build — never the installed one, because the glibc build will not start in most task containers. It is rebuilt by bench/build-portable.sh (which bench/run.sh calls), but ask it what it was built from before trusting a scorecard: a stale one measures old code and labels the result with today's model, and a fresh one built from the wrong branch does the same while looking current — which is the sharper failure, because nothing about it is stale.

The date and the version string answer neither question, so build-portable.sh writes the branch, the commit and the binary's digest beside it:

cat target-musl/release/mecha.source
#   main@984a1ea0
#   sha256 9f2c…
sha256sum target-musl/release/mecha      # must match the line above

Read the suffix, and check the digest. A bare main@<commit> is a build from a clean checkout — the only form that says the binary matches the commit named. All three suffixes mean MECHA_BENCH_ALLOW_DIRTY=1 was set, because nothing else gets past the guard:

• +dirty — the tree had uncommitted changes, so the binary matches no commit and the line names only where it started.
• +raced — the tree changed while the build ran. build-portable.sh bind-mounts the live checkout rather than a snapshot, so a commit or an edit landing mid-build is compiled in; the commit named is where it started, not necessarily what it contains.
• +unverified — git could not establish whether the tree was clean. The branch and commit may still be real (master@eb6906a +unverified is a repo whose git status failed on a corrupt index while rev-parse answered fine); they read unknown@unknown when git could not read this directory as the root of its own checkout — either no repo at all, or a tree sitting inside someone else's, which is what an rsync'd copy under another checkout looks like. That second case is the dangerous one: git reads a repository perfectly well there, just the wrong one, so do not go hunting a broken git. Either way the cleanliness is unknown, which is not the same as clean.

Suffixes compose when they say different things — +unverified +raced is a build whose tree could not be checked and whose HEAD moved under it — but the same token never repeats. None of these is a build to hang a scorecard on without saying so.

A digest that does not match means the binary was replaced without its provenance — most likely by building in a clean worktree and copying the static binary into the shared checkout's target-musl/release/mecha, which is a real procedure (docs/HANDOFF.md, the 2026-09-03 and 2026-09-11 entries) and moves the binary without the .source beside it. Copy both files, or rebuild in place. Stated here rather than pointed at, because this is where the mismatch is diagnosed and a pointer only beats restating while it points somewhere. A missing .source means the binary predates this (2026-09-11) or was built by hand: rerun bench/build-portable.sh from a clean checkout rather than guessing.

5. The factory client (different repository, different version line)

factory-publish comes from ~/Github/mecha-factory, versioned 0.2.x independently of mecha's 0.1.x. mecha-drain-follow is a hand-installed bash wrapper around it and is not in that repo, so it needs nothing.

Fetch before installing. On 2026-08-15 the local checkout was 16 commits behind origin and still said 0.2.1 while the installed binary was 0.2.2 — installing from that tree would have been a silent downgrade of a running service.

cd ~/Github/mecha-factory && git fetch origin --tags && git status -sb
factory-publish --version          # compare against the newest tag

6. The factory server (the DigitalOcean droplet)

Not this machine, and not covered by anything above. The server runs on a DigitalOcean droplet in NYC1 (ubuntu-s-1vcpu-1gb-nyc1, a 1 vCPU / 1 GB box) behind mecha-factory.ai; everything local is a client of it. Note the apex domain resolves to Squarespace, not the droplet — the origin you want is gate.mecha-factory.ai (compute. is the same host).

spark reaches it as root, using a dedicated key generated here for this purpose (~/.ssh/mecha_factory_deploy, commented with the box's hostname). No ~/.ssh/config entry exists, so the key has to be named explicitly:

# Read-only: what is actually deployed right now
ssh -i ~/.ssh/mecha_factory_deploy root@gate.mecha-factory.ai \
    'factory --version; systemctl is-active mecha-factory.service'

# Deploy a released version
ssh -i ~/.ssh/mecha_factory_deploy root@gate.mecha-factory.ai \
    'factory-deploy vX.Y.Z'      # download, checksum, prove binary+config, swap, health-check
ssh -i ~/.ssh/mecha_factory_deploy root@gate.mecha-factory.ai \
    'factory-deploy --rollback'  # reinstall /usr/local/bin/factory.prev

The unit is mecha-factory.service; the binary is /usr/local/bin/factory, with the previous one kept beside it as factory.prev, which is what --rollback restores. Adding a Host factory block to ~/.ssh/config would make all of this shorter and is worth doing.

Check before assuming it is behind. On 2026-08-15 the droplet was already serving factory 0.2.4 — the newest release — while the local client was two releases back at 0.2.2. The staleness ran the opposite direction from the guess, and one read-only factory --version settled it in seconds.

This is production, on a 1 GB box, serving people who are not the owner. The read-only check above is always fine. Ask before deploying, and never infer that "update everything" included the droplet — it is a different machine from a different repository on a different version line.

Ordering constraints

• Install before restarting. Reversing them restarts the old binary and looks like success.

• Install before editing ~/.mecha/config.toml, and restart anything long-lived after. A key a binary does not know is a fatal parse error — ProviderConfig and friends carry deny_unknown_fields, deliberately, so a typo'd setting fails loudly rather than being silently ignored. The cost is that a new key breaks every older binary, including one already running.

  That failure is deferred and partial, which is what makes it expensive. A long-lived process parsed the config at startup, before the edit, so it keeps working — until it hits a path that re-reads config, which then fails in a subsystem with no apparent connection to what changed. Measured 2026-08-21: a TUI session started at 01:20 kept running after the 03:07 install (/proc/<pid>/exe showed mecha (deleted) — a process executes the inode it was launched from, so an install is invisible to it), and the symptom was show_file reporting a config parse error two hours later, which sent the model reading config.toml to investigate a version skew.

  /proc/<pid>/exe ending in (deleted) is the check, and it is the one that catches a stale process where mecha --version catches a stale file:

  Walk exe, never argv. A process launched off $PATH has argv mecha tui, so any pgrep for the install path misses precisely the interactive session most likely to be stale — which the first version of this check did:

  for p in $(ls /proc | grep -E '^[0-9]+$'); do
    t=$(readlink /proc/$p/exe 2>/dev/null) || continue
    case "$t" in
      "$HOME"/.cargo/bin/*"(deleted)") echo "stale: pid $p -> $t";;
    esac
  done

  Since 0.1.12 a stale session degrades less: TUI modals spawn children via the /proc/self/exe link (mecha-cli/src/exe.rs), so they keep working — running the session's own version — instead of dying with os error 2. The sweep still matters: a stale session is still running old code.

  The services in step 2 clear themselves, since restarting them re-execs. What this finds is everything else: a TUI, a mecha chat, anything a person left open in another terminal — none of which any systemctl line reaches.

  Some of what it finds will not be yours. On a machine running several Claude sessions against one checkout, the sweep surfaces other sessions' test servers and their MCP children — 2026-08-26 it found three on a peer's ./target/debug/mecha serve --port 8894. Walk the parent (/proc/<pid>/status PPid, then that pid's cmdline) before doing anything: a stale process is a thing to report to its owner, not to kill. Killing another session's process in a shared tree is the same accident the worktree rule exists to prevent.

• Never build while a benchmark is running. This host has unified memory and llama.cpp inference is bandwidth-bound; a parallel cargo build starves it. scripts/start-moe-mtp.sh records the day a "50x slowdown" was blamed on a flag when the real cause was memory pressure. Either finish the benchmark or stop it, then build.

7. The sandbox image

~/.mecha/config.toml's [sandbox] runs confined shell in the mecha-sandbox docker image (built from scripts/sandbox.Dockerfile). It bakes in its own Rust toolchain, so it goes stale independently of everything above — after a host toolchain bump, rebuild it or confined cargo quietly diverges from the cargo that CI and the host run:

docker build -t mecha-sandbox -f scripts/sandbox.Dockerfile .
docker run --rm mecha-sandbox bash -lc 'cargo --version'   # compare to host

(bwrap would be the cheaper backend but is blocked on this host: kernel.apparmor_restrict_unprivileged_userns=1. The config comment above [sandbox] records the targeted fix if that's ever wanted.)

What no amount of this reaches

~/.mecha/ is not a git repository. config.toml and triggers/*.toml live on one disk and are in no repo's history, so nothing above updates, restores, or reviews them. When a change lands there — a trigger's tool allowlist, a provider block — say so explicitly in the handoff, because the next clone will not have it.

Related coupled values that no build step checks, each silent when wrong: a llama-server -c against context_window in config.toml, a --alias against that provider's model, and temperature — which mecha sends, so the config overrides the server's --temp.