shockwave / api

# CLAUDE.md — companion server (`api/`)

The **companion** is the backend the desktop app talks to over HTTP. It is the **single source of truth** for everything synced: settings, secrets, workspaces (identity), chats + transcripts, and Telegram/cron state. It also *runs the coding agent server-side* for Telegram messages and scheduled (cron) jobs, using the same `agent-core` runtime the desktop bundles. Postgres is private to the compose network; the companion holds the one master encryption key. Read the root `CLAUDE.md` first for terminology and the desktop side.

Node 22 + Express 5 + Postgres (drizzle). Source is TypeScript throughout — the pure policy modules (`keys.ts`, `gitRemote.ts`) used to be plain `.js` and no longer are. esbuild bundles `src/` **and** `../agent-core` into `dist/server.js`.

**`npm run typecheck` here is the only thing that checks this tree.** esbuild strips types without looking at them, and the root tsconfig's `include` is `src/**` with nothing under it importing `api/`, so tsc never reached a single file in this directory. `api/tsconfig.json` covers `api/src/**` plus `../agent-core/**` — the latter deliberately, since this tree compiles agent-core into the server bundle and it should be checked against *this* manifest's dependency versions, not only the desktop's. Run it before you ship; the build will not tell you.

## Install (`install.sh`)

One-liner for a fresh Linux box: `curl -fsSL https://raw.githubusercontent.com/stephengpope/shockwave/main/api/install.sh | sh`. Installs docker via get.docker.com if missing, sets up a ufw firewall (default-deny inbound; allows every configured sshd port + 80/443 — host hardening, the companion's surface is unchanged), pulls `:latest` and **copies the host files out of it** into `/opt/shockwave-companion` (same `/host-files` mechanism the upgrade path uses — see the boxed rule under "Remote upgrade"), generates `.env` secrets (never overwritten after first run), **resolves this server's public address once and records it as `COMPANION_HOST`**, installs the `shockwave` command on PATH, pulls the prebuilt image, waits for `/health`, **runs `shockwave check`** (below), then prints the server URL, API key, and — in self-signed mode — **the certificate fingerprint to approve in the desktop**.

Re-running is the update path **and the recovery path**. Secrets are never regenerated, and **only flags you actually pass are updated** (`env_set` rewrites one key in place), so `--domain=` on a re-run adds a domain to an existing install — it used to be silently ignored, leaving no supported way to move off self-signed. `COMPANION_HOST` is refreshed if the public IP changed.

**It also clears any `SHOCKWAVE_TAG` a remote upgrade pinned**, which is what makes it a recovery rather than a faithful rebuild of a stuck box. Compose reads `${SHOCKWAVE_TAG:-latest}`, so an empty value *is* `latest` — and the host files were just unpacked from `latest`, so leaving an old pin would run one release's server under another release's compose file. Before this, a box whose upgrades were broken came back up on exactly the image it was stuck on, while the script's own header promised it "pulls the newest image".

Flags: `--domain=`, `--cert-email=`, `--yes`, `--no-firewall`. (`--cert-email` ↔ `COMPANION_CERT_EMAIL`: env vars carry a project prefix since they share a global namespace, flags don't since they're already scoped — strip the prefix and the two should be the same word. The old pair was `--email` ↔ `ACME_EMAIL`, which matched on neither count and named a protocol nobody filling in the field has heard of.) Test hooks: `SHOCKWAVE_DIR`, `SHOCKWAVE_IMAGE` (point at a locally built tag).

The image is `ghcr.io/stephengpope/shockwave-companion` — published multi-arch by the `image` job in `.github/workflows/release.yml`, on the **same `v*` tag** that cuts a desktop release, tagged `<tag>` + `latest`, with the tag baked in as `APP_VERSION` (surfaced by `GET /health` as `version`; `'dev'` for local builds). Compose has both `image:` and `build:` with `pull_policy: missing`: plain `docker compose up -d` pulls (install path), `docker compose up -d --build` builds from source (dev path). The image ref is `:${SHOCKWAVE_TAG:-latest}` — fresh installs run `latest`; a remote upgrade (below) pins `SHOCKWAVE_TAG` in `.env` so the compose/traefik files and the image are always the same release.

## Remote upgrade (`updater/`, `POST /update`)

The desktop compares its own version against `GET /health`'s and, when the companion is behind, offers a one-click upgrade → `POST /update {tag}` (bearer-authed; tag strictly `^v\d+\.\d+\.\d+$`). The route writes the tag to `UPDATE_TRIGGER_DIR` (the `updater-trigger` volume); the **`updater` compose service** (stock `docker:27-cli` + `updater/watch.sh`, holds the docker socket, deliberately zero network surface — the trigger is a file, not a listener) picks it up and spawns a **detached one-shot helper** running `updater/apply.sh` (detached so `up -d` recreating the updater itself can't kill the run). `apply.sh`: pull the tag's image (nothing on disk changes until it is on the box) → `docker create` + `docker cp` the host files out of `/host-files` into a staging dir → `mv` them into place (rename, never cp — apply.sh replaces itself) → pin `SHOCKWAVE_TAG` in `.env` → `docker compose up -d --remove-orphans`. Every failure aborts with the old stack untouched. `SHOCKWAVE_IMAGE` is the test hook. Schema migrations ride along (the image carries `init.sql`; `ensureSchema` re-applies on boot). A 503 `updater-unavailable` from `POST /update` = a pre-sidecar deployment; the desktop tells the user to re-run install.sh once. Not covered (by design): a release adding a **required** `.env` secret still needs the install one-liner.

> **The host files ride in the IMAGE, and there is no file list on the server.** They used to be fetched from raw.githubusercontent by a `FILES=` list hardcoded in `apply.sh` — meaning the list a box installed *from* was the one baked into whatever `apply.sh` it had installed *last*. So removing a runtime file from the repo made every older box fetch a 404 and abort, permanently, because the step that replaces `apply.sh` (3) is downstream of the fetch that fails (1). Deleting `traefik/gen-router.sh` in **v1.0.85** — the cert fix that gave `tls.ts` sole ownership of the dynamic config — did exactly that to every box still on v1.0.84. It is silent from every angle: `POST /update` returns `{ok:true}` the moment it writes the trigger file, the sidecar spawns the helper, the helper aborts having changed nothing, and the desktop is still waiting for a reconnect on a version that will never arrive. Nothing on the box is broken, so nothing reports anything; the only evidence is `docker logs shockwave-update-run`.
>
> Both paths now `docker cp` the files out of `/host-files` in the tagged image (`api/Dockerfile`), whose layout mirrors the install directory. Four things follow, and they are the reason this is the fix rather than a patch: the file set comes from the release being **installed** rather than the one already installed, so adding or deleting a runtime file is no longer a breaking change; there is nothing on the server that can be stale; the host files cannot disagree with the image they configure, having been built from the same commit; and upgrading no longer needs GitHub reachable at all. **Do not reintroduce a list, in either script** — `tests/hostArtifacts.test.js` fails on one, and on any fetch of a runtime file over the network.
>
> One consequence to keep in mind: an upgrade to a tag cut **before this change** finds no `/host-files` and aborts cleanly, saying so. And a box jammed by the old mechanism cannot be rescued by a release, since it cannot install one — re-running `install.sh` is its recovery, which is why that script now also clears any `SHOCKWAVE_TAG` pin (see below).

## Deploy model (`docker-compose.yml`)

Five services (postgres, api, traefik, updater, autoheal):

- **postgres** (`postgres:16-alpine`) — private to the compose network, **not** port-mapped by default. `init.sql` is mounted into `/docker-entrypoint-initdb.d` (runs once on a fresh `pg-data` volume); `ensureSchema` re-applies it idempotently on every boot.
- **api** — built from the **repo root** (`context: ..`, `dockerfile: api/Dockerfile`) so the build can pull in `../agent-core`. Bound to **`127.0.0.1:8080` only** — localhost, never a public surface. Reaches Postgres over the compose net. **Owns the whole `traefik-dynamic` volume** — certificate, `tls.yml` and `router.yml` (see `settleTls` below).
- **traefik** (`traefik:v3.3`) — the **only** public surface; `:80`→`:443`, terminates TLS, reverse-proxies to `http://api:8080`. Self-signed by default; real Let's Encrypt when `COMPANION_DOMAIN` is a domain. It **does not wait** for the api's config write: the file provider watches the directory, so config that lands a second later is picked up (verified), and gating startup on the api's healthcheck would delay every deploy by up to a health interval to fix an ordering problem that does not exist.
- **updater** (`docker:27-cli` + `updater/watch.sh`) — the remote-upgrade sidecar. Holds the docker socket and has **deliberately zero network surface**: its trigger is a file on the `updater-trigger` volume, not a listener. See "Remote upgrade" above.
- **autoheal** (`willfarrell/autoheal`) — restarts any container labeled `autoheal=true` (today: `api`) once Docker marks it unhealthy, via the api's Dockerfile `HEALTHCHECK`. Compose's own `restart:` policy only covers a *crashed* process, not a wedged one, and there is no operator on the box to notice a companion that stopped answering.

### What the agent can actually run (`api/Dockerfile`)

The base image is bare, and the agent runs as `node` with **no root and no sudo** — so whatever is not baked in is unavailable forever, and a turn that needs one missing tool simply fails. Two rules follow from that.

**The bar is the DESKTOP host, not "what a server needs".** The other host this agent runs on is someone's Mac, where curl, jq and pip already exist. So a tool missing here doesn't just fail — it makes a skill the agent wrote for itself in the app break the moment the same agent runs it from Telegram or on a schedule, which is the divergence this whole tree exists to prevent. `curl` **and** `wget` were both absent until 2026-08-09, meaning there was no way to make an HTTP request from bash at all: firecrawl and playwright-cli fetch web *pages* and are no help calling a REST API, hitting a webhook, or downloading a file from a URL. On top of the file-handling set the Dockerfile already documents, the image now carries `curl`/`wget`, `jq`, `sqlite3`, `ripgrep`, `less`/`bc`/`tree`, `procps`, `dnsutils`/`ping`/`netcat`, and `requests`/`yaml`/`bs4`.

**Python is writable, on a volume of its own.** Debian's python is externally managed, so `pip install` into it is refused and the agent had no route to add a package. `/opt/pyenv` is a virtualenv built at image time, first on `PATH`, mounted from the `agent-venv` volume — so a package the agent installs survives a redeploy. Three things about it are load-bearing:

- **`--system-site-packages`, or the venv HIDES apt's python packages.** Putting its `bin/` first on PATH is what makes `python3` and `pip` mean this one; without the flag that same move makes `openpyxl`, `PIL`, `lxml`, `requests`, `yaml` and `bs4` invisible, and those are what the attachment path leans on.
- **Nothing seeds it, and nothing should.** Docker populates a **fresh** named volume from the image's content at the mount point and leaves a populated one alone — which is both halves of what's wanted, with no startup script to write: a new box comes up with `yt-dlp` already there and needs no network at boot, and an upgrade cannot overwrite a package the agent installed.
- **The cost is drift, and it is accepted.** A writable volume means two boxes on the same release can hold different packages. `docker volume rm shockwave_agent-venv` resets one to what the image ships.

`yt-dlp` lives in that venv rather than in apt for exactly this reason: Debian ships a 2023 build the sites it downloads from broke long ago, and upstream ships fixes weekly because those sites actively try to break it. In the venv the agent can `pip install -U yt-dlp` and heal it between releases. It merges streams with `ffmpeg`, which is already in the image for voice notes.

**Three exposure modes:** (a) localhost dev on `127.0.0.1:8080`; (b) public via Traefik TLS on `:443` — self-signed cert for `COMPANION_HOST` with no domain, Let's Encrypt with `COMPANION_DOMAIN`; (c) **ngrok raw tunnel** straight to `127.0.0.1:8080` (ngrok brings its own trusted cert, so set `COMPANION_DOMAIN` to the ngrok host and Traefik/self-signed is bypassed).

### TLS is settled at boot, before the server answers (`settleTls`)

- **No domain** → create-or-reuse the self-signed certificate for `COMPANION_HOST`, write `tls.yml`, write the catch-all router. **A failure is fatal**, exiting like a missing `MASTER_KEY` does. Coming up anyway means Traefik serves its own throwaway certificate, desktops approve *that*, and the real one replaces it later — a server whose identity is about to change is worse than one that didn't start.
- **Domain set** → `removeSelfSignedCert()`, then write the `Host(...)` + `certResolver: le` router. Let's Encrypt owns the certificate, so a spare private key claiming to be this server, still registered as Traefik's default, is pure risk with nothing using it.

> **Reuse applies to the CERTIFICATE, never to the configuration that publishes it.** `ensureSelfSignedCert` used to write `tls.yml` only on the branch that *generated* a certificate; reuse returned early. So a dynamic directory holding a valid `companion.crt` and no `tls.yml` was a **stable** state, and it is silent from every angle: Traefik's answer to having no default certificate is not an error, it is a throwaway certificate it mints at **every startup**, so the server serves HTTPS perfectly under an identity no desktop approved — while this process logs `self-signed certificate ready`. Nothing repaired it, because every repair path (restart, `POST /update`, re-running `install.sh`) came back through the same early return; `shockwave rotate-cert` was the only way out, and it fixes a config bug by **changing the server's identity**. It reaches the user as `check`'s `https on 443 is not serving a valid certificate for this address` — note that is a *different* line from `this server has no certificate of its own`, so the certificate being present and the error being shown are not in contradiction. The certificate is an identity and must not churn; the two YAML files are derived, deterministic and byte-identical on every boot, so there is nothing to preserve by guarding them. **Asserting the whole desired state on every boot is what makes a boot a repair** — the fingerprint does not move, so no desktop re-approves.

> **The dynamic config volume has ONE writer.** `router.yml` used to come from a one-shot `traefik-config` sidecar (`gen-router.sh`) while the certificate and `tls.yml` came from here — two containers writing one directory, neither able to see the other, each re-deriving the same IP-vs-domain decision by hand. It is `api/src/tls.ts` alone now, so the mode is decided once (`normalizeTlsEnv`) and written once, and the sidecar's runtime `chown` of the shared volume is replaced by the **Dockerfile** pre-creating `/etc/traefik/dynamic` as `node` — the identical mechanism already used for `/data/agent`, and it removes the ordering the sidecar implied. Both files are written through a temp file and renamed: Traefik watches the directory, a rename is atomic, and the `.tmp` extension is one its file provider skips.

`/telegram/connect` **reads** the certificate (`readCertPem`) and never creates one. It used to create it, which meant a fresh install had no certificate of ours at all — Traefik served its throwaway, the desktop approved that, and connecting Telegram swapped it. The fingerprint changed on a routine action, and a user who meets the desktop's identity-changed warning during normal setup learns to click through the one prompt that catches a real attack. Reuse is also why `certUsable` exists: regenerating when nothing changed would move the server's identity for no reason and force every desktop to approve again.

**One command on PATH: `shockwave`** (`api/host/shockwave`, symlinked once from the install dir into `/usr/local/bin`). Subcommands: `check` (below), `fingerprint` prints the certificate's fingerprint — the value the desktop asks you to compare against — `rotate-cert` deletes it and restarts the api so boot makes a fresh one (one code path, no second implementation), plus `status` / `logs` / `version`. Rotating is the recovery for a stolen private key; there is no detection for that, and rotating forces re-approval on every desktop, which is the point.

### `shockwave check` — the three things the installer prints, actually tested

Nothing else tests them. The installer's wait loop probes `http://127.0.0.1:8080/health`, which is the api container talking to itself: it proves the process booted and Postgres answers, and **nothing about the address, certificate or key that get printed underneath it**. So a Traefik router that didn't come up, or — the common one — a domain whose Let's Encrypt certificate never issued, both reach the user as `✓ Install complete` followed days later by `Couldn't connect` in Settings, with no way to tell which half is broken. ACME runs *after* `docker compose up -d` returns, so that gap is structural rather than a race.

`check` runs the containers → local `/health` → https on 443 → `GET /settings` with the bearer key, prints ✓/✗ per line and exits non-zero on any failure. Four decisions in it are load-bearing:

- **Every request is pinned to loopback with `--resolve <addr>:443:127.0.0.1`.** The address is still what's presented for SNI and matched against the certificate, so the TLS half is verified exactly as the desktop will verify it — but the connection never has to leave the box. AWS and GCP NAT a public address and don't hairpin it, so a direct probe there fails on a perfectly healthy server, and a check that red-flags working installs is worse than none.
- **Self-signed mode passes the server's own certificate as `--cacert`, never `curl -k`.** Skipping verification would pass while Traefik served the throwaway certificate it generates when ours is missing — precisely the failure worth catching, and the one whose fingerprint changes on every restart.
- **The API key goes in over stdin (`-K -`), not as an argument**, so it never appears in `ps` while the check runs.
- **One advisory line, never a failure:** a direct (unpinned) probe of the real address. It runs on the box, so it cannot speak for the path from the user's laptop; the summary says so and points at the cloud firewall rather than implying the server is at fault.

It lives in `host/shockwave` and not inline in `install.sh` for the reason in the boxed rule below: the dispatcher ships with every release, so this reaches existing boxes on their next upgrade — and the point of a diagnostic is being able to run it the day something breaks, not only the day it was installed.

> **Add subcommands to `host/shockwave`; never add a second file to `/usr/local/bin`.** This used to be one generated script per command, each written by `install.sh` from a heredoc and symlinked separately. Those scripts existed nowhere but inside the installer, so no upgrade could ever deliver them. A box installed before a command existed simply never had it, with no way to find out but typing it. Now the symlink's target path never changes, so replacing one file IS the update, and `apply.sh` never needs to write outside the install dir (it can't: `watch.sh` mounts only that dir into the helper). `tests/hostArtifacts.test.js` pins all of it — one symlink in `install.sh`, none in `apply.sh`, the image carrying every file the host needs, and neither script holding a file list or fetching one over the network. The dispatcher's **executable bit in the repo is now load-bearing**: `COPY` preserves it into the image and `docker cp` carries it to the host, so it is what makes the command runnable (both scripts `chmod 755` it anyway, since a 644 dispatcher is a command that says "permission denied" and nothing else).

**Env (`.env`, see `.env.example`):** required `POSTGRES_PASSWORD`, `MASTER_KEY` (32 bytes base64 — validated at boot, process exits if missing/wrong length), `API_KEY` (bearer token; the server stores only its SHA-256 hash). `COMPANION_HOST` — this server's public address, **written by the installer** and required in self-signed mode (the certificate is issued for it at boot). Optional `COMPANION_DOMAIN` (domain or ngrok host; empty ⇒ self-signed mode), `COMPANION_CERT_EMAIL` (Let's Encrypt expiry warnings), plus tunables `PORT`, `CRON_ENABLED`, `CRON_REFRESH_SCHEDULE`, `REVIEW_ENABLED`, `REVIEW_SCHEDULE`. `PI_CACHE_RETENTION: long` is set in compose and read by pi-ai straight from the environment: its default holds the Anthropic prompt cache for 5 minutes after the last message, which is shorter than the gaps in an ordinary Telegram conversation, so most turns re-read the whole system prompt and history from scratch. The trade is asymmetric — a cache write costs 2× base instead of 1.25×, but that applies only to the one new message, while a miss reprocesses everything before it and that grows with the conversation. (The per-run watchdog and the working-dir TTL used to be env here; they are now the synced settings `codingAgent.maxRunMinutes` / `codingAgent.scratchTtlDays`, so the desktop can set them and the desktop's own scratch cleanup expires on the same number.) `api/.env` is git-ignored — never commit it.

> **An IP in `COMPANION_DOMAIN` is normalized to `COMPANION_HOST`, in three places.** The two variables are one thing — this server's address — split by which TLS mode you want, and nothing checked that the value suited the variable. An IP in the domain slot took the worst path available: `settleTls` deletes the self-signed certificate because a real one is supposedly coming, Let's Encrypt can never issue for a bare IP so none arrives, and Traefik falls back to the throwaway certificate it regenerates at **every startup** — a new fingerprint for every desktop to approve after every restart, which is how you teach someone to click through the one prompt that catches a real attack. An IP can only mean self-signed, so `normalizeTlsEnv` (`server.ts`, before any reader), `install.sh --domain=`, and `host/shockwave`'s `load_env` all map it to the host and say so. (There used to be a fourth, in the `gen-router.sh` sidecar — a different container that could not see the first. Folding the router into `tls.ts` deleted it: the two survivors are a shell installer and a shell diagnostic, neither of which can call into the server.) The check rejects non-IP characters first so a hostname that merely starts like one (`10.0.0.1.nip.io`) keeps its real certificate.

**The three secrets are deleted from `process.env` right after boot reads them** (`server.ts`). pi spawns the agent's `bash` with `{ ...process.env }`, and `gitFixer`/`git.ts` inherit it too — so without this, anything the agent could be told to run could `env` and read the master key that decrypts every row in `secret_value`, the bearer key, and the Postgres password. Telegram and cron turns run unattended, so that instruction can arrive as a file in a repo, a `cron.json` prompt, or a DM. Not complete on Linux (`/proc/<pid>/environ` still holds the originals from exec); it closes `env`/`printenv`, and the full fix is to stop passing them as env at all.

**`COMPANION_HOST` is not looked up at runtime.** The installer already resolves the public IP to print your Server URL, and the certificate must be issued for the exact address you type into the desktop — two independent lookups can disagree, and then the certificate is for one address while you connect to another. One lookup, at install, written down.

## Logging (`log.ts`)

One pino root; every subsystem logs through `logger(sub)` — a child with a `sub` field (`git`, `fixer`, `cron`, `telegram`, `agent`, `sweeper`, `http`) — so `docker compose logs api` (or `shockwave logs`) is the single place to look and one grep follows a run across subsystems by `chatId`. Don't `import pino` anywhere else.

**What gets a line: every boundary result.** A turn started/finished, a check-in's outcome, a fixer attempt's verdict, a git call that failed *and its stderr* (`errStr` prefers `e.stderr` — the part that says why). Not per-step chatter. The rule that matters: **a failure that is caught and converted into a status (`'error'`, `'conflict'`, a silent retry) must log before the conversion.** The catch blocks in `git.ts`/`gitFixer.ts` used to swallow the only evidence of why a run failed — a fixer whose model provider was unreachable produced exactly the same visible outcome as a merge it genuinely couldn't resolve. Related: a cron run whose check-in returns `'conflict'`/`'error'` no longer records as a clean run — `fireJob` writes it to `cron_state.last_error`, since work that never reached GitHub is a failed run even though nothing threw.

**Never log a payload that can carry secrets whole** (settings objects, agent run payloads — they hold API keys). Pick fields.

## Files

- `log.ts` — the one pino root + `logger(sub)` + `errStr`. See "Logging" above.
- `server.ts` — Express app: boots the pool + companion agent runtime, registers all routes + the bearer-auth middleware, the SSE feed, scheduler + sweeper, graceful shutdown.
- `db.ts` — pg `Pool` + drizzle wiring; `int8`→`Number` parser (epoch-ms); `ensureSchema` (idempotent `init.sql` re-apply on boot).
- `schema.ts` — drizzle table definitions (source of truth); `bytea` custom type + `epochMs` bigint helper.
- `store.ts` — the data layer: every drizzle query; seals/unseals secrets; `readSettings`/`writeSettings`, chats, transcripts, cron history, telegram account.
- `crypto.ts` — AES-256-GCM `seal`/`unseal` under `MASTER_KEY`; fresh 12-byte IV per write; returns `''` on decrypt failure.
- `keys.ts` — **pure** key policy (no db/electron import, unit-testable): which `(owner, field)` pairs are secret, agent-secret field lists, OAuth-owned fields, settings flatten/`setPath`, agent-secret split/join, value encode/decode. It does **not** declare which fields are credentials — it derives all three lists from `agent-core/credentials.ts` (`settingsCredentialPatterns()`, `agentSecretFields()`, `oauthOwnedFields()`), the one declaration bundled into both builds. Add a credential there, not here.
- `oauth.ts` — server-side token minting: `mintToken(name)` → a static token or a fresh (refreshed) OAuth access token; `patchOAuth` writes tokens back. (The desktop runs the *interactive* OAuth browser flow; the companion stores + mints.)
- `agentHost.ts` — builds the companion `AgentHost` for `agent-core`: persistence → store, events → feed, per-run scratch dir (`AGENT_DATA_DIR`, on the `agent-data` volume — the Dockerfile pre-creates + chowns it so the volume inherits `node` ownership), `send_message` tool, `getToken` → `mintToken`.
- `openFileStub.ts` — `open_file`, registered here so it can be **refused rather than missing**. The real one opens a tab in the app window (`src/main/openFileExtension.ts`); there is no window on a server. But the system prompt is written once at chat creation and names the whole catalog, so a tool the prompt advertises and pi never registered is one the agent is told it has and doesn't — asked what it can do it answers honestly that `open_file` is absent, and a call gets pi's bare *"Tool open_file not found"* instead of a sentence it can act on. Its `execute` is unreachable in practice (`DENIED` refuses `open_file` on telegram and cron, and the gate blocks the call first) and is written out anyway, because "unreachable" depends on a table in another file staying the way it is.
- `liveTool.ts` — what a tool is printing **right now**, per chat, in memory only. A running tool's output isn't a row until it finishes (`syncEntries` writes at `message_end`); mid-run it exists only as `tool_execution_update` events crossing the feed. `feed.ts` folds every published event through `note()` — **before** its no-subscribers early return, so the snapshot is captured even when no desktop is watching — and `/btw` reads `current(chatId)` to answer "what is it doing" while there is still nothing to read. Ephemeral by design: replaced on each update (pi hands the whole accumulated output every time, so this is a pointer swap, not an append) and dropped on `tool_execution_end` / `agent_end` / `agent_settled`.
- `feed.ts` — in-memory ephemeral SSE pub/sub, ONE global channel (not per chat). Also **`publishSettingsChanged()`**: the companion changes settings on its own (the bot's `/voice` and `/workspace`, OAuth tokens refreshing mid-run) and the desktop otherwise never finds out — main only pushes `settings:changed` after its OWN writes, so a change made through Telegram left the app showing a stale value until a reconnect. Deliberately carries **no payload**: the desktop re-reads and pushes a full snapshot, so there is one route for the data and one for the notification rather than two copies of the truth that can disagree. Called from `store.ts`'s mutators (see `announce` there), never from the routes — the routes are one door and the bot commands are another. A desktop can't subscribe per chat: the point is to hear about turns it doesn't know exist yet (Telegram, cron, another machine). Every event carries its `chatId`, so the client routes. Never stored — it mirrors what the `message` table already holds, so a client that misses events re-reads with `?after=`.
- `scheduler.ts` — croner scheduler: one fire-cron per `cron.json` entry + a refresh cron that reconciles registrations non-destructively (ETag).
- `cronRun.ts` — executes one cron run: checkout → agent turn (stream to feed) → deterministic check-in (git-fixer on conflict).
- `backgroundSweeper.ts` — the clock for BOTH background processes: one croner tick that asks two questions and starts at most one run. See "Background runs" below.
- `backgroundRun.ts` — executes ONE background run of either kind: checkout → one turn under a watchdog → `checkInWithFixer`. The two kinds are `REVIEW` and `MEMORY`, four fields each (source, title, prompt builder). **Mechanism, not policy** — everything that makes the two processes different lives in the sweeper and the prompts, not here. It was two files first, and the diff between them was four string literals; the cost showed up as `timezone` being passed by one and not the other. `cronRun.ts` stays separate: it reads its prompt from `cron.json`, disposes one-time jobs, and delivers files, and shares the part that matters (`checkInWithFixer`).
- `git.ts` — server-side git CLI: `prepareCheckout` (claim-or-reuse-or-shallow-clone), `cloneFresh`/`refreshPristine` (shared with the checkout queue), `checkIn` (add/commit → `syncAndPush`), `syncAndPush` (fetch/merge/push, one retry), `landed`, `cleanup`. **`WORK_BASE` is `DATA_BASE/work`** — on the `agent-data` volume beside `runs/` and `files/`. It used to read a `CRON_WORK_DIR` env var that was set nowhere, so it fell back to the container's temp dir: the one thing here expensive to rebuild was the only one not kept, and every restart or `POST /update` made the next message in every chat re-clone. Derived from `DATA_BASE` now, so there is no variable to forget. **The PAT is never in the remote URL.** `clone` and `remote set-url` both persist whatever URL they're given into `<dir>/.git/config`, and `<dir>` is the agent's own cwd for the turn — so an embedded PAT was a file the agent could read (`git remote -v`), granting write access to every repo the token covers, for `RUN_DIR_TTL_DAYS` after the run. Auth now goes through a `GITHUB_PAT` child env (`gitEnv`) answered by a credential helper passed **on the command line** — nothing on disk, same mechanism as the desktop's `src/main/sync.ts`. Existing checkouts predating this still hold the old URL — `prepareCheckout` rewrites it on reuse, but wipe `WORK_BASE` on deploy to be sure. **Every PAT-carrying call also gets `guards()`** — see the boxed rule below.
- `gitFixer.ts` — LLM tool-loop (single `run_git` tool) that recovers merge conflicts and independently verifies the tree is clean with no surviving markers — trusting nothing the model claims. Retries up to `codingAgent.maxFixAttempts` (unset ⇒ 3), bounded overall by `codingAgent.maxRunMinutes`. It holds **no credentials**: it resolves and commits locally, and the caller pushes afterwards via `syncAndPush`. Deliberate — `run_git` is a model-controlled shell running over conflict text that came from outside, which is the last place a PAT should be reachable. Also exports **`checkInWithFixer`** — see the boxed rules below.
- `github.ts` — `fetchCronJson` over the GitHub Contents API, ETag-conditional (304 = unchanged, free).
- `sweeper.ts` — boot + hourly TTL sweep of per-run working dirs (checkouts + pi scratch), keyed by mtime. Sweeps `work/`, `runs/` and `files/`; the queue's `pool/` is a sibling it deliberately does not touch (the tick owns those). The rule itself is `agent-core/scratchSweep.ts`, shared with the desktop's boot cleanup — **a pinned chat's dirs are never swept**, whatever their age, and `store.pinnedChatIds` supplies the exemption. A failed pinned lookup **skips the sweep**; an unreadable settings row does not (it just means the default TTL), because not knowing the TTL is a smaller thing than not knowing what's protected.
- `checkoutPool.ts` — stocks the warm-checkout queue (the *taking* is `claimWarmCheckout` in `git.ts`). See "Starting a new chat shouldn't begin with a download" below.
- `telegram/webhook.ts` — connect/disconnect/status + the webhook handler and out-of-band turn runner.
- `telegram/commands.ts` — the slash commands (`/help`, `/new`, `/chats`, `/chat n`, `/workspaces`, `/workspace n`, `/status`, `/btw`) + `BOT_COMMANDS` + `activeWorkspace` + `switchableChats`/`chatNotice` (the one numbered list, and the catching-up notice read off it — see below). Answer in-chat, run no turn.
- `telegram/btw.ts` — `/btw <question>`: one short model call over the chat's stored messages. Not a turn — it never steers, never joins the conversation, touches no files, and works WHILE a job is running (which is the point). It can see an in-flight job because messages are stored as pi completes each one, **and the step still in progress via `liveTool.current(chatId)`** — without that, asking "what is it doing" during the slowest tool call is precisely when the answer would stop at the last finished row. **The setup and the question are separate messages**: the role, the facts and the rendered conversation are the `systemPrompt`, and the user message is just `Here is the question: …`. One user turn carrying all of it left the instructions competing for attention with the transcript they were meant to be read against — and the role is stated as what the reader IS (an observer who has been handed a conversation and is about to be asked about it), not as a list of what it isn't. Tool rows stay summarised as `[ran bash]`: `/btw` answers about the shape of the work, not its contents. **`get_agent_secret`'s live output is named and never quoted** (`SECRET_OUTPUT_TOOLS`) — the result is a usable token and this answer goes out as a Telegram message. That is a name match on the tool, not a judgement: the output is never read into the prompt, so there is nothing for the model to decide. `list_agent_secrets` returns metadata only and is deliberately not on the list.
- `telegram/transcribe.ts` — a thin adapter over `agent-core/transcribe.ts` for voice notes: bytes in, text back out, nothing written to disk. One speech-to-text implementation, shared with the agent's `transcribe` tool and with the desktop mic, running whichever vendor `transcription.provider` names (**AssemblyAI, Deepgram or ElevenLabs**). It takes **the whole voice config, not a key** — which key applies depends on which vendor is selected for listening, and that is `agent-core/voiceProviders.ts`'s job rather than every caller's. It calls **`transcribeVoice`** — the sync path on AssemblyAI, the single pre-recorded request on Deepgram; see "Voice notes go to the sync API" below.
- `telegram/client.ts` — minimal Telegram Bot API client over `fetch` (one 429 retry). `sendMessage`/`editMessageText` take optional `entities`. It no longer owns a chunker: `splitMessage` was deleted when its last caller moved, because a boundary now has to cut the formatting spans as well as the text — see below.
- `telegram/markdownEntities.ts` — `toTelegram(md)` → `{ text, entities }`, plus `splitFormatted` / `truncateFormatted` / `clampEntities`. **The one place the agent's markdown becomes Telegram formatting** — see "The agent's markdown, rendered" below. Unit-tested by `tests/telegramEntities.test.js`.
- `telegram/stream.ts` — renders the agent event stream to Telegram (placeholder bubble — animated while it waits, per-tool line, in-place streamed text, authoritative final from `agent_end`). Starts no typing indicator — `runTurn` owns that.
- `telegram/waitingBubble.ts` — the animated `...` message itself, shared by `stream.ts` and the 🤬 reaction. See "The placeholder bubble is a SLOT" below.
- `tls.ts` — **the one owner of Traefik's dynamic config**: the self-signed certificate, `tls.yml` (what makes it Traefik's *default*, which is what a bare-IP request with no SNI resolves to) and `router.yml`, for both modes. `ensureSelfSignedCert` create-or-reuses the certificate for `configuredHost()` (= `COMPANION_HOST`) and **always** writes `tls.yml`; `writeRouter(domain)` writes the route; `readCertPem` is what `/telegram/connect` uses to hand Telegram a copy; `removeSelfSignedCert` deletes the certificate half when a domain is set. Driven once, at boot (`settleTls` in `server.ts`), where a failure is fatal — see the two boxed notes above. Was `telegram/selfSigned.ts`, which named half of what it did and sat under the wrong subsystem.
- `gitRemote.ts` — **pure** remote-URL policy (`remoteUrl`, `hasEmbeddedCredentials`), unit-tested by `tests/gitRemote.test.js`. Pins the "no credentials in the URL" property that `git.ts` depends on.
- `telegram/sendTool.ts` — `sendTelegramMessage(pool, key, text, opts)`, `speakInto(...)` (see "A long reply is spoken in pieces" below) and `sendTelegramFile(pool, key, path, kind)`: the one place a DM is actually sent (the bot token lives only here). `opts.output` picks text or voice and is set ONLY by callers that legitimately know (the reaction read-back speaks; cron delivers what the job asked for) — never by the agent, whose `send_message` has no such argument and whose HTTP route ignores one if an older desktop sends it. Unset ⇒ the stored mode (`speech.telegramReply`), which is the normal path — read here rather than passed in, so no caller has to know about it. **The save happens BEFORE the send**, so a send that fails halfway still leaves the preference the user asked for — they are independent requests either way. **The text always goes out, unconditionally, first**: voice is an addition and never a replacement, because a voice note can't be skimmed, searched or quoted, and a synthesis failure must never cost the user the answer. Callers — the `send_message` agent tool (`agent-core/sendMessage.ts`), `POST /telegram/send` (how the *desktop's* copy of that tool reaches it), and `cronRun.ts` for files a scheduled job produced.
- `telegram/attachments.ts` — **one line of policy: which directory.** `cacheAttachment(chatId, …)` is `writeAttachment(chatFilesDir(chatId), …)`. Everything else — what the file is, what to call it on disk, whether it can be inlined, the note the agent reads, and the write with its containment check — is **`agent-core/attachmentPolicy.ts`** (+ `attachmentNotes.ts`), shared with the desktop composer, which takes files from the user the same way. It lived here, at `telegram/attachmentPolicy.ts`, until the desktop grew that feature; a second copy would have been two answers to "is this a PDF or a picture", and that answer decides what the agent is told it is holding. Unit-tested by `tests/attachmentPolicy.test.js`, which now covers both hosts.
- `dataDirs.ts` — where per-chat working files live (`RUNS_BASE`, `FILES_BASE`, `WORK_BASE`, and the queue's `POOL_SETUP_BASE`/`POOL_READY_BASE`). Path math only — which is both why it can be split out of `agentHost.ts` (needing a directory doesn't drag in Postgres and pi) and what lets `git.ts` own the claim while `checkoutPool.ts` owns the stocking, without the two importing each other. `FILES_BASE/<chatId>` is the agent's **scratch pad**: inbound attachments land there, the agent may write there, delivery reads from there, and the prompt names it. Outside the checkout, so nothing in it is committed — which is why the TTL sweep exempting pinned chats matters most here: nothing in this directory exists anywhere else.

### The PAT runs git inside a directory the agent controls — `guards()` is what makes that safe

The checkout is the agent's own cwd for the turn, and `checkIn`/`syncAndPush` run git **in that same directory afterwards** with the PAT in the child's environment. So anything git can be made to execute at that moment can read the token. `guards()` in `git.ts` precedes **every** PAT-carrying call (`git()` applies it whenever `auth` is passed, and the `clone` in `prepareCheckout` passes it explicitly). Command-line `-c` beats repository config, which is the whole point — every value below is one the agent could otherwise set in `.git/config`:

| Guard | What it closes |
|---|---|
| `credential.helper=` (empty, **first**) | the setting is a LIST; assigning empty resets it, so a helper planted in the repo can't run ahead of ours |
| `credential.https://github.com.helper=…` | **host-scoped on purpose.** `url.<base>.insteadOf` rewrites the URL *after* `remote.origin.url` is pinned, and no `-c` can clear it (the subsection name is the agent's to choose) — so the request can leave for any host. Scoping means git asks that host's credentials and finds no helper. A bare `credential.helper` answered everyone, because the helper echoes the PAT without reading the host git hands it on stdin |
| `remote.origin.url=…` | left to the repo it can be `ext::sh -c …`, a command rather than an address. Pinning also keeps refs at `origin/<branch>` — a bare URL lands in `FETCH_HEAD` and quietly changes what the merge compares against |
| `core.hooksPath=/dev/null` | **not a directory.** Git looks up `<hooksPath>/<hookname>`; under the null device that is ENOTDIR, always. This used to name an empty directory under `WORK_BASE`, sitting beside the agent's own checkout and owned by the same user — empty only until the agent drops a file in. `--no-verify` covers `pre-push` alone, not `post-checkout` (clone) or `reference-transaction` (fetch) |
| `core.fsmonitor=` / `core.sshCommand=` | both name a command git runs |

`checkIn` and `syncAndPush` also pass `--no-verify` on `commit`/`merge`/`push`, so two independent things would have to be wrong for a planted hook to see the token. `prepareCheckout` additionally deletes `.git/hooks` on reuse — nothing it does to the worktree touches `.git`, so yesterday's hook would otherwise still be sitting there; the hooksPath guard already neuters it, this just stops it waiting for a call that forgets the guard.

**`gitFixer.ts` gets no credentials at all** and the push happens after it, in `syncAndPush`. Its `run_git` is a model-controlled shell running over conflict text that came from outside — the last place a PAT should be reachable.

`tests/gitGuards.test.js` pins all of this against **real git**: each attack is planted and an actual push is run, because the claim is "git does not execute the agent's code while holding the token", and only git can settle that. It also checks the helper still answers for github.com — without that, a scoping typo would break sync silently instead of failing a test.

### Every agent run checks in the SAME way — and there is no other way left

Three server-side agent paths — cron (`cronRun.ts`), Telegram (`telegram/webhook.ts`), and the two background kinds (`backgroundRun.ts`) — are one operation with different triggers. All call **`checkInWithFixer(dir, branch, message, auth, model, limits)`** (`gitFixer.ts`): stage and commit → push → on `'conflict'`, hand to `gitFix` → `syncAndPush` when it verifies clean.

**`git.ts` used to export `checkIn`, and it does not any more.** That function did the whole add → commit → fetch → merge → push but stopped at a conflict, which made it a second, incomplete way to save a run's work. Both were exported, both read as the answer, and picking the wrong one committed the work without ever pushing it while reporting nothing.

Which is exactly what happened. Telegram ran `checkIn(...).catch(() => {})` — no fixer, and the result discarded. A conflict left the turn's work committed-but-unpushed in the run's checkout, said nothing in chat, and the **next** message's `prepareCheckout` `reset --hard`'d it away: the failure and its evidence vanishing together, looking identical to a turn that saved fine.

The fix is not a rule saying "call the other one" — a rule in a document is a thing to remember, and the whole failure mode was somebody not remembering. `git.ts` now exports only primitives that promise exactly what they do:

| | does | does not |
|---|---|---|
| `commitAll(dir, message)` | stages everything, commits, returns false if there was nothing to commit | push |
| `syncAndPush(dir, branch, auth)` | fetches, merges, pushes | commit |

Neither can be mistaken for saving a run's work, because neither claims to. `checkInWithFixer` is the only place they are composed into a landing, and it is therefore the only thing there is to call. **A fourth agent path cannot get this wrong** — not because it is told not to, but because the wrong function no longer exists.

### A fresh checkout clones at depth 1, then reaches back 7 days

`cloneFresh` runs two commands, and the split is the point:

```
git clone --depth=1 …                    always succeeds, fast
git fetch --shallow-since=7.days …       best-effort, failure costs only the history
```

A depth-1 checkout leaves the agent holding a repo whose history starts today — `git log` shows one commit, `git blame` blames all of it on that commit, and *"when did this change and why"* has no answer. An unattended run cannot ask for more either: `guards()` are command-line `-c` and **do not persist into the clone's config**, so the checkout carries no credential helper and any fetch the *agent* starts against a private origin fails with nothing to authenticate with. Whatever history it is going to have, it has to be given at clone time.

**The window cannot be a `--shallow-since` on the clone itself.** A repo with no commits inside it fails the clone outright — `fatal: error processing shallow info: 4`, no directory created — so a workspace nobody touched for a week would stop cloning entirely and every run against it would break. As a *second* step the identical error is harmless: the clone has already succeeded, so the failure costs the extra history and nothing else, and what remains is exactly the depth-1 checkout that shipped before. Hence the bare catch — there is no state to repair.

`HISTORY_WINDOW` is one constant in `git.ts`. Two properties worth knowing:

- **One edit covers both paths.** `cloneFresh` has exactly two callers — the inline clone in `prepareCheckout` and the queue's stocking in `checkoutPool.ts` — which is why the deepen lives there and not in `prepareCheckout`.
- **The boundary is cut at clone time and nothing moves it.** The reuse fetch and `refreshPristine` both carry no depth, so they preserve it — meaning a queued folder's window counts from when it was **stocked**, not from when a chat claimed it. Bounded rather than sliding: the pool refreshes hourly and folders age out on `scratchTtlDays`.

Blame and old file contents beyond the window are still unreachable, and deliberately so — closing that needs a privileged deepen the agent can *request*, not a credential it holds.

### Reusing a checkout: `fetch` (no `--depth`) then a GUARDED `reset --hard`

**`--depth=1` belongs on the clone and NOWHERE else.** On the initial clone it is the whole saving. On a fetch into an existing checkout it saves nothing — a fetch only ever transfers objects we don't already have (measured: 3, for a one-file change in a 200-file repo) — and it rewrites `.git/shallow` so the remote branch arrives as its own root commit with no link to what we hold.

That flag was on the reuse fetch, and it broke every reused checkout:

```
reuse:   fetch --depth=1 + merge --ff-only   → fatal: refusing to merge unrelated histories  (swallowed)
         the agent then reads a stale tree
checkIn: behind=1 → merge                    → fatal: refusing to merge unrelated histories
         no unmerged files → "pushing anyway" → push rejected, non-fast-forward
         3 retries, all identical             → 'conflict'
```

The user is told the save failed while the work sits in the folder. It fires whenever anything else pushes between two turns of the same chat — the desktop syncing, another chat, a cron job. A checkout that only ever pushes its own commits stays healthy, which is why it wasn't constant.

**A folder already grafted by the old code is not repaired** — the break is written into the repository, and only `git fetch --unshallow` can undo it. That is deliberately not done: those folders age out on `scratchTtlDays` (unset ⇒ 7 days, and never for a pinned chat), so the condition disappears on its own, and carrying migration code in a permanent path to cover it costs more than it saves. A folder in that state keeps failing its push until it is removed.

With a connected history, *"is there anything to lose?"* has a real answer — so `reset --hard` is both safe and the operation actually wanted: one round trip, nothing that can half-succeed, an exact match with the remote.

| state | outcome |
|---|---|
| clean, already current | reset is a no-op |
| clean, behind | lands exactly on the remote |
| local unpushed commits | `nothingToLose` refuses, folder untouched |
| dirty tree | `nothingToLose` refuses, folder untouched |

**The guard is the whole difference** from the unconditional `reset --hard` + `clean -fd` that was removed here earlier — that one deleted work which hadn't reached GitHub, because a turn's changes are only safe once pushed and the push happens *after* the agent has replied. The objection recorded at the time was that shallow history makes ancestry unresolvable so the question can be answered wrong. **That was true of the code, not of git**: it was unresolvable *because* the fetch threw the link away. `nothingToLose` returns false on any error, so "I couldn't tell" still never licenses a wipe.

Nothing the guard declines to fold in is stranded: the turn's own `git add -A` sweeps it into the next commit, and `checkIn` reconciles with the remote at the end.

**The accepted cost is two agents briefly sharing one folder** — a confusing commit, or git refusing a concurrent operation. Both are loud and recoverable, which a deleted file is not. That trade is deliberate: the fixer's prompt tells it files may appear mid-resolution and to fold them in. Pinned by `tests/checkoutReuse.test.js` against real git — **which clones shallow**, because it used to clone full and therefore passed throughout the entire life of the bug.

### `'diverged'` is not `'conflict'` — the fixer can only fix one of them

`syncAndPush` returns `'conflict'` when the merge left markers in the tree: something `gitFix` can work on. It returns **`'diverged'`** when the merge could not START (no common history, or it would clobber local changes). Nothing is conflicted then, so the fixer's `verify` — clean tree, no markers — passes the moment it arrives, it reports success without doing anything, and `checkInWithFixer` retries the same doomed push. One status for both is how that hid; `checkInWithFixer` hands off only on `'conflict'`.

**Ask `landed(result)`, never `=== 'conflict' || === 'error'`.** That spelled-out list lived in three files, and a missed one reads a failure as a success and says nothing.

### Starting a new chat shouldn't begin with a download (`checkoutPool.ts`)

A chat's first message used to pay for a full clone while the user waited. The queue keeps one cloned ahead of time, and **a folder's LOCATION is its state**:

```
pool/setup/<owner>__<repo>__<branch>__<uuid>   being cloned — never read
pool/ready/<owner>__<repo>__<branch>__<uuid>   complete, usable
work/<chatId>                                  claimed; a chat owns it
```

Every move is a rename, and always forward. No marker file, no status column, nothing that can disagree with the disk — a clone that dies halfway is stranded in `setup/` and *cannot* be mistaken for usable, because being in `ready/` is what usable means. Renames are also what make it safe without a lock: `rename` is atomic within one filesystem, so two chats claiming at once cannot get the same folder — one wins, the other gets ENOENT and takes the next or clones. **All three directories live under `DATA_BASE` for exactly this reason**; across filesystems `rename` fails and the property is gone.

**Claiming is the only thing a turn does, and it has no side effects.** Restocking, refreshing and cleaning are one per-minute tick that reconciles the directory to the target, so a turn is never slowed by maintenance and the queue can be reasoned about on its own. Every failure path degrades to a normal clone.

**Every companion chat claims — Telegram and cron alike.** `prepareCheckout` calls `claimWarmCheckout` unconditionally, so there is one way to obtain a checkout rather than a fast path for some callers and a slow one for others. That is why **the claim lives in `git.ts`, not here**: it is a rename, and putting it there lets `prepareCheckout` call it without importing this module, which is built on `git.ts`'s clone and refresh. The path constants live in `dataDirs.ts` (path math, no dependencies) so nothing has to import in a circle. This module only stocks.

Cron consuming a slot was the argument for keeping it Telegram-only — a job at 09:00 takes the folder a person wants at 09:01. Real, but small: the tick restocks every minute and cron fires nowhere near that often, so two spares covers both, and what it buys is one code path.

**The queue holds ONE repo** — whatever Telegram is pointed at. A cron job on a different workspace finds no match and clones, exactly as before. Slots are already keyed by `owner__repo__branch`, so holding several repos later is widening a loop, not a redesign.

**One setting: `codingAgent.checkoutPoolSize`** (unset ⇒ 2, 0 disables), on Settings → Agent. The refresh window is a constant, deliberately: the claim always fetches, so it can only change how much that fetch pulls and never whether the result is correct — a knob that cannot affect an outcome is a knob to explain and get wrong. Refreshing a queued folder uses `refreshPristine`, an *unguarded* `reset --hard`, legitimate only because these folders have never been worked in; anything a user or agent has touched goes through `prepareCheckout`, which asks first. Claim semantics are pinned by `tests/checkoutPool.test.js`.

### The fixer is bounded by time and attempts, not tool calls

`gitFix` loops up to `codingAgent.maxFixAttempts` (unset ⇒ 3), re-running `verify` between attempts, with **one deadline across the whole loop** from `codingAgent.maxRunMinutes` — so the number means what it says however many attempts it takes.

Retrying works because **the folder persists between attempts and only the agent's memory doesn't**: attempt 2 opens a repo where whatever attempt 1 resolved is already resolved and committed. Three conflicted files, two attempts — the first clears A and B, the second sees only C. The other reason verification fails is the concurrency above: a second turn writing into the folder while the fixer works, where the fixer was right and the tree simply moved afterwards.

It used to abort the session at 12 tool calls. Resolving *one* conflicted file costs roughly seven, so that cap mostly severed legitimate work partway and reported it as a failure. **The fixer runs after the turn has already replied, so nothing user-facing waits on it** and there is no reason to hurry it. The time bound covers the only real hazard — a model looping on an unattended server.

## HTTP API (`server.ts`)

**Public (no bearer):**
- `GET /health` — `SELECT 1`; 200/503, plus `version` (the image's release tag; `'dev'` locally). Registered before auth.
- `POST /telegram/webhook` — Telegram inbound. Auth is the per-account `X-Telegram-Bot-Api-Secret-Token` header, checked **inside** `handleWebhook`. Registered before the bearer middleware, with its own JSON parser.

**Auth:** `authed` middleware compares `Bearer <token>` SHA-256 against the stored `API_KEY` hash with `timingSafeEqual` (401 otherwise). `app.use(authed, limiter, express.json())` protects + rate-limits (600/60s) + parses everything below.

> **The JSON body limit is 64mb, and 1mb was silently breaking things.** Two routes legitimately carry media and both failed as a 413 nobody saw: `PATCH /chat/:id/transcript` sends pi's **entire** session JSONL, re-uploaded every turn, which passes a megabyte on any chat with an image or merely enough tool output; and `POST /chat/:id/messages` carries a message's images base64 (+33%) on the row. Telegram already refuses anything over 20MB (`MAX_INBOUND_BYTES`), so that bounds the largest single file; the ceiling sits well above it because a transcript accumulates. What this widens is one bearer-authed, rate-limited server with a single user.

**Protected:** `POST /update` (remote upgrade — see above); `GET/PATCH /settings`, **`DELETE /settings/credential/:path`** (the only thing that removes a stored settings credential — the path is re-validated here against `agent-core/credentials.ts`, since this route is reachable with the bearer key alone); `GET /agent-secrets`, **`DELETE /agent-secret/:name`** (the only thing that removes one), `GET /agent-secret/:name/token` (mint), `POST /oauth/:name`; chats (`GET /chats`, `/chats/pinned`, **`/chats/pinned-ids`** (just the ids — the TTL sweep's exemption list, on both hosts), `/chats/search`, `/chat/:id`(+`/messages` — `?after=<seq>` for just the newer ones, `/transcript`, `/running`), `POST /chat`, `POST /chat/:id/messages`, `PATCH /chat/:id/{title,pinned}`, `DELETE /chat/:id`); the **`search_chats` trio** — `GET /chats/fulltext`, `GET /chat/:id/window`, `GET /chats/recent` — which is how the DESKTOP's copy of that tool reaches this data (the companion's own runtime calls the same `store.ts` functions in-process; see `chatSearch` in `agent-core/CLAUDE.md`), one route per shape the tool takes: search, read a window around a hit, list recent; **`GET /attachment/:id`** (one chat image — the only route that answers **raw bytes**, so it deliberately bypasses the `handle()` wrapper, which would wrap it in `{result}` JSON; ids are fresh uuids and the content never changes, so it's served `immutable` and a re-opened chat re-downloads nothing); live feed (**`GET /events`** — one SSE stream for every chat; `POST /chat/:id/events` for a client relaying its own local run); cron (`POST /workspace/:id/cron/:job/run`, `GET /workspace/:id/cron/state`); workspaces (`GET/POST/PATCH /workspaces`, `DELETE /workspaces/:id`); telegram (`POST /telegram/{connect,disconnect,send,workspace}`, `GET /telegram/status` — `send` backs the desktop's `send_message` tool: `{text}` in, `{ok}`/`{ok:false,error}` out; `workspace` sets the bot's active workspace from the desktop's Telegram settings page, same start-a-fresh-chat semantics as `/workspace` in the bot; `status` includes the resolved `workspaceId`/`workspaceName`). The `handle()` wrapper returns `{result}` and never leaks error detail (`500 {error:'request failed'}`).

## Data model (`schema.ts` / `init.sql`)

- **workspace** — identity = a GitHub repo: `id`, `name`, `repo_owner`, `repo_name`, `default_branch`, `sort_order`. (Checkout path / active / sync-toggle are machine-local — they live on the desktop, not here. `voice_reply` used to be here and is **dropped** by `init.sql` after seeding `speech.telegramReply` — see "Replies can come back as a voice note" below.)
- **setting** — non-secret scalar settings, one row per dotted leaf key: `key`, `value`, `type` (`string|number|boolean|json`), `updated_at`.
- **agent_secret** — agent-secret entity metadata (no crypto columns): `name`, `description`, `kind` (`static|oauth`), the `oauth_*` columns, timestamps.
- **secret_value** — **every** encrypted value: PK `(owner, field)`, `ciphertext` (base64), `iv`+`tag` (`bytea`, `NOT NULL`), `key_version`, `updated_at`. `owner` ∈ {`settings`, `telegram`, an `agent_secret.name`}.
- **chat** / **message** — chats: session metadata + `source` (`desktop|cron|telegram|review|memory`)/`source_id`/`machine` provenance + `running`/`running_machine` cross-client flag + `last_reviewed_seq` (how far review has looked — see below) + `checked_in_at` (when its work last finished being pushed — see "A chat is not reviewable until its work has landed"), plus the whole pi JSONL in a **`transcript` column** (it was a 1:1 `chat_transcript` side table, which bought nothing — Postgres TOASTs a big text column out of line and never reads it unless selected). `message` holds one row per pi session ENTRY, appended as pi completes it; identity is `entry_id` (pi's own id), and `seq` is an ordering/read cursor **assigned by the server**. It also carries a GENERATED `search_text` tsvector (user+assistant content only — tool output is deliberately unindexed) with a GIN index, backing the agent's `search_chats` tool.
- **attachment** — images the user sent with a message: `id`, `chat_id`, `entry_id`, `idx`, `mime_type`, `bytes`, `created_at`. **This is what the chat UI draws.** Keyed by `entry_id` (the message's identity) and NOT `seq`, which the server assigns afterwards and the writer therefore doesn't know. Inserted in the same transaction as the message and **only when that message actually inserted**, so a retried or re-sent turn can't duplicate its pictures. The bytes also live inside `chat.transcript` — that's pi's own session file, stored whole and never parsed; this table is our copy in a shape that can be served one image at a time.
- **telegram_account** — single row (`id='default'`): authorized user, dm chat id, active chat, **active workspace** (switchable via `/workspace`; falls back to the first workspace by `sort_order` — the top of the desktop's list), `last_update_id` (dedup), bot username, enabled. Token + webhook secret are encrypted in `secret_value` under owner `telegram`.
- **telegram_sent** — what each bot bubble said and which chat said it, keyed by Telegram's own message number: PK `(chat_id, message_id)` (Telegram's chat, i.e. the DM), `content`, `origin_chat_id` (one of *ours*, nullable), `created_at`. A reaction update and a reply update both name only that number, so this is what makes either one answerable — see "Pointing at a bubble" above. Never *expired*: a row is the only link between a message still sitting in the user's Telegram history and the chat that produced it, so a TTL can only make a gesture stop working on a bubble that is still on screen. The one deletion is `deleteTelegramSent`, fired from the client's `onDeleted` hook — a message the bot deleted itself (a waiting bubble) can never be pointed at again, so its row describes something that is not there.
- **cron_state** — run **history** only, PK `(workspace_id, job_name)`: `last_run_at`/`last_error`/`last_chat_id`. Next-run is computed in memory by croner, never persisted.

## Settings + secrets

`readSettings` builds the object from `setting` rows (decoded by `type`), splices decrypted `secret_value` rows owned by `settings` at their field paths, and attaches `agentSecrets` + `workspaces`. **No defaults are applied — it returns exactly what is stored.** (The desktop merged defaults on read and faked unset values; that is gone. See "Defaults" below.)

`writeSettings` flattens a patch to dotted leaf keys and, in one transaction, routes each via `isSettingsSecretKey` → `putSecret(owner='settings', field=key)` or an upserted `setting` row, then reconciles `codingAgent.providerKeys` and `agentSecrets`. `putSecret` seals + upserts on `(owner, field)`; **an empty value is a no-op** — see the boxed rule below.

### Destroying a credential is a REQUEST, never an inference

`putSecret` used to delete the row when handed an empty value, and `writeAgentSecrets` used to delete any name missing from the incoming list. Both read as reasonable ("absent = unset") and between them they were **four** ways to silently lose a key, because *empty* and *missing* arrive constantly for reasons that have nothing to do with wanting rid of the thing:

| What arrived | Why | What it destroyed |
|---|---|---|
| a renamed agent secret | `secret_value.owner` **is** the secret's name, so a new name read as a new entity and the old one was deleted for being absent | the token — or, for OAuth, the client secret and both tokens, while the row still displayed as `connected` |
| an empty credential | the desktop never *receives* credential values, so everything it holds reads as empty | whatever the save happened to mention |
| an undecryptable row | `unseal` returns `''` on failure so one bad row can't fail a whole read — laundering "I can't read this" into "delete it" through any read-modify-write | every affected key, on the next launch that wrote the list back |
| a stale list | the caller's copy is legitimately behind (another machine added one, the live feed dropped, an offline boot seeded `[]`) | everything the caller hadn't heard of |

The rule now: **only a function whose name says "delete" deletes.** `dropSecret` is the primitive; `deleteSettingsCredential` and `deleteAgentSecret` are the two callers, each behind its own route (`DELETE /settings/credential/:path`, `DELETE /agent-secret/:name`). `tests/secretDeletes.test.js` scans this file and fails if a delete appears anywhere else — the queries need a live Postgres, so a source scan is the only tripwire available, and it was verified by reintroducing the bug and watching it fire.

**One exception, named explicitly in the code:** `patchOAuth` still clears a credential from an empty value, because disconnecting genuinely sends `accessToken: ''` and means it. It is a targeted write from the OAuth flow, not a bulk save.

**A rename is a re-file.** `writeAgentSecrets` reads `previousName` off an entry (set only by the save that renames it — see `AgentSecret` in `src/shared/settings.ts`) and moves the `agent_secret` row and every `secret_value` row to the new owner. Nothing else can: the desktop holds no copy of the credential to resend, which is exactly why its empty token box means *keep what is stored*. The marker is tolerated when stale (a re-sent list) and ignored when the target already exists. It also covers a case-only change, which the name box can produce by itself by upper-casing what it is handed.

**Encryption (`crypto.ts`):** AES-256-GCM under the single `MASTER_KEY`. Fresh 12-byte IV per write; `iv`/`tag` are `NOT NULL` in `secret_value`, so a plaintext credential is structurally unrepresentable. `unseal` returns `''` on failure so one bad row can't fail a whole read.

**Routing policy (`keys.ts`, derived from `agent-core/credentials.ts`):** `SETTINGS_SECRET_PATTERNS` = `codingAgent.providerKeys.<slug>`, `voiceKeys.<vendor>`, `sync.pat` (owner `settings`). The two wildcard MAPS (`codingAgent.providerKeys`, `voiceKeys`) cannot travel as ordinary flattened leaves — a map arrives carrying only the slot just edited and the server MERGES it — so `writeSettings` lifts each one out of the patch by `WILDCARD_MAP_PATHS` and hands it to `reconcileCredentialMap`. **One function for both**, because the merge rule is a property of the shape rather than of which map it is; a per-map copy is how one starts deleting rows the other owns. `AGENT_SECRET_FIELDS` = `token`, `oauth.{clientSecret,accessToken,refreshToken}` (owner = the secret's `name`). `OAUTH_OWNED_FIELDS` (`oauth.accessToken`/`refreshToken`) are written **only** by the OAuth flow — a bulk `writeSettings` can't author them, so a client echoing pre-refresh state can't clobber a token the server just rotated.

### `secret_value` is shared — reconciliation must be SCOPED

`secret_value` holds three owner kinds: `settings`, `telegram`, and each agent-secret `name`. **Anything that deletes must be scoped to one owner** — never "everything not in this list", and now never a reconcile at all:

- **`deleteAgentSecret`** deletes strictly `agent_secret`/`secret_value WHERE owner = <name>`, for the one name asked for. Two earlier versions of this got it wrong in opposite directions: one deleted every `secret_value` row except a hardcoded safelist, clobbering the `telegram` `botToken`/`webhookSecret` on every settings save and breaking the bot; the next scoped correctly but still fired on *absence from a saved list*, which made a stale list a data-loss event. **Never reintroduce a "delete all owners except X", and never delete on absence.**
- **`reconcileCredentialMap`** merges the slots present and deletes nothing at all. Removing one wildcard slot is `DELETE /settings/credential/:path`, confined to owner `settings` and validated against `isDeletableCredential`.

## Defaults — there are none on read

The companion applies **no** defaults. A setting is set (a row exists) or unset. Consumers handle it in two ways:

- **Required** — no default; error if unset. `sync.pat` → `cronRun.ts` throws, `webhook.ts` replies in-chat, `scheduler.ts` skips. `codingAgent.provider`/`model` + the provider API key are read straight through; `agent-core` errors if empty ("provider not configured").
- **Optional** — fall back **at the point of use** in the consumer, not on read: `timezone → 'UTC'` and `thinkingLevel → 'off'` in both `cronRun.ts` and `telegram/webhook.ts` (and `scheduler.ts` for timezone).

Do **not** add a defaults object or seed default rows. The desktop learned this the hard way — a client-side default layer made an unset value look configured while the server (reading the DB directly) saw the hole and failed.

## Telegram

**Commands** (`telegram/commands.ts`, registered with `setMyCommands`): `/help` (what the bot is + every command), `/new`, `/chats` + `/chat n`, `/workspaces` + `/workspace n`, `/status` (workspace, chat, busy or idle, model), `/btw <question>`. Switching workspace starts a fresh chat — a chat belongs to one workspace. Which workspace Telegram runs against is `telegram_account.active_workspace_id` — set by `/workspace n` in the bot or the desktop's Telegram settings page (`POST /telegram/workspace`) — falling back to the first workspace by `sort_order` (the top of the desktop's workspace list). There is no env var for this; an earlier `TELEGRAM_DEFAULT_WORKSPACE` env fallback is gone. The `/new`, `/chat n`, `/workspace n`, and `/status` replies all name the workspace, so the user knows where the work lands.

**One numbered list — `switchableChats`.** `/chats`, `/chat n` and the catching-up notice all read positions off the same array, so they cannot disagree about what "3" means. It is `[…up to 10 recent, …up to 3 pinned]`, each half `updated_at` desc, with the **active chat excluded from both** (it prints above the list as "Current chat", and a number that switches you to where you already are is a broken row). `listChats` filters pinned out at the query, so the halves never hold the same chat twice. **The recent half is `desktop` + `telegram` only** — chats a person started. The server opens chats of its own (cron, review, memory) and those are work that happened rather than a conversation to carry on; there are enough of them to push every chat you actually had off a ten-row list. **The pinned half is unfiltered**, whatever opened it: pinning is an explicit act, so a pinned cron chat was pinned on purpose, and that outranks where the chat came from. The narrowing is a `sources` option on `listChats` and therefore happens **in SQL** — filtering after the fetch would let a burst of background chats fill the page and leave the list short while older desktop chats existed. A null `source` reads as `desktop` (the default `upsertChat` writes), so rows predating the column stay visible. The desktop's `GET /chats` passes no `sources` and is unchanged. **Pinned sit below recent** on purpose: numbering is continuous across the sections, and the notice only ever lists recent chats, so keeping those at 1..10 is worth more than starting pinned at 1. Sections are headings only — an empty one is dropped and the numbering doesn't move.

**Catching up.** The bot answers in whichever chat it was last left in, and `telegram_account.active_chat_id` is sticky forever — so a message sent after a week away lands in a week-old conversation with nothing on screen to say so. Before the turn, `chatNotice` sends `N new chats since we last talked:` with up to 3 numbered rows and `/chat <number> if you want to switch`. **"New" means `created_at`, not `updated_at`** — a chat is new when it came into existence, and an old conversation someone replied to is recent rather than new; listing that one sends you looking for something you have already seen. The two timestamps do different jobs here and both are load-bearing: the **staleness** check is `updated_at` (a chat opened three weeks ago but used yesterday is not stale), the **newness** filter is `created_at > ` the active chat's `updated_at` (since we last talked *in this chat*), and the age printed on each row is `updated_at` — the same rows with the same numbers appear in `/chats`, so a timestamp meaning one thing there and another here is the confusion the message exists to prevent, and last-activity is what says which of them is live. It returns null (and nothing is sent) unless the active chat's `updated_at` is older than the window AND something newer exists — and never on a command, on an already-running chat, on a chat that doesn't exist yet, or on a reply-switch, which just said which chat this is a different way. **It self-limits without any stored flag**: running the turn bumps `updated_at`, so the second message of a session can't retrigger it. The count in the header is the count of rows printed, so there is nothing for the reader to reconcile; pinned chats never appear (they're deliberate and long-lived — not what you drifted away from) though they still hold numbers. Settings are `telegram.chatNotice` (`enabled`, `afterHours`, `limit`), declared with their defaults in **`agent-core/chatNotice.ts`** because the desktop's Telegram page renders the same numbers — see `tests/chatNotice.test.js`. Started in parallel with the checkout and awaited just before `runTurnInner`, so the send overlaps the clone but can never land after the reply it introduces.

**Attachments in.** Any file — photo, document, video, album — is downloaded to the chat's staging dir (`FILES_BASE/<chatId>`, outside the checkout, TTL-swept) and described to the agent by a bracketed note giving the path and telling it to **act**, using the same policy module the desktop composer runs (`agent-core/attachmentPolicy.ts`), asking only when the intent is genuinely unclear. That imperative wording is copied from hermes-agent and is the whole reason a path pointer works; passive wording there made the model reply "what would you like me to do with this?" to a message that already said. Small text files are inlined instead, gated on the **extension** — never on whether the bytes decode, since PDF/zip/docx all start with decodable ASCII. Images are additionally attached as pi `ImageContent` when the configured model lists `image` input (`modelCatalog`); the note says so when it can't, rather than letting the agent claim it looked. **An image's type comes from its magic bytes**, not the filename or the sender's `mime_type` — a Telegram photo has neither, and providers reject a declared type that doesn't match the bytes, failing the whole turn. `msg.caption` is read as the prompt (without it a file arrives with no instructions), and albums are debounced 800ms into one message, because Telegram sends each item as its own update and the second would otherwise interrupt the first.

**Files out.** The agent names a path in its reply — `MEDIA:/abs/path` or bare — and `agent-core/mediaTags.ts` extracts it, strips it from the visible text, and the file is sent. Delivery reads from exactly two folders (the checkout and the chat's staging dir) with symlinks resolved first, which replaces hermes' hand-maintained denylist of credential paths. Telegram's path is `stream.ts` (which also strips tags from the live-edited message, or the raw tag is visible while it streams); cron's is `cronRun.ts`, since a scheduled run posts no reply to attach a file to. **Both scan text accumulated from this turn's deltas, never `agent_end.messages`** — that carries pi's whole session, so scanning it re-sends a file on every later turn. Desktop does neither, and `agent-core/defaults/companion.ts` keeps the syntax out of the desktop prompt so the agent can't promise a delivery that won't happen.

**Voice notes**: a message with no text but `voice` is downloaded (declined over 20 MB — Telegram's bot ceiling), transcribed, then run as the prompt. No key configured → says so rather than ignoring the message. **`audio` is NOT transcribed** — an audio file is a file and goes through the attachment path above. This read `voice ?? audio` and transcribed both, so sending an mp3 made its whole transcript the prompt. `resolveInput` reads settings **once** and passes the whole voice config into `transcribeAudio` (which holds no store import of its own), because it needs the neighbouring `echoTelegramTranscript` from the same object: when that is on, the transcript is posted back as `🎤 "…"` before the turn runs, so a misheard word is distinguishable from a misunderstood instruction. **Default off** — `?? false` at the point of use, since the companion stores no defaults. The desktop toggles it at Settings → Telegram.

### Voice notes go to the sync API, and that is why ffmpeg is on the hot path

A recording and a voice note want different APIs. `transcribeFile` (the agent's `transcribe` tool) submits an async **job** and polls it, which is right for an hour of audio with several people in it. A voice note is one person, seconds long, and its transcript is thrown away the moment it becomes a prompt — but it was paying the job API's price, and **the SDK's polling interval is 3 seconds**, so a clip transcribed in under a second was still discovered on the next tick.

`transcribeVoice` uses AssemblyAI's **sync** endpoint instead: one request, transcript in the response, nothing to poll. Measured on the same six-second clip through the shipped module: **~0.25s against ~3.5s**. The saving is roughly constant rather than proportional, because what it removes is the tick.

**The sync API does not accept OGG/Opus, which is what Telegram sends.** The SDK labels every non-PCM multipart as `audio/wav` and the server trusts that label rather than sniffing, so an Ogg page comes back `malformed WAV: file does not start with RIFF id`. So the audio is decoded to **raw PCM** first (16 kHz mono, piped through ffmpeg in both directions, ~18ms). PCM and not WAV on purpose: a WAV written to a pipe carries a placeholder RIFF length because ffmpeg cannot seek back to fix it — the same class of malformed header that was just rejected — while headerless PCM has nothing to get wrong. This is why **ffmpeg is now required for every voice note**, not only for video (it is in the image; `api/Dockerfile` installs it).

**The job API remains the fallback and nothing removes it.** A clip over the sync ceiling (**2 minutes**, decided from Telegram's own exact `voice.duration`), a missing ffmpeg, or any sync error all fall through to the path that has always worked, with a `console.warn` so a permanently broken fast path can't hide behind a working slow one. A voice note *is* the message, so failing one loses what the user said.

### Replies can come back as a voice note

You can ask for spoken replies. **Three modes, not a scale**: `text` (the default), `voice` (audio only), `both`. The mode is **one `setting` row, `speech.telegramReply`** (`VOICE_REPLY_SETTING_PATH` in `agent-core/voiceReply.ts`) — not in the checkout, because `/voice` is a slash command and those are answered from this database with no checkout prepared; a file there would have made changing a preference cost a clone.

> **It lived on `workspace.voice_reply` until 2026-08-06, and being per-workspace was the bug.** The bot answers in one workspace at a time (`telegram_account.active_workspace_id`) and that is chosen independently of whichever workspace the desktop has open — so the settings page wrote one workspace's mode while the bot read another's, and the replies stayed text with nothing on screen able to explain it. The reason it sat on that row was never scope, only *not being in the checkout*, which a settings row satisfies identically. `init.sql` seeds the new row from the workspace the bot was actually obeying and then drops the column; N values collapsing to one is inherent, and a workspace left on `text` seeds nothing, since `text` is what unset already means. Note the downgrade hazard: an older companion image selects a column that no longer exists.
 It reaches `makeTelegramSink` as two independent options — a `speak` callback and `textToo` — so `stream.ts` stays a renderer of events with no store, no settings and no idea what a vendor is. `sendsText` / `speaks` in `agent-core/voiceReply.ts` are the one place the three modes are turned into those two booleans.

Three rules, each of which is the reason something isn't broken:

- **Speaking never precedes the text, and never blocks on it.** Synthesis is a network call that can fail, so the words are rendered first and the audio follows, on the CLEANED text (file tags cut, so a path is never read aloud mid-sentence).
- **Voice-only never writes the answer at all.** Progress still shows — the placeholder bubble, a line per tool call, and the chat action switching to `record_voice` while the audio is made — so the chat is not empty. What it does NOT do is type the reply out and then delete it, which is what it did first: you watched the whole answer appear and vanish. If synthesis fails the text is written after all, here and in `sendTelegramMessage` both: a mode preference is not worth delivering nothing over.
- **The chat action is owned by the typing loop**, which can be told to say something else (`typing.set('record_voice')`). Telegram expires an action after ~5s so it is re-sent on a timer — a one-off sent from elsewhere would be painted over by the next tick.
- **The mode is read at the START of the turn.** The agent can change it mid-turn via `send_message(save: true)`; reading at the end would apply a request to *start* speaking retroactively to the reply already composed as text.
- **A script longer than the vendor's input limit is CUT, not rejected.** Over the limit the request fails outright, which would lose the audio entirely; cutting costs the listener the tail of a long answer and costs the reader nothing, because the full text is already there. hermes does the same.

`speakToFile` never throws (see `agent-core/speak.ts`) and `speakInto` logs and drops everything else, so a workspace with speaking switched on but no key configured simply behaves like one on text.

### A long reply is spoken in pieces

Every byte of a voice note is synthesised before a single one is sent — one vendor request, all the audio, an ffmpeg conversion when the format needs it, then the upload. So a long answer is a long silence followed by everything at once, and the wait grows with the length of what is being read.

`speakInto` splits the script (`agent-core/speechChunks.ts`) and sends a bubble per piece, keeping **exactly two in flight**. Two is the vendors' number, not ours — Deepgram's REST text-to-speech allows 2 concurrent requests and ElevenLabs allows 2 on its free plan — so firing every piece at once takes 429s on the ones it hurts most to lose, the end of the answer. But one at a time is just as wrong: the second piece cannot start until the first has gone out, so its whole budget is the first piece's playing time, and the opener has to be long enough to cover it. With one always cooking, the second is made ALONGSIDE the first, which is what lets the opener be short — and short is the only thing that makes the first sound arrive quickly, since synthesis is linear in length with no fixed cost to amortise. Measured end to end on a real reply: **first sound at 1.26s** (2.7s one-at-a-time), then 554ms, 2515ms and 2366ms of audio still buffered as each later piece lands — no silence anywhere. Under ~20 seconds of speech nothing changes, one voice note as before.

Four rules:

- **The opener's size is decided by the delivery, not by taste.** One at a time it is the whole budget for the second piece, so shrinking it starves everything after it — measured, a 27-character opener drove every following piece to the minimum, a stutter of tiny clips. With two in flight that coupling is gone and the same 27 characters are the right answer: first sound in 1.26s and the second piece already waiting.
- **The dots come back BETWEEN pieces, and deliberately not after the last one.** A gap with no bubble is how you know the answer has finished rather than stalled. `speakInto` owns those; the FIRST piece's wait belongs to the caller, which already has something on screen (the reaction's bubble, the turn's placeholder), and `onFirstSent` takes it down once there is audio to replace it. Two waiting displays at once is one too many.
- **Only the FIRST piece is a reply** to the message being answered. The rest follow it in sequence and are plainly part of the same reading; five voice notes pointing at one message is noise, and Telegram draws a quoted copy of that message above every one of them.
- **The dots go up before the audio and come down after it**, never the other way round — deleting first leaves a moment with nothing on screen at all, which is the gap they exist to fill.
- **It fixes the silent truncation.** `speakToFile` CUTS a script over the vendor's input limit, because the full text is normally delivered alongside. That is exactly wrong for a voice-only workspace and for a read-back, where the audio *is* the delivery — the tail was simply never spoken and nothing said so. The split treats the limit as a hard cap and keeps going instead.
- **A failure halfway says so** and reports `partial`, which every caller treats as a failure: the voice-only paths write the full text after all, since the part that was never spoken exists nowhere else. That is why `speakInto` returns `'none' | 'partial' | 'all'` and not a boolean — the middle state is the one that loses an answer if it reads as success.

**None of this shape applies to Deepgram**, which is why the seam was worth having. It has ONE pre-recorded endpoint for both a voice note and an hour-long recording — no job to submit, nothing to poll, no duration ceiling — and it sniffs the container itself, so Telegram's OGG/Opus is forwarded byte-for-byte and **ffmpeg never runs**. `diarize` is the only thing that varies: off for a voice note (one person; the labels would be noise), on for a recording. A Deepgram failure still falls through to the same job path, which for Deepgram is the identical endpoint reached via a temp file.

`warmTranscription` opens the connection while Telegram is still handing over the bytes — the sync API is one request/response, so a cold call pays the whole DNS+TCP+TLS handshake on the critical path (~150ms of ~370ms). Both are round trips and neither needs the other, so they overlap for free. It is best-effort, never throws, and does nothing when the key is unset or the provider isn't AssemblyAI (Deepgram has no SDK session to open, so there is nothing to warm). The pooled connection is process-global (the SDK uses the global `fetch`, so Node pools per origin), which is why warming through one client helps a call made through another.

Progress shows **on the voice message itself**, via `setMessageReaction`: ✍ before the download, 👍 once there are words and the turn is about to run, cleared on every bail-out so a dead ✍ never outlives the message explaining what went wrong. Bots get one reaction per message and a new one replaces the old, which is what makes the transition possible. The calls are best-effort but **awaited** — fired and forgotten, a ✍ that lands after the 👍 leaves the wrong state on screen permanently, and one round trip against seconds of transcription is not worth racing. Only `voice` gets them, because only `voice` is transcribed.

> **The reaction emoji are written as `\u{…}` escapes, and must stay that way.** Telegram's allowed set is a fixed list of 73 and **not one carries a variation selector** — the writing hand is `U+270D` alone, never `U+270D U+FE0F`. A glyph pasted from an emoji picker or a browser brings the selector with it, which is a different string than the one Telegram accepts, and the call returns `REACTION_INVALID`. The escape is the one spelling an editor can't silently change. Verified against the bytes in Telegram's own docs (each emoji is published as an image whose filename *is* its UTF-8 encoding: `E29C8D`, `F09F918D`).

### The agent's markdown, rendered — `entities`, never `parse_mode`

For the whole life of the bot, replies went out as raw text with no `parse_mode`, so every `**bold**` the agent wrote arrived with the asterisks showing. The desktop had no such problem — its chat sidebar runs `react-markdown` — so the same reply read correctly in one place and looked broken in the other.

Telegram offers three ways to fix that and **only one of them cannot lose a message.** `MarkdownV2` and `HTML` both encode the formatting INTO the string, so every character the user's prose happens to contain that the dialect reserves must be escaped perfectly. MarkdownV2 reserves `_ * [ ] ( ) ~ \` > # + - = | { } . !` — `.`, `-` and `(` appear in ordinary sentences constantly, and one miss is a 400 with the message simply gone. **`entities` sends the text untouched** and describes the spans beside it (`{type:'bold', offset:6, length:5}`), so there is no string to malform, no escaping, and no failure to fall back from. `toTelegram` never throws: a parse that goes wrong returns the original text unformatted, which is what shipped for a year.

**This is why formatting runs on every streamed frame, and hermes' cannot.** ../hermes-agent (`plugins/platforms/telegram/adapter.py:4819`) sends every intermediate edit raw and formats only on `finalize`, wrapped in a `try/except` that strips the markup and resends plain — because a half-typed `**bo` is invalid MarkdownV2, and at ~1 edit/second that is a rejection per frame against a flood budget the same file documents as costing 200s+ penalties. A half-typed `**bo` here just parses as the literal text `**bo`: it renders as itself until the marker closes, then becomes bold. So a port of hermes' converter would have left the streaming bubble exactly as broken as it was — the formatting only appearing once the reply finished.

**What IS taken from hermes is its decisions, which are dialect-independent:** headings become bold lines, list bullets become `•`, code is protected before anything else is touched. Telegram has no heading, list or table syntax at all, so those are presentation choices rather than translations, and hermes made them against far more traffic than we have.

Three things worth knowing:

- **Offsets are UTF-16 code units, which is what a JS string index already is.** `.length` and `.slice()` agree with Telegram for free, so the astral-emoji off-by-one that Python has to encode around cannot occur here. `tests/telegramEntities.test.js` pins it, because "iterate code points instead" looks like a correctness fix and is the one change that would break it.
- **Chunking moved out of `client.ts`.** A boundary has to cut the spans as well as the text, so `splitFormatted` replaced `splitMessage` outright rather than sitting beside it — two chunkers where one is never reached is how they drift. It also needs none of the old one's code-fence bookkeeping: a code block is a span, not a pair of ``` markers, so one straddling a cut becomes one span per chunk and both halves render. hermes carries a further patch (`_separate_chunk_indicator_from_fence`) for the `(1/2)` marker landing on a reopened fence line; that bug is unrepresentable here, since the marker is appended after the spans are rebased.
- **Only the agent's own prose is converted** — the streamed reply (`stream.ts`) and the `send_message` tool (`sendTool.ts`). Tool lines, waiting dots and the fixed strings in `webhook.ts` are our copy and go out plain, which is why `entities` is optional on the client rather than mandatory.

**CommonMark only, deliberately.** The one dependency is `mdast-util-from-markdown` — the parser already under `react-markdown` in the desktop, so both surfaces interpret the agent identically. GFM tables and `~~strike~~` would need further extensions, and `agent-core/defaults/helper.ts` already tells the agent not to use either; a stray pipe table arrives as plain text, exactly as it does today.

### Pointing at a bubble: `telegram_sent`

A reaction update and a reply update both name **only a message number** — never what that message said, never which chat said it. So both gestures are the same lookup, and `telegram_sent` is what makes either answerable: `(telegram chat id, message id) → { content, origin_chat_id }`, and **never expired** — only ever deleted alongside the message itself.

**Every text bubble is recorded, by the thing that sends it — and forgotten the same way.** `TelegramClient` takes an `onSent(messageId, text)` hook at construction and fires it on `sendMessage` *and* `editMessageText`, plus an `onDeleted(messageId)` fired on `deleteMessage` — so a row exists because a message went out, not because the code that sent it remembered to save one. That covers the agent's reply, `/help` and every other command answer, the `⌛ Got it` ack, the transcript echo, the error replies, the placeholder and the per-tool progress lines. There is nothing to add when a new kind of message is introduced, which is the whole reason it sits at that level: the save used to live beside three individual send sites, and everything sent from a fourth — every slash command, as it happened — was silently unpointable.

Three consequences worth knowing:

- **The hook is injected, never imported.** `client.ts` keeps no database access (same seam `stream.ts` uses for `speak`), and the callback is what supplies `origin_chat_id` — which chat a bubble belongs to is something the client cannot know. In `runTurn` it closes over a mutable `chatId`, so a bubble written before the chat is minted records none and one written after records it.
- **Mid-stream edits record too**, which the old per-site version deliberately skipped. It's an upsert on the same message number, so the row simply tracks what the bubble currently says and settles on the final text. The cost is a write per edit (~one per 1.3s while streaming) instead of one per turn.
- **`sendFile` does NOT fire it.** A voice bubble's text is the script that was spoken, which only the caller has — `speakInto` records that itself. Voice matters because a voice-ONLY workspace writes no text bubble at all: without recording the audio, the only thing on screen would be the one thing you couldn't point at.
- **`onDeleted` is the same rule pointed the other way**, and lives at the same level for the same reason. Every waiting bubble is recorded on the way up and most are deleted on the way down; without this each one leaves a row saying `...`. Unreachable rather than harmful — the message it describes is gone, so no gesture can reach it — but there is no reason to keep it, and putting the cleanup beside each `deleteMessage` call is how the recording half went wrong the first time.

- **A reaction speaks it back.** `REACT_SPEAK` on one of my messages pipes the stored `content` through the same synthesis path as voice replies, replying to the bubble it reads. No agent run. A miss does nothing — there is no text to read and nothing useful to say about that — but a synthesis *failure* does reply, since the user asked for something and silence reads as being ignored. **It posts the same waiting bubble a turn does** — the wait is the whole clip being synthesised before a byte is sent, which grows with the length of what is being read, and the `record_voice` action alone sits up beside the bot's name saying the same thing whether that is one second or ten. The bubble is **deleted** rather than taken over, the one way this path differs from a turn: a text message cannot be edited into audio. It is a **plain message, not a reply** — the reply-anchor belongs on the answer (the voice note carries it, and so does the failure message, which otherwise names nothing at the bottom of a busy chat); a bubble that is about to be deleted pointing at something is noise. Answer first, then the delete — the other order leaves a moment with nothing on screen, which is what the bubble is for.
- **A reply switches into its chat.** `switchChatForReply` reads `origin_chat_id` and moves the bot there before the turn starts, announcing `🔀 Switched to: <title>`. That is the whole feature: **a conversation is picked by pointing at it**, with no list to print and no number to type, which is what `/chats` + `/chat n` cost otherwise.

Four rules on the switch:

- **A chat belongs to one workspace, so the workspace moves with it** — otherwise the turn resumes a conversation about one repo inside a checkout of another. Workspace **first**, because `setTelegramActiveWorkspace` clears the active chat and would undo the switch it is part of.
- **A missing chat or a missing workspace refuses the switch outright** rather than half-making it. `activeWorkspace` falls back to the first workspace, so a dangling workspace id would silently run someone's conversation against the wrong repo. Both cases say so and carry on where they were.
- **It happens before anything reads which chat this is** — the busy check, the checkout, the session boot are all keyed on it. Commands included: replying to a bubble and typing `/status` is asking about *that* chat.
- **A miss is silent and harmless.** Replying to your own message, or to a bubble older than the TTL or sent before this existed, just runs the turn where it would have run anyway.

`origin_chat_id` is nullable for exactly that last reason, and because a sender may not know its chat: the desktop's `send_message` posts `chatId` to `/telegram/send` from v1.0.63 on, and an older desktop doesn't. The proactive path is where this is worth most — a cron job or a desktop turn reaches out, and answering it lands where the work is.

**Setup** (desktop Settings → `/telegram/{connect,disconnect,status}`): `connect` validates the token (`getMe`), mints a random webhook secret, registers the webhook (`ALLOWED_UPDATES` — `message` **and** `message_reaction`; `syncWebhookConfig` reconciles a subscription registered before a kind existed, at boot, against `getWebhookInfo`), and saves the account (botToken + webhookSecret encrypted under owner `telegram`; `dmChatId = authorizedTgUserId`, since private chats). `server.ts` resolves the public URL/cert first: `COMPANION_DOMAIN` set → `https://<domain>` (trusted, no PEM); unset → detect public IP + self-signed cert (Traefik serves it) → `https://<ip>` with the PEM uploaded to Telegram.

**Webhook (`handleWebhook`):** account enabled? → secret-token header timing-safe-checked (403 on mismatch) → sender must be `authorizedTgUserId` (single user, DM-only; unknown senders silently 200) → `markTelegramUpdate` dedups retries → **fast-ack 200**, then run the turn out-of-band.

**Concurrency — a second message while the agent is working.** The chat is marked busy for the whole job. A message arriving meanwhile is relayed into the RUNNING turn (pi delivers it at its next step), acknowledged with `⌛ Got it — after I finish the last task.` replied under the offending message, and then the handler **returns**. It must not fall through to the finish-up steps: those belong to the turn still in flight, and running them early committed half-edited files and abandoned the first reply mid-sentence. `agent-core` checks for a steer BEFORE validating provider/model, so the relay can pass none.

**Two halves of a turn run at once.** Resolving the message (downloading a voice note and transcribing it, or fetching attachments) needs no workspace; getting the workspace ready needs no message. Run in sequence they add up — transcription is seconds of network and so is a clone — so `runTurn` starts `prepareRun` (workspace, PAT, checkout, and for an EXISTING chat the pi session boot) alongside `resolveInput` and joins them before prompting. Three rules make it legal:

- **A `/command` never fans out.** It's answered from the database and runs no turn, so it must not drag a checkout behind it. Decided from the raw message (`typedTextOf`) *before* `resolveInput` runs — which is the point. A voice note or file is never a command.
- **A busy chat is never prepared.** Its session is live and owns its event sink; preparing would re-point `emit` mid-reply, the bug that once froze a reply half-written. Checked before the fan-out **and again after the join**, because with both sides running the turn can start during the window.
- **Failures are held, not thrown** (`settle`). An unhandled rejection while the other side is still working takes the process down.

The pi session is pre-booted **only for a chat that already exists**. Booting also creates the chat row, and a row whose transcript never arrived — because the transcription came back empty and no turn ran — is a chat that refuses to resume once its scratch dir ages out. An existing chat has both already, and is where the pre-boot is worth most anyway (resuming downloads and parses the transcript). `agentPrepare` lives in `agent-core`.

**The typing indicator starts at the ack**, owned by `runTurn` for the whole turn. It used to come from `makeTelegramSink`, built *after* the checkout — so a plain text message left the user watching an empty chat through the slowest part of the turn. `stream.ts` no longer starts one.

### The placeholder bubble is a SLOT, claimed by whichever output comes first

A `...` message goes up before the agent has produced anything, so the wait for the first token happens inside a visible bubble instead of an empty chat. It is the same bubble the reply is then edited into — the first flush becomes an edit instead of a post, so posting it costs exactly one extra API call per turn. It applies to every turn, text or voice.

**It is not "the text bubble, posted early."** On most turns the agent reads or greps before it says anything, and `toolLine` posts its own message and resets `messageId` — so a bubble reserved for text alone would leave a bare `...` stranded above the tool line, never updated again. Instead **text and tool lines both take it over**: text by editing it, `toolLine` by editing the tool line into it rather than posting below. One rule, correct in either order.

### The bubble is its own module, and ONE WRITER AT A TIME is why

`waitingBubble.ts` owns the message: it posts it, animates it, and hands it over exactly once. `claim()` waits for the post (and any frame still in flight) to land, stops the animation, returns the id and gives up ownership; `remove()` deletes it; `stop()` just freezes it. After a claim the bubble never touches that message again.

The animation started life inside `makeTelegramSink`, sharing one closure and one promise chain with the reply text — which worked, but only because both lived in the same function. The 🤬 reaction needs the same bubble (see "A reaction speaks it back" above) and renders no events at all, so the choice was a second copy of the animation or a piece that can be handed over. Pulling it out with its own queue and no ownership rule would have been the worst of the three: two writers on one message, and a dot landing on top of the first words of a reply. Pinned by `tests/waitingBubble.test.js`, because that failure is intermittent and not reproducible on demand.

Five details that are load-bearing:

- **`unwritten` clears only after a write LANDS** (inside `flushInner`'s `try`), so a rate-limited edit leaves the message a slot. The one case where a *landed* write still leaves it holding nothing is a body that cleaned down to the static `…` — a segment that was only a file tag — and that is now an explicit `body === PLACEHOLDER` check. It used to ride on the *"message is not modified"* throw that an unchanged body produces, which is why `PLACEHOLDER` is a **separate constant from the animation frames**; the check says what it means and no longer depends on an exception it doesn't control.
- **`tookBubble` is separate from `messageId`.** A failed post yields no id, so asking the bubble twice would return null and wipe an id already held. It also decides which half of `dropPlaceholder` runs: nobody has taken the bubble (it deletes itself) or we hold an unwritten message (we delete it).
- **`toolLine` resets `text` BEFORE any await**, as it always did — `emit` appends to `text` outside the chain, so a delta arriving mid-claim or mid-send would be wiped by a reset that ran afterwards. The reset now sits ahead of `takeSlot()` for exactly that reason: deltas that arrive while the handover is in flight belong to the next segment.
- **The bubble grows a dot every 600ms while it waits**, cycling `...` → `......`, so a slow first token looks like waiting rather than like nothing happening. It stops on exactly two conditions and there is deliberately no third: being claimed or removed (or never posted), and **any edit failing**. 600ms is faster than Telegram's ~1 edit/sec ceiling and `call()` retries a 429 *inside the chain* — so a throttled frame would stall the first tool line behind it. Stopping on the first error bounds that at one retry ever, and frozen dots still read as a placeholder. A time cap was considered and left out: the two conditions already cover every way a turn ends, and `done()`/`dispose()` stop it regardless.
- **`dispose()` exists for the turn that throws.** `sink.done()` sits after the try/finally in `runTurnInner` and is never reached on that path, which stranded the placeholder above `runTurn`'s error reply — and, already true before any of this, leaked the 1.3s edit timer for the life of the process, one per failed turn. The `catch` calls `dispose()` and rethrows. **Both timers are stopped in both teardowns.**

A turn that ends with nothing to say drops the bubble (`dropPlaceholder`) rather than leaving a bare `...` as the whole reply. `done()` takes the slot first, so a turn that never rendered anything edits its final message into the bubble instead of posting below a stray `...`.

**Turn (`runTurn` → `runTurnInner`):** `runTurn` wraps the inner run in try/catch so **any failure replies in-chat** (`⚠️ Something went wrong running the agent:\n<message>`) and then rethrows for server logging — a silent failure reads as the bot ignoring you. `runTurnInner` handles `/new`,`/status`,`/help`; picks the workspace via `activeWorkspace` (in-chat error only when no workspaces exist); requires `sync.pat` (in-chat error if absent); `prepareCheckout` clones/refreshes via `git.ts`; runs `runtime.agentSend` under a `codingAgent.maxRunMinutes` watchdog (unset ⇒ 30); dual-publishes each event to the `feed` (desktop watches live) and the Telegram sink; then lands the work via `checkInWithFixer` — the same path cron uses, git-fixer included — and reports a `'conflict'`/`'error'` result in-chat. `source: 'telegram'`, `sourceId` = DM chat id.

## Cron

`scheduler.ts` (gated by `CRON_ENABLED`): one croner per enabled `cron.json` entry (`protect:true`, workspace timezone), plus a refresh croner that `reconcileAll`s — fetches each workspace's `cron.json` via `fetchCronJson` (ETag/304) and updates registrations **non-destructively** (unchanged jobs keep running; changed schedules are replaced; vanished jobs dropped). `fireJob` mints a chatId, runs `runCronJob`, records history to `cron_state`. `cronRun.ts` is shared by the scheduler and the manual `POST …/cron/:job/run`: checkout → read the job prompt from the checkout's `cron.json` → agent turn streamed to the feed (watchdog) → `checkInWithFixer` (shared with Telegram). Checkout dirs are keyed by chatId (re-runs reuse) and reclaimed by `sweeper.ts`.

### One-time jobs (`"once": true`)

A `cron.json` entry with `"once": true` and an **ISO datetime** `schedule` (`"2026-03-14T18:50:00"`, interpreted in the workspace timezone — croner takes a date as a pattern natively) runs once and **deletes its own entry**. There is no separate one-shot store, no new endpoint, and no bookkeeping: `cronRun.ts` calls `dropJob()` to remove the entry from the checkout's `cron.json`, the run's existing `checkIn` commits + pushes that, and the next reconcile sees the job gone and drops the registration. The agent-facing docs are the **`cron` tool's description** (`agent-core/cronTool.ts`) — there is no longer a prompt section, and `once` is derived from the schedule rather than asked for, so a dated job can't be written without it and then linger in the file forever.

**A manual run does not dispose of a one-time job.** `runCronJob` takes `{ manual: true }` from `POST /workspace/:id/cron/:job/run` and skips `dropJob`. It is shared with the scheduler, so pressing Run on a reminder set for tonight used to consume it — the test ran, the entry vanished, and the scheduled moment passed with nothing happening and nobody told. Trying a job out must not be the same act as spending it.

`scheduler.ts` needs **no** one-shot handling: croner accepts a date as a pattern, fires it once, and reports `nextRun() === null` afterwards. Disposal also happens when the turn **fails** — the turn is wrapped in `try/catch` into `turnError` so `dropJob` + `checkIn` still run, then it rethrows. Once means once, and a failed job that kept its entry would leave a permanently dead line in the file.

**A missed moment is missed**, exactly like any cron: if the companion is down at the fire time, nothing runs, and the (now unfireable) entry sits in `cron.json` until someone removes it. Deliberately not caught up — a reminder arriving hours late is worse than none, and the catch-up machinery cost more than the case is worth.

**Registration latency is ~70s** — a desktop-authored edit needs a sync tick (10s) to reach GitHub plus a reconcile cycle (≤60s). One-time jobs less than ~2 minutes out don't reliably register; the helper prompt tells the agent to act immediately instead.

## Background runs (`backgroundSweeper.ts`, `backgroundRun.ts`)

The agent gets better at a workspace by writing down what it learned. It could already write skills and save facts, but only when a user thought to ask — so most of what was worth keeping was never captured. This removes the human trigger.

**Two processes, not one.**

| | counts | mark | setting | writes | prompt |
|---|---|---|---|---|---|
| **review** | the agent's TOOL CALLS | `last_reviewed_seq` | `reviewInterval` | skills, via `manage_skill` | hermes' `_SKILL_REVIEW_PROMPT` |
| **memory** | the USER's messages | `last_memory_seq` | `memoryInterval` | `MEMORY.md` / `USER.md`, via `memory` | hermes' `_MEMORY_REVIEW_PROMPT` |

They come due at different times because they measure different things, and that is the reason they are separate rather than one pass with a combined prompt. A chat that ran forty tool calls and two sentences reveals almost nothing about the user; one that ran none and twenty sentences may reveal a great deal. Merging them would also mean pointing the skill instruction — *"be ACTIVE, most sessions produce at least one skill update"* — at a chat that only talked, which is how an agent writes a skill about nothing. The memory instruction deliberately carries no such bias; `tests/memoryPrompt.test.js` pins the asymmetry.

**Each is cron with a different trigger.** Find a due chat, **clone its row into a new chat**, and resume it. Its own checkout, landed with `checkInWithFixer` — agent paths three and four, exactly as the boxed rule above requires.

**It is a RESUME, not a fresh chat handed a description of one.** `cloneChatForBackground` (`store.ts`) copies four fields — the conversation, the stored system prompt, the workspace and the model — and `agentSend` then boots it exactly like any other continued chat: row found, conversation pulled from the database, session reopened. So the agent picks up the real thing, every tool call with its arguments, the reasoning, the images.

It used to flatten the conversation into text and paste it into the run's first message, and that rendering **dropped the tool arguments** — it emitted `ASSISTANT [called bash]` followed by what the command printed, so the run read an output with no idea what produced it. Which is most of what a skill is made of.

**`source` is the field that must NOT be copied**, and the reason the clone is a function rather than a spread: both sweep queries exclude chats whose source is `review` or `memory`, so a run that inherited `desktop` would cross its own threshold, come due, and review itself forever. The watermarks are fresh for the same reason — they belong to the chat being examined, not to the examination.

**What the run is told in words** is only what the inherited, frozen prompt gets wrong: it names the source chat's working directory, and it was assembled for a chat with a user in it. So the first user message says the checkout path to use instead, and that nobody is present. That is `backgroundInstruction` in `agent-core/defaults/conversation.ts` — the whole of what that file does now.

### One tick, because `protect: true` does not bound siblings

Both run off ONE croner (`REVIEW_SCHEDULE`, default `*/5 * * * *` — the name predates the memory pass and is kept because renaming it would silently re-enable background runs on any box that had set `REVIEW_ENABLED=false`), which starts **at most one run per tick** and picks whichever chat has the older unexamined work (`oldest` in the query, so a busy process cannot starve a quiet one). What they share is the clock and nothing else.

Sharing it is the whole guarantee. `protect: true` bounds a croner against *itself*, not against a sibling — two timers could wake in the same second and start two unattended agent runs against the same repo, which then push over each other and leave one resolving a conflict the other caused. One timer that awaits its run makes that unrepresentable, with no lock for either side to forget to take. (Note this is not a global cap: cron can still fire several jobs alongside it. Nothing in the app caps total concurrent runs today.)

Separate croner from cron deliberately — a job scheduled for 2am should fire at 2am rather than queue behind background maintenance; neither of these has a deadline.

### Why the companion and not the end of a turn

Turns happen three ways — desktop, Telegram, cron — across two processes, and only this server sees all of them, because every turn's messages land in its `message` table whoever ran it. Hooking the turn would mean writing it twice and having it never fire while the desktop is closed. It also means there is no counter to keep: "how much has happened" is a count of rows past a mark on the chat, which cannot drift and survives restarts. hermes carries incremented counters precisely because it cannot count what actually happened.

### A chat is not reviewable until its work has landed AND you have stopped talking

Two conditions, in `settledAndQuiet(quietMs)` (`store.ts`), and they answer different questions. Both sweeps use it.

**Landed.** `running` clears when the agent stops talking, which is **before** the check-in — `agentSend` ends with `setRunning(chatId, null)` and only then does the caller push. Both sweeps pick chats on `running = false`, so there is a window where a chat looks finished and its work is still in flight. A run starting in that window clones a checkout **missing the work its conversation describes**, and two agents end up pushing to the same repo at once. So every server-side run lands through **`checkInAndStamp`** (`gitFixer.ts`), which checks in and then records `chat.checked_in_at`:

```
c.checked_in_at is null                                   -- desktop: no check-in exists
or c.checked_in_at > (max message created_at for the chat) -- this turn's check-in has finished
```

Nothing else pushes for a given chat, so a check-in later than the conversation's last word must be that turn's — which is why no turn-start column is needed.

**Settled.** Landed says the last *turn* ended. It says nothing about whether the *conversation* has, and `running` clears the instant the agent stops talking — so without a wait a run can start while the user is composing their next message. It then examines half a conversation, out of a checkout the next turn is about to write into. So the settling timestamp must also be `quietMs` old:

```
coalesce(c.checked_in_at, max message created_at) <= now() - quietMs
```

`quietMs` is `codingAgent.backgroundQuietMinutes` (synced; unset ⇒ 10, **0 means no wait**, not off — the two intervals are what switch a process off). **One number for both processes**, deliberately, and the one place they share something beyond the clock: they come due at different times because they measure different work, but *"is the user still in this conversation?"* is a fact about the source chat, not about which pass is asking. Two knobs would be two answers to one question. Nothing waits on these runs, so a longer wait costs nothing.

**The NULL arm is desktop chats, it is not a loophole, and the quiet window is what makes it defensible.** They never check in — their files reach GitHub through the desktop's sync engine on its own timer (10s default), which reports to nobody, so the server has no landing to verify. Before the quiet window that arm was a bare pass: a desktop chat became eligible the instant the agent stopped talking, with its work possibly still sitting on the user's machine. Measuring the wait from the last message instead is a **bet, not proof** — minutes of silence against a ten-second tick — and it is knowingly wrong in one case: if sync is switched off for that workspace, offline, or stopped on a conflict, the work never lands and no wait length helps. Closing that properly means the desktop reporting its pushes to the server, which is a bigger change and not made. Dropping the NULL arm is not the alternative: without it, desktop chats — most of them — would never be examined again.

**Stamped on failure too.** It records "we finished trying", not "we succeeded". A chat that is never stamped drops out of review permanently, and a conversation whose push conflicted is one worth learning from.

**Holding `running` across the check-in would have been simpler and is wrong.** The desktop freezes its composer for a chat running on another machine (`remoteMachineOf` in `chatStore.ts`), so a Telegram chat would be un-typeable for as long as the fixer ran — up to `maxRunMinutes`. A separate stamp blocks the sweep and nobody else. **Never block a user message to protect a maintenance pass.**

Not covered by an automated test: the due queries need a live Postgres, like the rest of this tree.

### Five things the loop depends on, each of which breaks it if removed

- **`source='review'` and `source='memory'` chats are excluded from BOTH sweeps, and refused again at the clone.** A background run makes tool calls and holds a conversation like any chat, so without this it crosses a threshold and examines itself, forever — and a memory run's own messages must not make it due for review either. The list is `BACKGROUND_SOURCES` in `store.ts`, used three times: the `notBackgroundChat` SQL fragment both sweeps filter on (one fragment, because the two queries are deliberate near-copies and this is the line in them that must never differ), and a runtime check in `cloneChatForBackground`. Two guards covering different failures — the query decides what gets **picked**, the clone decides what may be **opened however it was picked**, which is the half a second caller (a hand-run query, a manual re-run route added later) cannot bypass. Pinned by `tests/backgroundSources.test.js`, which reads the source: the due queries need a live Postgres, so nothing else can reach them.
- **Each process moves only its OWN mark**, and moves it **BEFORE** the run. A run that throws must not leave the chat eligible on the next tick — that is a failing run retried every five minutes indefinitely. Nothing is lost: later work makes it due again normally.
- **The mark is the chat's own high-water mark**, not `max(seq)` over the joined rows — the join is filtered by role, so that stops short of the rest of the conversation while the run read all of it.
- **The mark also jumps to wherever the agent last did that job ITSELF** (`selfSaveMark` in `store.ts`): the newest `role='tool'` row whose `tool_name` is `manage_skill` / `memory`. hermes resets its equivalent counter on the foreground tool call and we had never ported that half — so a chat where the user said "remember that" and the agent did was still counted as owing the work, and got a background run to learn something already learned. Computed from data we already store, so nothing has to reach back from `agent-core` into Postgres to move a counter.
- **`protect: true` plus awaiting the run inside the tick** bounds this to one background run in flight. That is also why their claim on the warm-checkout queue is *lighter* than cron's: cron can fire several jobs at once, these cannot.
- **`settledAndQuiet` gates both sweeps, and both halves of it matter.** Drop the landed half and a run clones a checkout mid-push; drop the quiet half and it opens a chat the user is still typing into. `quietMs` is passed in rather than read here so `store.ts` keeps no settings import.

### Settings and migration

`codingAgent.reviewInterval` and `codingAgent.memoryInterval` (synced; unset ⇒ 10, **0 disables** either one independently) are the thresholds — the numbers a user would actually change, which is why they are synced rather than env. `codingAgent.backgroundQuietMinutes` (synced; unset ⇒ 10, 0 = no wait) is the third, shared by both — see the section above. The cadence (`REVIEW_SCHEDULE`) and master switch (`REVIEW_ENABLED`) are env, like `CRON_REFRESH_SCHEDULE` and `CRON_ENABLED`: server tuning, not user behaviour. The two char budgets (`memoryCharLimit` / `userCharLimit`) are synced too and travel to every run through `RunOpts`, so a memory pass never consolidates to a size the next desktop turn considers over the limit.

**Migration note.** Both `last_reviewed_seq` and `last_memory_seq` are seeded to each existing chat's current high-water mark rather than 0, or every chat in the database would be due the first time the sweep ran.

## Agent execution (`agentHost.ts`)

`makeCompanionRuntime(pool, key)` builds an `AgentHost` and calls `agent-core`'s `createAgentRuntime` — the same runtime the desktop implements, but wired to direct I/O instead of IPC: persistence → the drizzle store, events → `feed`, a per-run scratch `dataDir` keyed by chatId (isolates concurrent runs' pi `settings.json`), `extraTools = [send_message]` (built from `agent-core/sendMessage.ts` with `sendTelegramMessage` injected — the desktop offers the same tool, backed by `POST /telegram/send`), `getAgentSecrets` from `readSettings`, `getToken` → `mintToken`. Both cron and Telegram drive it via `runtime.agentSend(payload, emit)` / `runtime.agentAbort(chatId)`. The git-fixer (`gitFixer.ts`) runs a **separate** pi session from the turn.

## When you touch this

- **Adding a settings field:** it's just a `setting` row (or a `secret_value` row if it's a credential — declare it in `agent-core/credentials.ts`, which `keys.ts` derives from; editing `keys.ts` itself is the wrong layer and desyncs the desktop's strip + send guard). No default to register anywhere — required fields error at their consumer, optional fields fall back at point of use.
- **Adding a `secret_value` owner category:** every reconciliation that deletes from `secret_value` must be scoped to its own owners (see the boxed rule above).
- **Schema change:** edit `schema.ts` *and* `init.sql` (both are re-applied idempotently). Keep them in sync.