pr-cockpit

# pr-cockpit / PR Cockpit

## What this project is

- This is the user's local batch PR review workbench. The product name is PR Cockpit; the repo usually lives at `/Users/openstudio/work/products/tools/pr-cockpit`.
- Core flow: pull the GitHub PR list, give each PR an isolated worktree, have the AI produce a structured review, and let the human vet it in the web UI before posting line-level/summary comments to GitHub.
- Stack: Nuxt 4 + @nuxt/ui/Tailwind v4, better-sqlite3 + drizzle, Nitro `server/api/`, the local `gh` CLI, `@anthropic-ai/claude-agent-sdk` and `@openai/codex` (the Codex CLI binary, driven through `codex app-server`).
- Main directories: `core/` is the business logic and agent engine, `server/api/` is the Nitro API, `app/` is the Vue UI, `tests/` holds lightweight contract/regression tests, `data/` is the local SQLite plus the migration source for the old central worktrees.

## Running locally

- Usual checks: `pnpm typecheck`, `pnpm test`; also run `pnpm build` for riskier UI/bundling changes.
- Dev server: `pnpm dev`, README defaults to `http://localhost:3001`.
- The long-running pr-cockpit instance on the user's machine is usually on port `5332`; `4737` is a different project. Confirm the port before touching processes — don't kill the wrong project's server by matching on the `.output/server/index.mjs` process name.
- SQLite has no formal drizzle migration flow. Tables are actually created by `ensureSchema()` and `ensureColumns()` in `core/db/client.ts`; `core/db/schema.ts` only provides query types. When changing the DB, update both places and keep them idempotent, so that re-running them on every startup stays safe.
- Observability tables (phase 0 of the session-host rework, 2026-08): every agent execution writes a `runs` row (review / guided / recheck / skillgen get their own row per execution; fix / feature / global sessions reuse the entity id as the run id and append per turn) plus `run_usage` rows per model. Cost comes from the Claude result's `modelUsage` / `total_cost_usd`; Codex only reports tokens, so its cost is estimated from `data/codex-rates.json` (see `docs/codex-rates.example.json`) or left `NULL` — never write 0 as a placeholder. Skill text is versioned in `skill_versions` (`core/skillVersions.ts`; edits insert a new version, `skills.content` mirrors the current one) and reviews record `skill_version_id`. `findings.checked_by` distinguishes human clicks from the drawer's auto-adjust and the automation engine; the dashboard's precision metric only counts `human`. Dashboard: `app/pages/dashboard.vue` ← `/api/metrics/overview` (aggregates) + `/api/metrics/runs` (the run list, `offset`/`limit` ≤ 100 + `total`) ← `core/metrics/queries.ts`. Pages that list things page client-side with `app/components/PagerBar.vue` (UPagination, 1-based) unless the list is unbounded (the run list pages server-side).
- Session host (phase 1, 2026-08): Claude chats no longer spawn `claude -p` per turn — `core/host/claudeHost.ts` keeps ONE long-lived Agent SDK `query()` per session (streaming-input mode, `core/host/queue.ts`), fed a user message per turn. `core/host/options.ts` is the only place SDK options are assembled: settings are loaded like the CLI (no `settingSources: []`), Claude Code's own system prompt is kept and ours appended, `permissionMode` is per session (default / acceptEdits / plan / bypassPermissions, switchable mid-session), `allowDangerouslySkipPermissions` is always on so the switch works later. Safety = the permission bridge (`core/host/permissions.ts`: `canUseTool` → a `permission_requests` row + `permission_request` RunEvent → the UI answers via `POST /api/runs/:id/prompts/:pid` → the parked promise resolves; AskUserQuestion = kind `question`, ExitPlanMode = kind `plan`) + a PreToolUse hook that turns dangerous Bash (`isDangerousCommand` in `core/agent/dangerGuard.ts`) into an `ask` in every mode unless the run's `allowDanger` switch is on. All SDK messages are normalized (`core/host/normalize.ts`) into RunEvents, fanned out on bus channel `run:<id>` (SSE `/api/runs/:id/stream`, and the global session stream) and persisted to `run_events` (text deltas are live-only). Result cost/usage is cumulative per query lifetime — `diffCumulativeUsage` turns it into per-turn deltas. Idle sessions close after `HOST_IDLE_MS` (20 min) and resume via `resume` on the next message; `server/plugins/recover.ts` expires pending prompts on restart; `shutdown.ts` closes live queries. Every session run (PR / feature branch / cwd) runs on the host through `core/runs/session.ts` (worktree kinds default to `bypassPermissions` + the danger hook, cwd to `default`; the UI can change the mode per session; pipeline events share the `run:<id>` channel). Codex sessions run on the Codex host (next bullet) behind the same `SessionHost` interface — the session pipeline picks a host with `hostFor(provider)` and the `/api/runs/:id/*` endpoints with `hostOf(runId)` (`core/host/index.ts`). The UI side is shared: `app/composables/useRunHost.ts` (RunEvent reducer, pending prompts, mode, context meter, cost) + `RunPromptCards.vue` + `RunHostStrip.vue`, used by `session/SessionView.vue`; `core/host/pending.ts` decodes pending rows for the detail endpoints. Verified live: Write → permission card → allow; AskUserQuestion; plan card → approve → CLI leaves plan mode. Note the CLI auto-allows some read-only commands (e.g. `echo`) in default mode without a prompt.
- Review family on the host option factory (phase 2, 2026-08): review / guided / recheck / skillgen build their SDK options with `buildReviewOptions()` in `core/host/options.ts` — the user's configuration (CLAUDE.md, rules, skills, plugins) is loaded like the CLI and the operating contract + review skill are appended to the Claude Code preset. Read-only is enforced by three independent layers in `core/host/readonly.ts`: a PreToolUse hook (a hook deny beats every allow rule, and `settings.disableAllHooks` does NOT switch off SDK callback hooks — verified live), inline `settings` deny rules + `disableAllHooks: true` (the user's own hooks never run in a review worktree), and `disallowedTools` + `canUseTool`. All Bash decisions go through `isDangerousBash` in `core/agent/guard.ts` — a blacklist that can never be complete (variables, backslashes, exotic redirects); treat it as defence in depth and extend `tests/host-readonly.test.ts` when touching it (write primitives are matched only in command position so paths like `patch.ts` stay allowed). MCP servers are not even connected unless the owner's "let reviews use MCP" switch is on (`core/agent/settings.ts`, `meta` keys `agent.chrome` / `agent.reviewMcp`, edited on `/agent-config`; on = every configured server is callable, like a session, and Claude in Chrome comes along when `agent.chrome` is set — the review verdict takes a boolean, there is no per-server list any more). Servers the PR branch itself declares are never enabled: `buildReviewOptions` lists `<cwd>/.mcp.json` keys in `settings.disabledMcpjsonServers` (the CLI would otherwise auto-approve and spawn them at init — a server SPAWN is invisible to all three read-only layers), and the Codex host sends `mcp_servers: {}` to an unattended thread whose worktree has a `.codex/config.toml`. `agent.chrome` applies to every Claude session kind (cwd / PR / branch worktree), not only directory sessions. A running review can be stopped (`core/agent/reviewAborts.ts`, `POST /api/reviews/:id/stop`, also wired into the Codex runner's stop handle). One-shot text helpers (commit message, feature title, comment rewrite, JSON repair) use `runHelperText` in `core/host/helpers.ts` — SDK, no settings, no tools, one turn, explicit cwd — instead of `claude --print`. `CLAUDE_CODE_PROJECT_DIR_NAME` pins the memory/transcript directory of worktree runs to the project's main clone (`projectDirNameFor`).
- Codex on `codex app-server` (phase 4, 2026-08): `@openai/codex-sdk` is gone. `core/codex/appServer.ts` keeps ONE `codex app-server --listen stdio://` process per Nitro process (JSON-RPC over NDJSON in `core/codex/rpc.ts`; never a ws:// listener or the daemon — no second locally reachable control surface; the user's shell aliases `codex` to bypass sandboxing, so never `shell: true`), started lazily, restarted with backoff after a crash (live threads get a `crashed` callback and their turns fail). `core/codex/codexHost.ts` implements the same `SessionHost` surface as the Claude host: a thread per run (`thread/start` / `thread/resume`, stale ids fall back to a fresh thread + a `note` event), `turn/start` per message with the sandbox/approval policy of the CURRENT mode (`core/codex/policy.ts`: plan → read-only sandbox; default/acceptEdits → workspace-write + `on-request`; bypassPermissions → `never`; the danger switch = full access + network), `turn/interrupt`, `/compact` → `thread/compact/start`. Approvals (`item/commandExecution|fileChange|permissions/requestApproval`, legacy `execCommandApproval`/`applyPatchApproval`) and `item/tool/requestUserInput` go through the shared permission bridge (`core/host/permissions.ts` rows + `permission_request` events, answered by the same endpoint/cards; "always" → `acceptForSession` or the proposed execpolicy amendment). Notifications map to RunEvents in `core/codex/mapEvents.ts` (pure, fixture-testable); token usage is per-thread cumulative (`thread/tokenUsage/updated.total`) and differenced per turn; USD is still the rate-table estimate or null. Review / guided / recheck / skillgen use `runCodexReadonly` in `core/codex/oneshot.ts`: an ephemeral thread with `readOnly` sandbox + `untrusted` approvals, so EVERY command is submitted before it runs and `isForbiddenRemoteOrGitMutation` declines git/GitHub mutations pre-execution (verified live: `git push` declined, `outputSchema` honoured). Sessions keep the post-execution guard (`shouldBlockCodexCommand` on completed commands → the turn is interrupted and errors) because in-sandbox `git commit` never asks. Binary resolution lives in `core/codex/bin.ts` (env → packaged `.output/vendor/codex/bin` → pnpm vendored → PATH, logged as unpinned); `scripts/prepare-electron-codex.mjs` copies the vendored binary into `.output` for packaging (macOS signing of that nested binary is unverified). `codexStatus.ts` / `codexModels.ts` read `getAuthStatus` / `model/list` from the live server; the transparency page's Codex section is `core/codex/describe.ts`. Tests: `tests/codex-host.test.ts` drives the host against `tests/helpers/mockCodexAppServer.mjs` (set `CODEX_EXECUTABLE` to a `.mjs` file and the RPC layer runs it under node). The user's Codex hooks/plugins fire inside threads (a `stop` hook was observed) — they are part of the loaded configuration, not something we disable.
- Verify-before-post + eval replay (phase 5, 2026-08): `projects.verify_before_post` (project config switch) makes a fresh review run a second read-only pass (`core/agent/verify.ts`, same read-only policy as reviews; Codex via `runCodexReadonly` with an output schema) whose only job is to refute each finding; verdicts land in `findings.verify_status` / `verify_note` (refuted findings stay visible but unchecked, the drawer shows a tag) and the pass has its own `runs` row (`subkind = 'verify'`). A failed verify never fails the review. Eval replay: `pnpm eval run --golden eval/golden/<name>.json --project <id|name> [--provider] [--model a,b] [--effort] [--skill-version|--skill|--methodology] [--verify]` (`scripts/eval.ts` → `core/eval/runner.ts`) replays labelled PRs at a fixed head sha (`prepareWorktree({ checkoutSha, prNumber })` falls back to `refs/pull/<n>/head` when the branch moved or is gone; no merge of the default branch so the input is exactly the labelled head), scores findings against labels with path + title/problem token matching (`core/eval/judge.ts`, greedy one-to-one, no LLM judge in v1), reports precision / recall / F1 / cost with and without the verify pass, writes `eval_runs` / `eval_cases` / `eval_findings` and a markdown report under `eval/reports/` (git-ignored). It never posts and never writes to git/GitHub. Golden format: `eval/golden/example.json`.
- Unified session runs (phase 3 completion, 2026-08): the fix / feature / global chat stacks are ONE thing now. A session is a `runs` row (`kind = 'session'`) bound to a workspace — `pr_worktree` (a PR branch worktree; edits stay uncommitted until the upload path commits+pushes), `branch_worktree` (a fresh branch cut from the default branch; the agent may open the PR) or `cwd` (any directory). Turns live in `run_turns`, events in `run_events`, prompts in `permission_requests`; the workspace state that used to sit on fixes/feature_tasks/global_sessions (base/fix/push shas, pushed_at, reviews_at_push, pr_url, upload_state, busy_action, description) is on `runs`. `core/runs/migrate.ts` copied the legacy tables in once (same ids; marker `runs.migrated.v1` in `meta`; the old tables stay as a rollback net and nothing reads them). `core/runs/session.ts` is the single turn pipeline (`runSessionTurn`, `isRunBusy`, `stopRun`, `fixStatusOf` = the legacy open/ready/pushing/pushed/error status derived from upload_state/busy_action, used by automation and the PR list). API: `POST/GET /api/runs`, `GET/PATCH/DELETE /api/runs/:id`, `POST /api/runs/:id/{messages,stop,push,fork,open}`, `DELETE /api/runs/:id/workspace`, plus the host endpoints (`stream`, `events`, `interrupt`, `mode`, `prompts/:pid`). Provider follows the project/runtime defaults until a native session exists, then the run's own row pins it; before a provider takes over, the other host's live session for that run is closed (hostOf must never route to a stale one). UI: `app/components/session/SessionView.vue` is the one chat surface (turns, host cards, ask-user card, slash palette, danger/mode/ultracode switches, upload preview, open/update PR, worktree tools, open in VS Code/Cursor/Terminal); `PrDetailDrawer` (fix tab) and `GlobalChat` (the project assistant: FAB + slideover on project pages ONLY, workspace picker cwd / new branch worktree for a new session, per-project history of both kinds with rename / delete / fork, `?session=<id>` deep link via `useOpenGlobalSession`) are thin shells around it — the separate feature tab / `SessionsTab` was removed 2026-08-27; sessions without a projectId are adopted into a project's history when their path lies under its clone (`GET /api/runs`). History handoff between providers is `/api/agent/history/run/:id` (`core/agent/historyAccess.ts`). Automation dispatches to `POST /api/runs` + `/messages` + `/push`; `server/plugins/recover.ts` reconciles `busy_action = pushing` and streaming `run_turns` on boot.
- Housekeeping that closed the plan (2026-08): the per-turn `claude -p` chat runner (`runClaudeAgentChat`, `claudeCli.ts`, the danger hook file writer, `ChatRunner`) is deleted — only the system prompt builders remain in `core/agent/{fixer,featureChat,globalChat}.ts`. Sessions in worktrees pin `CLAUDE_CODE_PROJECT_DIR_NAME` to the project clone (one memory dir per project). `REVIEW_MAX_BUDGET_USD` caps a review-family execution (unset = no cap). `core/host/recover.ts` is the boot-time host recovery (tested). The session stream renders host events as cards (`session/RunEventCard.vue`: tool call + result, Edit/Write change, thinking, subagent, compaction, denial). Codex: `core/codex/protocol/` holds the generated app-server bindings for the pinned `@openai/codex` (`pnpm codex:types` regenerates; `EXPECTED_CODEX_VERSION` in `core/codex/bin.ts` must match package.json — a test checks it, and the handshake warns on a different binary); the RPC layer retries `-32001` (overloaded) with backoff; `/fork` is a local slash command; stopping a PR session reports that the PR's automation was paused. `tests/host-config.probe.ts` is the manual CLI probe (plugins / Chrome / connectors / memory files).
- Session composer extras (2026-08, the items rescued from the plan's §6 cut list): (1) Slash palette — `core/host/commands.ts` classifies the CLI's command list (user/project skills by their "(user)"/"(project)" description suffix, plugin commands by namespace, the rest built-in; only `CURATED_BUILTINS` show without "show all", `HIDDEN_COMMANDS` never; MCP prompts lose their display-only " (MCP)" suffix). The catalogue comes from `GET /api/agent/commands?provider&cwd|projectId` (the probe cache) and is replaced by a live `commands_changed` push (RunEvent `commands`). Matching is PREFIX matching on the name, any `.`/`:`/`__` segment and aliases (`/plan` → `speckit.plan`, `Notion:tasks:plan`). `session/CommandPalette.vue` is the dropdown + grouped browser; cockpit-side commands (`/clear /new /resume /fork /cd /copy /model /effort /stop /push /pr`) shadow same-named built-ins and are intercepted in `SessionView.handleSlash` (`/model` `/effort` → `POST /api/runs/:id/settings` → persisted on the run + `host.setModel`). Codex: `skills/list` feeds the same palette and `/name args` becomes a `{type:'skill'}` input item in `codexHost.startTurn`. (2) Message queue — a message sent while a turn runs is a `run_turns` row with status `queued` (`submitSessionTurn` / `cancelQueuedTurn` in `core/runs/session.ts`; `DELETE /api/runs/:id/queue/:turnId` withdraws it); the next queued turn starts when the running one ends, Stop drops the queue, `recover.ts` marks leftovers stopped. (3) File rewind — sessions run with `enableFileCheckpointing`; the SDK user-message uuid is stored on the user turn (`run_turns.message_uuid`) and `POST /api/runs/:id/rewind {turnId}` resumes the live query if needed, dry-runs for the file list (a real rewind returns no counts) and restores the tracked files; the conversation is kept. Claude only. (4) `pnpm eval golden-from-reviews --project <id|name>` bootstraps a golden set from human-accepted / posted findings (`goldenFromReviews` in `core/eval/golden.ts`) — review the labels before trusting scores. Deliberately still out: message priorities, the override editor on the transparency page, Codex profile pools, Electron packaging.
- Agent configuration transparency: one provider at a time (Claude / Codex picker in the first block's header). `core/host/config.ts` probes Claude Code without running a turn (`initializationResult` / `mcpServerStatus` with tool annotations / `getContextUsage` with per-file, per-tool and per-skill tokens / `reloadPlugins` / the undeclared `getSettings` for layer names; 5-minute cache) and marks the disk scan of candidate files with what the CLI reports as loaded (this CLI version never loads AGENTS.md; nested rules count recursively). `core/codex/describe.ts` reads the live app-server (`config/read` layers, `mcpServerStatus/list` with tools, `skills/list`, `hooks/list`, `plugin/installed`, and an ephemeral MCP-less `thread/start` for `instructionSources`); the protocol has NO startup-context token figure, so the Codex tab shows instruction-file sizes instead. `/api/agent/config` (GET/PATCH), `app/pages/agent-config.vue`; MCP servers, commands and skills render through `app/components/CatalogList.vue` (grouped, filterable, collapsed by default, first sentence until a row is opened). Never call `app/list` / `plugin/list` for the page (multi-MB catalogues).
- Inbox (phase 3, 2026-08): `core/inbox/queries.ts` ← `/api/inbox` ← `app/pages/inbox.vue` lists what waits for the human (pending prompts, drafts with findings, author updates, runs that failed in the last 24 h, automation notes); the sidebar badge polls it every 30 s. Deep links: `/projects/:id?pr=<n>&review=<id>` opens the PR drawer, `useOpenGlobalSession()` opens the global chat drawer on a session.
- Smoke-testing a built server: `runtimeConfig` values are baked at build time, so `DB_PATH=… node .output/server/index.mjs` silently uses the production `data/cockpit.db`. Override with Nuxt's runtime names (`NUXT_DB_PATH`, `NUXT_REPOS_DIR`, `NUXT_WORKTREE_LOCATION=central`, `NUXT_AUTOMATION_ENABLED=false`) on a copy of the DB, and confirm with `lsof -p <pid> | grep .db` before running anything that talks to an agent or GitHub.
- The default worktree location is `.pr-cockpit-worktrees/<taskId>` inside each project's local clone; that directory is written into the target repo's `.git/info/exclude` (local only, not committed) — do **not** touch the target repo's shared `.gitignore`. That exclude line does not stop IDEs from discovering those worktrees — editors find repos by scanning the filesystem, not by reading gitignore/exclude; what actually decides discovery is the editor's own scan depth setting (in VS Code, `git.repositoryScanMaxDepth`, default 1, needs to be ≥2). On startup, recovery moves any still-existing old `./data/worktrees/<taskId>` persistent fix/feature worktrees over with `git worktree move`, and clears paths pointing at directories that are gone. Only `WORKTREE_LOCATION=central` keeps using `REPOS_DIR`.

## Constraints when changing code

- Don't trust old docs or Claude memory alone. This evolves fast — when judging behavior, read the current source, tests and recent commits first.
- The review agent's hard constraint is read-only: it may only review — it must not edit files, must not perform any git write, and must not perform any gh write. Posting comments externally must be done by the engine, after the user confirms.
- Provider-related changes must keep the Claude/Codex boundary clean. Project provider, model, effort and session id all have to follow the current provider; don't resume a Codex thread with a Claude session id, or vice versa.
- Codex code paths go through the host (`core/codex/codexHost.ts`) or its one-shot wrappers (`core/codex/oneshot.ts`); never spawn the binary yourself — binary resolution is `core/codex/bin.ts` and the production bundle has no `node_modules` copy of it.
- app-server `warning` / `configWarning` / `hook/*` notifications are non-fatal (`note` events); only a `turn/completed` with status `failed`, an `error` notification or a dead process fails a turn — don't mechanically throw on every notice.
- Fix / Feature / Global chat are the areas that write to worktrees or run commands. By default, don't let the agent push or run `gh pr create` on its own; for changes involving `allowDanger`, the network, or the danger guard, verify the current provider's real execution boundary case by case.
- PR automation is a high-risk area. Any live verification can trigger review/comment/fix/push — unless the user explicitly asks for it, don't run automation smoke tests against real PRs. Prefer unit tests and mocks.

## Frontend/UI conventions

- This is a workbench, not a landing page. The UI should be dense, scannable and lightly decorated, and should reuse the existing @nuxt/ui, lucide/iconify and local component styles.
- Don't nest a regular modal inside a drawer for confirmation; modal-on-drawer interactions have caused problems historically. Use the existing inline confirmation pattern, e.g. `useInlineConfirm`.
- When an async `load()` writes component state, guard against stale results: record a load token and the current id, and after the await confirm it is still the same entity before committing the value; SSE handlers must also check the id captured when they opened. Clear stale live/log/detail state when switching entities.
- Don't wrap operational screens in a narrow `max-w-2xl` / `max-w-3xl` without reason; tables, drawers, diffs and logs here should all use the horizontal space fully.

## Historical risk areas

- `post.post.ts` used to have a concurrency race when posting comments; the fix was a posting state plus CAS claiming. When touching review posting, recheck or automation dispatch, make sure that window hasn't been reopened.
- Automation has previously had problems such as push hot loops, fixing other people's PRs by default, and misjudged findings state. The current code has fixes for these, but the logic still has to be locked down by tests.
- The historical review verdict on the LAN remote access PR #51 was request changes: risks such as CSRF/Host/DNS rebinding/SSE/token exposure must not be assumed fixed unless the current code or a later PR clearly proves it.
- `runClaudeStream` defaults to a 20-minute idle timeout plus a 4-hour hard limit; unattended paths reusing it may need an explicitly shorter timeout.