security-audit

A skeptical, evidence-first review of the whole system — the rift-ds package, the Next.js website, the AI chat layer, the CI/release pipeline, and the analytics/privacy posture. Every claim is confirmed against source or a live request before it reaches the report; nothing is taken on trust.

When invoked

Run when asked to "run the security audit" (/security-audit), or to check how secure/private the site is.

Scope guardrails (read first)

• Report only, by default. This skill audits and reports; it does not change code. Applying fixes is a separate, explicit request. Say what is wrong and how to fix it, not "I fixed it."
• Read-only and non-destructive. Scans, reads, and a small, bounded set of live requests. No state-changing calls to any service.
• Live probing is light and bounded. A handful of requests to each production model route, /api/chat, /api/table-command, /api/merchant-chat and /api/merchant-chat/starters among them, read fresh from the routes directory (injection, rate-limit, oversized-body, malformed-body). If the deployment is unreachable, skip the live probes and note the gap in the report. Stop the moment rate limiting kicks in — being throttled is the positive result, not a reason to push harder. Each request bills real API tokens, so keep the count low.
• A clean pass is a valid outcome. If a category has nothing worth flagging, say so and credit what is already strong. Never invent a finding to have something to report.
• Evidence or it doesn't ship. Every finding needs a file:line, a command result, or a captured live response. Kill false positives in a verification pass before writing anything up.

The sweep

Work the surfaces below. The list says where to look, not what is there — read everything fresh each run, because the code moves.

1. Dependencies & secrets (automated)

• npm audit --json at the repo root (the single lockfile covers the website workspace too; run it in website/ as well to be sure). Capture vuln counts and whether fixes exist.
• Secret scan of the working tree — secretlint with the recommended preset (write its config to a scratch dir, not the repo), or an equivalent scanner if one is installed.
• Secret sweep of the full git history (git log -p --all) for key patterns (sk-ant-, ghp_, github_pat_, AIza…, PEM headers, AWS keys) — check the repo's visibility first (gh repo view --json visibility); a public repo's history matters as much as its tree, and it is worth sweeping either way. Confirm no .env file was ever committed (git log --all --diff-filter=A --name-only).
• Probe for the scanner binaries (gitleaks, trufflehog, osv-scanner, semgrep, trivy) and Docker before planning the sweep rather than assuming either way — on the Windows checkout they are generally absent. Where they are missing, lean on npm audit, npx secretlint, and git history greps, which need no install, and offer to download a standalone scanner only with permission.

2. API routes & the AI chat (OWASP LLM Top 10)

The routes live under website/src/app/api/ — read the directory fresh rather than working from a remembered list, it grows. The chat routes are thin callers of the package: createChatHandler and its siblings in src/chat/server.ts own the body, turn and message caps and the event protocol, src/chat/upstash.ts owns the rate limits, the spend circuit breaker, the fail-open path, the exchange log and the salted visitor hash, and website/src/app/api/chat/guardrails.ts only configures them; the system-prompt boundary is website/src/app/api/chat/persona.ts. Model output is rendered by src/chat/AssistantTurn.tsx (react-markdown, with isInternalHref deciding which links stay in-app), which is where the insecure-output check lands. It has a second path: a question set the model asks through the opt-in ask_user tool is drawn by src/chat/TurnQuestion.tsx as a QuestionCard, after sanitizeQuestionSet in src/chat/questions.ts bounds it on the server and again in the hook (at most MAX_QUESTIONS questions, two to MAX_QUESTION_OPTIONS options, every label inside SUGGESTION_MAX_CHARS, plain strings only, and asking ends the exchange with no further model call). Check those bounds still hold, and that a visitor's "Something else" text reaches the model only as the plain-text user turn serializeReply writes. The same handlers ship to every consumer, and npx rift-ds init chat writes copies of the routes and an env example from src/cli/templates/chat/ into consumer apps, so a finding in the package is a finding in every consumer's app, and the sweep is not confined to website/src/app/api/. The /api/mcp route is a different animal: it calls no model, so the LLM Top 10 does not apply — check instead that every tool still reads only generated, already-published data, that tool arguments stay bounded, and that the deliberate absence of auth and rate limiting (recorded in that doc block) still holds up against what the tools now cost. The tool implementations themselves live outside the routes directory, in website/src/lib/site-tools.ts — the shared module behind both the MCP registrations and the chat's model-invokable tools, whose own doc block asserts the read-only-published boundary; a sweep confined to app/api/ never opens it, so open it. The merchant sales template's two routes are the other exception to "thin callers": api/merchant-chat/route.ts holds its own system prompt and its refusal, and takes sanitiseTableContext from website/src/lib/merchant-chat-context.ts, which rebuilds browser-sent table state before it lands in the prompt, and its tools, like api/table-command's, live in website/src/lib/order-ledger-tools.ts, also outside the routes directory. Its set_table and reset_table are a third path for model output: their table-view cards are applied to the live table by website/src/components/templates/MerchantSales/MerchantChat.tsx, so check viewProblems bounds the change and only that kind is applied. Check:

• Input validation & limits — body shape, per-message and total body size, turn count. A cap on only the last message leaves earlier history unbounded.
• Rate limiting & cost control — the per-IP, daily, and spend guardrails, and crucially their failure mode (fail-open vs fail-closed) when the store is unreachable.
• Prompt-injection defense — the persona's refusal clause, and the architectural containment: the chat's knowledge is the build-generated corpus (website/src/data/site-corpus.generated.ts) plus the two lookup tools over the generated prop and token data, all three generated from already-published sources (the third model-invokable tool, ask_user, reads nothing: it only returns answer options to the visitor), so a jailbreak should yield nothing private. Its request carries one thing more from the browser than the message: the page and what the visitor highlighted on it with what it sits in, which lands in a system block. Verify website/src/lib/site-chat-context.ts rebuilds it rather than trusting it: the path through sanitisePath, the highlight through the package's sanitiseSelection (known fields only, each capped), every word quoted by selectionLines and named as the site's own only when it appears in the published corpus, and the facts beside it (a prop, a token, a component) looked up in the generated data, never taken from the browser. The merchant sales chat's knowledge is narrower still: its prompt's page facts plus the seeded fictional ledger through order-ledger-tools.ts. But its request carries more from the browser than a message: the table's setup, the reader's focus (ticked ids, or highlighted page text with its source), the edits the page holds for the visit, a change log and a card-confirmation mark, in a context of up to 2 MB. Verify each is rebuilt rather than trusted (website/src/lib/merchant-chat-context.ts, sanitiseEdits in website/src/lib/order-actions.ts): ids must be real orders, fields only values an action could write, highlighted text cut to length and framed as data, and a confirmation honoured only from its mark, never from the words "yes, cancel". Its one write tool, propose_order_action, must stay propose-only: nothing on the server changes an order, and the page applies a change only on the reader's Confirm. Verify the tool layer too (website/src/lib/site-tools.ts — its doc block asserts the boundary), not just the corpus.
• Insecure output handling — how model output is rendered. Markdown-only with an internal-path link allowlist is the safe pattern; raw HTML (dangerouslySetInnerHTML, rehype-raw) is the thing to hunt for.
• PII / retention — what the exchange log stores and for how long, whether it is disclosed, and whether visitor keys are truly anonymised (an unsalted IP hash is reversible).

3. HTTP hardening (verify against the live site)

Read website/next.config.ts for the CSP and headers, then confirm against what the server actually sends (curl -sI the live domain — headers can be edge-supplied and differ from source). Look for: a CSP without 'unsafe-inline'/'unsafe-eval', Strict-Transport-Security, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy, base-uri/form-action, and whether X-Powered-By leaks the framework (poweredByHeader). Note any route that fetches a third-party URL without a timeout or runtime validation.

4. CI, release & supply chain

Read everything under .github/ — the workflows and dependabot.yml, which lives beside the workflows directory, not in it. Check each workflow for a least-privilege permissions: block (its absence hands builds the repo-default token scope) and whether actions are pinned to SHAs vs movable tags. For dependency-scanning automation, know that a scanner can exist with no file at all: check whether CodeQL is enabled for this repo (gh api the code-scanning default-setup endpoint, the Security tab, or gh run list --branch main --json workflowName) — do not assume it either way; a files-only sweep reports a false gap when default setup is on, and a remembered "it runs here" reports a false pass when it is off. Credit the release workflow's posture (OIDC Trusted Publishing, provenance, pre-publish consumer smoke test) where it holds.

5. Analytics & privacy

Read the analytics embed in website/src/app/layout.tsx. Check whether tracking scripts load before consent, whether a consent banner / consent-mode default exists, whether a privacy policy page exists, and what other third parties load at runtime (fonts, embeds, CDNs). For a public, internationally-reachable site this is usually the highest-impact category even though it is compliance, not exploitation.

6. Existing self-defenses (credit them)

The repo has build-time guards worth verifying and crediting: the corpus leak validator (scripts/validate-site-corpus.mjs), the component-API leak validator (scripts/validate-component-api.mjs — it screens the prop JSDoc served by /api/mcp, and a hit there is a leak in the published npm tarball too), and the token/CSS directive validators. Note where a good guard has a coverage gap (e.g. it scans the corpus but not the other files sent to the model as system prompts).

Verify, then report

Run a verification pass over every candidate finding: confirm it against the current source or a fresh live request, and downgrade or drop anything the scans contradict (a clean npm audit turns "no dependency scanning" from an exposure into a process note). Rank what survives by real-world impact, not theoretical severity.

Then write the report:

• A private Artifact page (default-private — this is security content; do not commit it to the public repo or publish it publicly), plus a markdown copy in the scratchpad.
• Lead with a plain-English executive summary for a non-expert owner: the overall posture, the two or three things that actually matter, and what is merely cosmetic hardening.
• Each finding: what it is, the evidence (file:line or captured response), the realistic impact, and a concrete suggested fix — described, not applied.
• A "what's strong" section. An honest audit says what is good, not only what is wrong.
• A methodology/scope footer and the standards referenced (OWASP LLM Top 10, Next.js/Vercel hardening checklists).

Never state a count of findings from a past run as if it were fixed truth — each run counts its own. Cite paths that exist; the build's validate-doc-refs check holds this file to real repo paths.