flight-finder

# CLAUDE.md — Flight Finder

> **Flight Finder** — The price trail airlines don't show you. Flight price evolution tracker with natural language search and shareable charts.

## Tech Stack

| Layer | Technology |
|-------|------------|
| Frontend | Next.js 15+ (App Router), TypeScript, CSS Modules |
| Database | PostgreSQL 16 + Prisma ORM |
| Cache | Redis 7 (rate limiting + response caching) |
| AI | Anthropic Claude, OpenAI GPT, Google Gemini, Claude Code CLI, Ollama, llama.cpp, vLLM |
| Browser | Playwright (headless Chromium for Google Flights scraping) |
| Charts | Plotly.js (interactive price evolution) |
| Hosting | Hetzner VPS (Docker Compose + Caddy) — flight-finder.org |
| CI/CD | GitHub Actions (CI + Deploy on push to main) |

## Monorepo

npm workspaces: `@flight-finder/web` (`apps/web/`), `@flight-finder/cli` (`packages/cli/`).
Root `package.json` proxies common scripts to `@flight-finder/web`. `apps/desktop/` is a Tauri (Rust) launcher: deliberately NOT an npm workspace member and excluded from `npm run ci`. `.github/workflows/desktop-ci.yml` tests it on macOS, Linux, and Windows; `.github/workflows/desktop-release.yml` builds the installers after the required checks pass.

The CLI bundles the shared scraper (`apps/web/src/lib/scraper/*`) via relative imports but maps `@/*` to its own `packages/cli/src/`, so any new `@/lib/<x>` import added to a shared scraper file needs a matching shim in `packages/cli/src/lib/` (re-export the real module like `secret-crypto.ts`/`prisma.ts`, or a stub like `admin-recovery.ts`). Web-only checks pass without it; only the full `npm run ci` (web + cli) catches a missing shim.

**Versioning (locked):** `apps/web`, the root `package.json`, `packages/cli`, and `apps/desktop` all carry the SAME version number. `apps/web/package.json` is the source of truth; git tags `vX.Y.Z` track the web release and `desktop-vX.Y.Z` tags the desktop build at the same number (distinct prefixes, no collision). `/create-release` must bump all of them to the new version — `apps/web/package.json`, root `package.json`, `packages/cli/package.json`, and `apps/desktop` (its `package.json`, `src-tauri/Cargo.toml`, and `src-tauri/tauri.conf.json`) — and regenerate both lockfiles (`package-lock.json` and `apps/desktop/src-tauri/Cargo.lock`).

## Environment Variables

Supply secrets through the caller's environment. Never use `.env` files or commit credentials.
Keep shared scripts independent of secret-manager accounts. Operators can wrap commands externally with their preferred secret manager.

Critical: `DATABASE_URL`, `REDIS_URL`, `ADMIN_SESSION_SECRET`, and `CRON_SECRET`.

## Build Commands

```bash
npm install                    # All workspaces
docker compose -f docker-compose.prod.yml up -d db redis
npm run db:push                # schema plus shared travel constraints
npx prisma generate --schema=apps/web/prisma/schema.prisma
npm run dev                    # Web app on :3003 (next dev --port 3003)
npm run ci                     # lint + typecheck + test + build (both web and cli workspaces)
```

## File Index

### `apps/web/src/app/` — Pages & API routes

| Path | Purpose |
|------|---------|
| `page.tsx` | Landing page — natural language search bar |
| `layout.tsx` | Root layout — fonts, metadata |
| `q/[id]/page.tsx` | Public shareable chart page (no auth) |
| `admin/(dashboard)/page.tsx` | Admin dashboard — active queries, costs |
| `admin/(dashboard)/queries/page.tsx` | Query management — pause/resume/delete/reassign |
| `admin/(dashboard)/config/page.tsx` | LLM agent config — provider/model selection |
| `admin/(dashboard)/notifications/page.tsx` | Notification channels + new-low alert thresholds |
| `admin/(dashboard)/users/page.tsx` | User management (multi user mode only) — create/reset/delete |
| `login/page.tsx` | Unified login (multi user mode only) — admin + non admin |
| `account/page.tsx` | Logged in user's tracker list (multi user mode only) |
| `account/settings/page.tsx` | Per user preferences — currency, country, airlines, cabin |
| `api/parse/route.ts` | POST — LLM parses natural language flight query |
| `api/queries/route.ts` | POST — create new tracked query (401 anon in multi user mode) |
| `api/queries/[id]/route.ts` | PATCH/DELETE — update or delete a query by deleteToken or user session |
| `api/queries/[id]/prices/route.ts` | GET — public price data for chart |
| `api/queries/[id]/scrape/route.ts` | POST — force-scrape a single query immediately (deleteToken or session auth) |
| `api/queries/active/route.ts` | GET — active non-seed queries for the current user (scoped in multi user mode) |
| `api/queries/status/route.ts` | POST — batch-check active/expired status for up to 20 query IDs |
| `api/cron/scrape/route.ts` | GET — trigger scrape run (CRON_SECRET auth) |
| `api/auth/login/route.ts` | POST — user login (multi user mode only); rate limited |
| `api/auth/logout/route.ts` | POST — clears the shared ft-session cookie |
| `api/auth/me/route.ts` | GET — current user; 401/404 outside multi user mode |
| `api/admin/auth/logout/route.ts` | POST — admin logout |
| `api/admin/queries/route.ts` | GET — list all queries |
| `api/admin/queries/[id]/route.ts` | PATCH/DELETE — manage query; PATCH accepts userId reassignment |
| `api/admin/config/route.ts` | GET/PATCH — extraction config (exposes isSelfHosted) |
| `api/admin/multi-user/route.ts` | POST — enable multi user mode atomically (creates first admin, backfills); DELETE — disable (admin only, clears admin hash) |
| `api/admin/users/route.ts` | GET/POST — list/create users (admin only) |
| `api/admin/users/[id]/route.ts` | PATCH/DELETE — reset password, toggle isAdmin, delete |
| `api/admin/notifications/route.ts` | GET/POST — list/create global notification channels (admin) |
| `api/admin/notifications/[id]/route.ts` | PATCH/DELETE — update/toggle/delete a channel |
| `api/admin/notifications/[id]/test/route.ts` | POST — send a test notification (rate limited) |
| `api/admin/insights/route.ts` | GET — admin-only airline reliability stats (cached 5 min) |
| `api/admin/analytics/aggregate/route.ts` | POST — aggregate daily page-view events and clean up old raw data (admin only) |
| `api/admin/analytics/query/route.ts` | GET — query aggregated analytics (page views, devices, countries, referrers, etc.) (admin only) |
| `api/admin/seed-routes/route.ts` | GET/POST — list or create seed demo queries (admin only) |
| `api/admin/seed-routes/[id]/route.ts` | PATCH/DELETE — update or delete a seed query (admin only) |
| `api/admin/local-models/route.ts` | GET — probe local LLM servers (Ollama, llama.cpp, vLLM) and return available model IDs (admin only) |
| `api/admin/providers/route.ts` | GET — list all LLM providers with readiness status (admin only) |
| `api/account/settings/route.ts` | GET/PATCH — current user's preferences |
| `api/alerts/route.ts` | GET — active queries with their current low-price alert state (scoped in multi user mode) |
| `api/airports/route.ts` | GET — airport autocomplete search against bundled IATA dataset |
| `api/analytics/track/route.ts` | POST — internal analytics write endpoint; middleware calls fire-and-forget; gated by ADMIN_SESSION_SECRET |
| `api/analytics/event/route.ts` | POST — record a page-engagement event (scroll depth, dwell time) from the client |
| `api/community/register/route.ts` | POST — mint a community API key (rate limited; COMMUNITY_REGISTRATION_OPEN must be true) |
| `api/community/ingest/route.ts` | POST — accept a batch of price snapshots from a registered community node (API key auth) |
| `api/community/routes/route.ts` | GET — list all routes that have community snapshot data (cached 5 min) |
| `api/community/routes/[route]/route.ts` | GET — price history for one community route, formatted as ORIGIN-DESTINATION |
| `api/preview/route.ts` | POST — start a create-time preview scrape; returns a PreviewRun ID |
| `api/preview/[id]/route.ts` | GET — poll a preview run for status and results |
| `api/setup/route.ts` | POST — one-time setup: store admin password and initial provider config (403 after first run) |
| `api/setup/status/route.ts` | GET — setup completion state and detected LLM providers (used by setup wizard) |
| `api/stats/route.ts` | GET — public instance stats (active queries, total scrapes, price points, cron info) |
| `api/version/route.ts` | GET — current version, commit SHA, and whether an update is available |
| `api/test/scrape/route.ts` | GET — smoke test for extraction pipeline: checks DB, Chromium, and LLM (CRON_SECRET auth) |
| `api/vpn/status/route.ts` | GET — VPN provider config and whether the sidecar container is reachable |
| `api/health/route.ts` | GET — health check (DB + Redis) |

### `apps/web/src/components/` — UI components

| Component | Purpose |
|-----------|---------|
| `SearchBar` | Natural language flight query input with syntax highlighting |
| `ConfirmationCard` | Parsed query display with "Track this flight" button |
| `PriceChart` | Plotly.js wrapper — price evolution, airline colors, click→book |
| `BestPrice` | Highlight card for cheapest price found |
| `PriceHistory` | Table with trend arrows and booking links |

### `apps/web/src/lib/` — Core logic

| File | Purpose |
|------|---------|
| `prisma.ts` | Prisma client singleton |
| `redis.ts` | Redis client + cache helpers |
| `api-response.ts` | `apiSuccess()`/`apiError()` response helpers |
| `sidedoor/` | Shared password, passkey, invitation, session, provider, job, and telemetry integration |
| `user-auth.ts` | User session tokens, parseSession discriminated union, getCurrentUser (DB-backed) |
| `multi-user.ts` | `isMultiUserEnabled()` (hard gated on SELF_HOSTED, authoritative DB read for authorization) |
| `rate-limit.ts` | Redis backed login throttling (5 per 15 min per IP+username) |
| `password.ts` | scrypt hashing and verification |
| `secret-crypto.ts` | AES-256-GCM encrypt/decrypt for secrets at rest (keyed on ADMIN_SESSION_SECRET) |
| `notifications/` | New-low alert detection + dispatch + pluggable channel senders (Telegram, email, ntfy, webhook) |

### `apps/web/src/lib/scraper/` — Extraction pipeline

| File | Purpose |
|------|---------|
| `ai-registry.ts` | Provider registry (Anthropic, OpenAI, Google, Claude Code) |
| `parse-query.ts` | LLM parses natural language into structured flight query |
| `navigate.ts` | Playwright navigates Google Flights, captures HTML |
| `extract-prices.ts` | LLM extracts structured price data from page |
| `run-scrape.ts` | Orchestrates full scrape run across active queries |

## Prisma Schema

Models:
- `Query` (tracked flights, optional `userId` owner)
- `PriceSnapshot` (price data points; optional `vpnCountry` for VPN comparison runs)
- `FetchRun` (scrape run logs; optional `vpnCountry`)
- `ExtractionConfig` (application settings singleton; setup state, provider selection, `multiUserMode`, RPM caps, and preview concurrency fields. Sidedoor owns access and encrypted provider credentials.)
- `ApiUsageLog` (cost tracking per provider/model)
- `User` (product profile data for self-hosted accounts; Sidedoor owns credentials and session revocation)
- `NotificationChannel` (per-channel notification config; nullable `userId` for admin-owned global channels)
- `PreviewRun` (create-time preview scrape job; `clientIp` for per-IP concurrency cap)
- `CommunityApiKey` (registered community node keys; tracks snapshot count and last-seen time)
- `CommunitySnapshot` (anonymized price snapshots ingested from community nodes)

## Design System: "Altitude"

Supports light/dark themes via `data-theme` attribute on `<html>`.

**Dark (default):** bg `#031820`, surface `#072530`, elevated `#0e3640`, border `#1a4a52`, accent `#80a8a5` (mid teal), text `#ecdfc0` (warm cream), secondary `#d4a574` (muted gold).
**Light (basic-light):** bg `#faf6ed` (cream), surface `#f1ead9`, elevated `#e8dfc8`, border `#d6cbae`, accent `#1a4a52` (deep teal), text `#031820`.
**Price up alert (shared):** `#c1272d` scarlet. Reserved for emphasis on rising prices, alerts, and editorial callouts.

Fonts: Bricolage Grotesque (display), Outfit (body), IBM Plex Mono (data).

Vintage travel poster aesthetic. Deep teal and cream with a scarlet alert accent. Pan Am and French Line ocean liner heritage. No amber primary, kept out to steer clear of the AI tool palette.

Other themes (cyberpunk, tron, autumn, solar-red) remain as user-selectable alternates in `theme.ts`.

## Scraping Constraints

- **Rate limit:** Google returns HTTP 429 after ~30 sustained requests from the same IP. The default 3h cron interval stays well under this.
- **RT pricing:** Google Flights shows the full round-trip price on each flight result. The extraction prompt accounts for this -- do not sum outbound + return prices.
- **Google internal API:** undocumented endpoints exist (`GetShoppingResults`, `GetCalendarGraph`, `GetExploreDestinations`) but lack booking URLs, currency control, fare classes, and seat counts. We use Playwright for data completeness. See README comparison.

## Engineering Patterns

- **Component**: `Name.tsx` + `Name.module.css`. Named export, `styles.root`.
- **API Route**: Validate → query → `NextResponse.json()` with `apiSuccess()`/`apiError()`.
- **Scraper**: Playwright navigate → capture HTML → LLM extract → store snapshots.
- **Provider credentials**: configure them through setup or Settings. `resolveProviderCredentials()` reads the encrypted Sidedoor vault. Provider execution never reads API provider credentials from deployment environment variables.
- **Access**: Sidedoor owns password, passkey, session, device, invitation, recovery, and household profile authority. Flight Finder stores product profile fields in the same database transaction.

## DO

- Use CSS Modules for all styling
- Use TypeScript strict mode
- Use Server Components by default
- Return proper HTTP status codes
- Cache API responses in Redis (5min TTL)
- Read secrets from the supplied environment without hardcoded secret-manager wrappers

## Pre-Release Gate (MANDATORY before `/create-release`)

All four tests must pass before tagging a release:

```bash
./scripts/docker-smoke-test.sh    # Docker infra: build, health, chromium, extraction, DB
./scripts/install-flow-test.sh    # Static + grep regression checks on install.sh / flight-finder-cli
./scripts/cli-runtime-test.sh     # Behavioral CLI runtime matrix (docker v1/v2, podman compose, podman-compose)
./scripts/migration-test.sh       # Static checks on ~/.flight-finder to ~/.flight-finder migration + deprecated alias
```

If any fails, fix the issue and re-run. Do NOT tag without all four passing.

The release commit must bump every package to the same new version — `apps/web/package.json`, the root `package.json`, `packages/cli/package.json`, and `apps/desktop` (`package.json`, `src-tauri/Cargo.toml`, `src-tauri/tauri.conf.json`) — and regenerate both lockfiles (`package-lock.json` and `apps/desktop/src-tauri/Cargo.lock`), so every version stays accurate for tooling that reads them.

The runtime-matrix harness (`cli-runtime-test.sh`) is what catches the
"works on docker, broken on podman" class of bug (issues #62, #72). It
shims `docker`/`podman`/`*compose`/`curl` and asserts the recorded
invocations — every CLI command across every compose flavor.

## DON'T

- Use Tailwind, inline styles, or styled-components
- Use `any` type
- Use `.env` files. Supply secrets through the caller environment.
- Commit API keys or secrets