Working Approach

1. Think Before Coding

Don't assume. Don't hide confusion. Surface tradeoffs.

Before implementing:

• State your assumptions explicitly. If uncertain, ask.
• If multiple interpretations exist, present them - don't pick silently.
• If a simpler approach exists, say so. Push back when warranted.1
• If something is unclear, stop. Name what's confusing. Ask.

2. Simplicity First

Minimum code that solves the problem. Nothing speculative.

• No features beyond what was asked.
• No abstractions for single-use code.
• No "flexibility" or "configurability" that wasn't requested.
• No error handling for impossible scenarios.
• If you write 200 lines and it could be 50, rewrite it.

Ask yourself: "Would a senior engineer say this is overcomplicated?" If yes, simplify.

3. Surgical Changes

Touch only what you must. Clean up only your own mess.

When editing existing code:

• Don't "improve" adjacent code, comments, or formatting.
• Don't refactor things that aren't broken.
• Match existing style, even if you'd do it differently.
• If you notice unrelated dead code, mention it - don't delete it.

When your changes create orphans:

• Remove imports/variables/functions that YOUR changes made unused.
• Don't remove pre-existing dead code unless asked.

The test: Every changed line should trace directly to the user's request.

4. Goal-Driven Execution

Define success criteria. Loop until verified.

Transform tasks into verifiable goals:

• "Add validation" → "Write tests for invalid inputs, then make them pass"
• "Fix the bug" → "Write a test that reproduces it, then make it pass"
• "Refactor X" → "Ensure tests pass before and after"

For multi-step tasks, state a brief plan:

1. [Step] → verify: [check]
2. [Step] → verify: [check]
3. [Step] → verify: [check]

Equity Data Agent - Project Conventions

Core Philosophy: Intelligence vs. Math

This project enforces a hard separation between calculation and reasoning:

• Calculation Layer (Python/SQL): Handles 100% of the math
• Reasoning Layer (LLM): Interprets pre-computed results only

Non-Negotiable Rules

1. The LLM never does arithmetic. No RSI calculations, no percentage changes, no YoY growth in agent code. All math lives in Dagster assets or SQL.
2. The Agent never touches the database. It calls FastAPI endpoints via tools. If the agent needs data, it calls a tool that hits an API.
3. Three roles, no overlap:
   • Dagster = Worker (fetches and transforms data)
   • FastAPI = Interpreter (turns DB rows into readable reports)
   • LangGraph = Executive (reasons over reports, synthesizes theses)

Architecture

Data Sources → Dagster → ClickHouse/Qdrant → FastAPI → LangGraph Agent

• ClickHouse: equity_raw (ingested data) + equity_derived (computed indicators)
• Qdrant Cloud: Vector store for news embeddings
• All tables use ReplacingMergeTree for idempotency

Stack

• Python 3.12+, uv workspaces
• Dagster, FastAPI, LangGraph, ClickHouse, Qdrant Cloud
• LiteLLM Proxy (Ollama Cloud / Claude API)
• Langfuse for agent tracing

Repo Structure

Monorepo with 4 packages under packages/:

• shared - Pydantic schemas, config, ticker registry (the glue)
• dagster-pipelines - assets, sensors, schedules
• api - FastAPI endpoints
• agent - LangGraph agent

Code Style

• Lint: ruff check
• Format: ruff format
• Type check: pyright
• Test: pytest

Git Workflow

Branching

• One branch per Linear issue
• Use Linear's auto-generated branch names: noahwinsdev/qnt-XX-description

Issue Structure

Every Linear issue body follows this skeleton (QNT-293 / QNT-318 are the model). Match it exactly when creating or rewriting a ticket:

## Context          - why this exists; the incident/precedent that motivates it
## Scope            - what to build
## Out of scope     - (optional) what it deliberately doesn't do, and who owns that
## Acceptance Criteria
- [ ] AC1 (code) -- <verifiable statement>
- [ ] AC2 (dev execution) -- <needs a real command + output receipt>
- [ ] AC3 (docs) -- <statement>
## References        - related QNT tickets, ADRs, file paths

AC rules:

• Heading is ## Acceptance Criteria (not ## AC).
• Each AC is a numbered checkbox - - [ ] AC1, AC2, … - so it can be referenced as "AC4" in commits, PRs, and docs/project-plan.md.
• Each AC carries a class tag: (code) / (dev execution) / (prod execution) / (docs) (slash combos like (code/docs) allowed). The tag drives /sanity-check: dev/prod execution ACs need a literal command+output receipt; code/docs are verified by reading the implementation.
• Always set project: "Equity Data Agent" on create; keep bodies plain ASCII (Linear WAF). Updating a description can silently reset milestone - re-assert it.

Commit Format

QNT-XX: type(scope): description

Examples:

QNT-34: feat(shared): add Settings class with env-aware config
QNT-37: feat(db): add ClickHouse DDL migrations
QNT-41: fix(dagster): handle yfinance 429 with exponential backoff

Types: feat, fix, refactor, test, docs, chore

Pull Requests

• One PR per issue
• PR body must include Closes QNT-XX for Linear auto-close
• Squash merge to main
• Run make build locally before pushing if Dockerfile, docker-compose.yml, or any pyproject.toml was changed

Environment

• Dev: MacBook M4, ClickHouse via SSH tunnel (CLICKHOUSE_HOST=localhost)
• Prod: Hetzner CX41, Docker Compose (CLICKHOUSE_HOST=clickhouse)
• Switch via ENV=dev|prod in .env

Ticker Scope

10 US equities - defined in packages/shared/src/shared/tickers.py. Adding or removing a ticker touches four registry structures plus several backfill surfaces and an eval-golden sweep - follow docs/guides/ticker-lifecycle.md. Never hardcode ticker lists in business logic.

Working Docs

The docs/ folder is the shared brain for this project. Read relevant sections when starting work on a new area.

• docs/architecture/ - how the system works (read first)
• docs/patterns.md - established code recipes (read before implementing)
• docs/decisions/ - ADRs: why we chose X over Y (read when questioning a design choice)
• docs/guides/ - how to do common tasks
• docs/api/*.http - API test files (open with VS Code REST Client)

When making a significant architectural decision, create a new ADR using docs/decisions/TEMPLATE.md.

Observability

• Langfuse: Agent tracing - LLM calls, tool calls, latency
• Sentry: FastAPI error tracking in production
• Health Monitor: Cron on Hetzner (every 15 min) - checks API /health + Docker services, logs failures. make monitor-log to check. Session-start hook auto-warns on failures.
• Alerting: Two independent channels (QNT-101) - external uptime probe on /api/v1/health (UptimeRobot or equivalent, native Discord on free tier) + docker-events-notify.service on Hetzner streaming die/kill/oom/restart to a Discord webhook (alerts ≤30 s). Setup: docs/guides/uptime-monitoring.md. Self-monitored via heartbeat file + optional external heartbeat URL (Healthchecks.io free tier). Install: make events-notify-install. Test: make events-notify-test.
• Logs UI: Dozzle (QNT-103) tails every compose container at http://localhost:8082 via SSH tunnel - replaces the ssh hetzner "docker compose logs <svc>" workflow.
• Metrics & Trends: Prometheus (15d retention) + Grafana dashboards (QNT-103) - host CPU/mem/disk via node_exporter, per-container metrics via cAdvisor. Reach Grafana at http://localhost:3030 via SSH tunnel (admin/admin on first boot only - UI forces a password change on first login and the new password persists in the grafana_data volume). Alert rules (memory >80% per container, host mem >90%, restart loop, disk >80%) route to the same Discord webhook as QNT-101. Status: make obs-status. Synthetic alert test: make obs-alert-test.
• Ops Runbook: docs/guides/ops-runbook.md - failure-mode catalog with symptoms, diagnosis, response, and prevention. Grep this first when prod breaks; every Ops & Reliability ticket extends it.
• ClickHouse Play: http://localhost:8123/play - SQL editor for data exploration (via SSH tunnel)
• Dagster UI: http://localhost:3000 - asset lineage, run history, sensor status

Common Commands

Dev Commands (Makefile)

make setup                          # First-time: hooks + deps + .env
make dev-dagster                    # Start Dagster UI (terminal 1)
make dev-api                        # Start FastAPI (terminal 2)
make dev-litellm                    # Start LiteLLM proxy on :4000 (terminal 3, from Phase 5)
make dev-frontend                   # Start Next.js on localhost:3001 (terminal 4)
make tunnel                         # SSH tunnel to Hetzner ClickHouse (required for dev)
make test                           # Run pytest
make lint                           # ruff check + pyright
make format                         # Auto-format code
make migrate                        # Run ClickHouse DDL migrations (via HTTP)
make seed                           # Quick seed: 30 days, 3 tickers (fast dev data)
make build                          # Build prod Docker images locally (run when changing Dockerfile, docker-compose.yml, or deps)
make rollback                       # Rollback prod to previous commit and rebuild (emergency use)
make monitor-install                # Install health monitor cron on Hetzner (every 15 min)
make monitor-log                    # Show recent prod health failures
make events-notify-install          # Install docker-events → Discord notifier on Hetzner (QNT-101)
make events-notify-status           # Show notifier systemd status + last heartbeat
make events-notify-test             # Kill litellm to verify the Discord webhook path end-to-end
make obs-status                     # Observability stack health: prom targets + grafana ping + host headroom (QNT-103)
make obs-alert-test                 # Synthetic memory spike to verify Grafana → Discord alert routing (QNT-103)
make issue QNT=34                   # Checkout branch for Linear issue
make pr QNT=34 TITLE="description"  # Push + create PR

MCP servers (configured in .mcp.json)

• clickhouse - query the warehouse directly (read-only). Requires make tunnel running and CLICKHOUSE_USER/CLICKHOUSE_PASSWORD set in .env.
• github - read PRs/issues/Actions runs without prompts; write actions (create PR, comment, merge) prompt for approval. Requires GITHUB_PERSONAL_ACCESS_TOKEN in .env (run gh auth token to fetch).

Workflow Commands (Claude Code slash commands)

Session & Cycle

/session-check            # Full context restore: branch + Linear + AC status
/status                   # Quick glance: branch, commits, uncommitted (no API calls)
/cycle-start              # Start of week: show cycle issues, check plan staleness, suggest next pick
/cycle-end                # End of week: summarize shipped, roll over incomplete, prompt retro if milestone done
/retro                    # End of milestone: review, capture lessons, review upcoming phases, sync docs

Issue Lifecycle

/go QNT-34                # Full orchestrator: pick → implement → sanity-check → review → ship
/pick QNT-34              # Start an issue: checkout branch + Linear In Progress + show acceptance criteria
/implement QNT-34         # Implement: read patterns → write code → WIP commits → targeted tests → AC validation
/sanity-check QNT-34      # Pre-PR gate: lint + types + tests + AC (code/dev/prod classification) → In Review
/review QNT-34            # Adversarial code review: logic errors, security, architecture, edge cases
/ship QNT-34              # Ship: squash WIPs → tick project-plan.md → PR → CI → merge → post-deploy verify → Done
/fix QNT-34               # Error recovery: diagnose failure → fix → resume pipeline from failed step
/sync-linear QNT-34       # Manual override: sync issue status when Linear has drifted

Docs & Scope

/change-scope             # Formalise a requirement change (add/drop/modify): spec + Linear + ADR if warranted
/sync-docs                # Reconcile project-plan.md with Linear: tick Done, remove Cancelled, surface gaps

Ops

/server-audit             # Audit Hetzner prod: durability / host / security / drift. Proposes Linear tickets for gaps, files on approval.

Hooks (Automatic - no manual invocation)

Configured in .claude/settings.json, scripts in .claude/hooks/: