CLAUDE.md - WorldOfTaxonomy

What this project is

WorldOfTaxonomy is a unified global industry classification knowledge graph. It connects 1000+ classification systems as equal peers through equivalence mappings (crosswalk edges).

1000+ systems, 1,305,191+ nodes, 326,375+ crosswalk edges.

Three surfaces: Web App (Next.js), REST API (FastAPI), MCP Server (stdio).

Architecture

WorldOfTaxonomy/
├── world_of_taxonomy/           # Python backend
│   ├── api/                     # FastAPI REST API
│   │   ├── app.py               # App factory with lifespan (pool management)
│   │   ├── deps.py              # DI: get_conn, get_current_user, validate_api_key
│   │   ├── schemas.py           # Pydantic models (system, node, auth, etc.)
│   │   ├── middleware.py        # Rate limiting (slowapi)
│   │   └── routers/
│   │       ├── systems.py       # GET /api/v1/systems, /api/v1/systems/{id}
│   │       ├── nodes.py         # GET .../nodes/{code}, /children, /ancestors
│   │       ├── search.py        # GET /api/v1/search?q=
│   │       ├── equivalences.py  # GET .../equivalences, /stats
│   │       └── auth.py          # POST register, login; GET /me; CRUD /keys
│   ├── mcp/                     # MCP server (stdio transport)
│   │   └── server.py
│   ├── ingest/                  # One ingester per classification system
│   │   ├── naics.py             # NAICS 2022 (Census Bureau CSV)
│   │   ├── isic.py              # ISIC Rev 4 (UN CSV)
│   │   ├── nace.py              # NACE Rev 2 (Eurostat XLSX)
│   │   ├── sic.py               # SIC 1987 (OSHA HTML + GitHub CSV fallback)
│   │   ├── anzsic.py            # ANZSIC 2006 (ABS XLS via xlrd)
│   │   ├── nic.py               # NIC 2008 (Indian govt PDF/CSV)
│   │   ├── jsic.py              # JSIC 2013 (skeleton: 20 divisions)
│   │   ├── nace_derived.py      # WZ 2008, ÖNACE 2008, NOGA 2008 (copy NACE nodes)
│   │   └── crosswalk.py         # ISIC↔NAICS concordance
│   ├── query.py                 # Core query functions (get_system, search, etc.)
│   ├── db.py                    # asyncpg pool (set statement_cache_size=0 behind pgbouncer)
│   ├── schema.sql               # Core tables: classification_system, classification_node, equivalence
│   ├── schema_auth.sql          # Auth tables: app_user, api_key, usage_log
│   └── __main__.py              # CLI: serve, mcp, ingest, init-auth
├── frontend/                    # Next.js 15 + TypeScript + Tailwind CSS + shadcn/ui
│   └── src/
│       ├── app/
│       │   ├── page.tsx         # Home: Industry Map + Galaxy View
│       │   ├── explore/page.tsx # Full-text search with ?q= param support
│       │   ├── system/[id]/page.tsx # System detail with sectors + crosswalks
│       │   ├── dashboard/page.tsx   # Stats overview + crosswalk matrix
│       │   ├── layout.tsx       # Root layout (Geist fonts, Providers)
│       │   └── globals.css      # shadcn/ui theme tokens (oklch), dark + light
│       ├── components/
│       │   ├── IndustryMap.tsx   # 22 industry sectors with icons → /explore?q=
│       │   ├── ThemeToggle.tsx   # Dark/light mode toggle (next-themes)
│       │   ├── Providers.tsx     # ThemeProvider + React Query
│       │   ├── visualizations/GalaxyView.tsx  # D3.js force simulation, animated
│       │   ├── layout/Header.tsx # Nav with systems dropdown
│       │   ├── layout/Footer.tsx
│       │   └── ui/              # shadcn/ui components
│       └── lib/
│           ├── api.ts           # Typed API client (all endpoints)
│           ├── types.ts         # TypeScript interfaces matching Pydantic models
│           └── colors.ts        # System tint colors + sector colors
├── tests/                       # pytest suite (18 files)
│   ├── conftest.py              # test_wot schema isolation, seed data, session pool
│   ├── test_api_*.py            # API contract tests (systems, nodes, search, equivalences)
│   ├── test_auth.py             # hashing, JWT, registration, API keys, usage log
│   ├── test_node_detail_contract.py  # node detail page API contract
│   ├── test_mcp_*.py            # MCP protocol + tool handlers
│   ├── test_ingest_*.py         # per-system ingester tests
│   └── test_cli.py              # CLI argument parsing
├── .env                         # DATABASE_URL, JWT_SECRET (not committed)
└── requirements.txt             # asyncpg, fastapi, uvicorn, bcrypt, PyJWT, slowapi, etc.

Tech stack

Backend: Python 3.9 · FastAPI · asyncpg · PostgreSQL 14+ (set statement_cache_size=0 when behind pgbouncer in transaction mode) · bcrypt · PyJWT · slowapi · MCP SDK

Frontend: Next.js 15 (App Router) · TypeScript · Tailwind CSS v4 · shadcn/ui · D3.js · React Query · next-themes · lucide-react

Database: PostgreSQL 14+ (provider-agnostic). Three core tables (classification_system, classification_node, equivalence) plus three auth tables (app_user, api_key, usage_log).

How to run

# Backend
source .env
python3 -m uvicorn world_of_taxonomy.api.app:create_app --factory --port 8000

# Frontend (requires Node.js - use nvm if npx not found)
cd frontend && npx next dev --port 3000

Frontend proxies /api/* to :8000 via next.config.ts rewrites.

How to test

# All tests (uses test_wot schema, never touches production)
python3 -m pytest tests/ -v

# Specific area
python3 -m pytest tests/test_auth.py -v
python3 -m pytest tests/test_ingest_naics.py -v

Test isolation: conftest.py creates a test_wot PostgreSQL schema, seeds NAICS/ISIC/SIC test data, and tears down after each test. Production data in public schema is never touched.

Development practices

• TDD - Red → Green → Refactor, strictly enforced:
  1. Red: Write the test first. Run it. Confirm it fails for the right reason before writing any implementation code.
  2. Green: Write the minimum code to make the test pass. Nothing more.
  3. Refactor: Clean up implementation and tests while keeping all tests green.
  • Never write implementation before a failing test exists.
  • Never skip the "run it red" step - a test that was never red proves nothing.
  • Never refactor while tests are red.
• No em-dashes: Never use the em-dash character (U+2014) anywhere in the project - code, comments, docstrings, markdown, or configuration. Use a hyphen - instead. The CI pipeline enforces this with a grep check.
• No speculative code: Don't add features, abstractions, or error handling beyond what's asked.
• Type safety: All frontend code is TypeScript. All backend models are Pydantic. Keep types.ts in sync with schemas.py.
• Theme support: Both dark and light modes must work. Galaxy View text uses SVG shadow filters for contrast in both themes.
• Test schema isolation: Tests MUST use the test_wot schema. Never run test queries against public.
• Verify from primary source, never from error text: Tool denial messages and error output are signals to investigate, not facts to relay. Before asserting what a config file contains, what a DATABASE_URL points at, what an env var is set to, what's in production, what a file or API returns, or any other environmental claim, read the actual file or run the actual introspection this turn. A sandbox denial that says "Production Reads" is a heuristic, not proof the target is prod. If a denial blocks the introspection, say so explicitly and stop; do not backfill the unknown with guesses from the error text.

Auth system

• Sign in: visit /login, enter email, click the one-time magic link. No password.
• Backend: POST /api/v1/developers/signup mints the link, GET /api/v1/auth/magic-callback?t=... consumes it and sets dev_session (httponly JWT, 60-min TTL) + wot_csrf (JS-readable double-submit token)
• Sign out: POST /api/v1/auth/logout clears both cookies
• API keys: wot_ / rwot_ / aix_ prefix + 32 hex chars, bcrypt-hashed, 8-char prefix-indexed. Mint and revoke from /developers/keys (cookie-gated, CSRF-protected)
• Rate limits: anonymous 30 req/min, authenticated 1000 req/min, plus per-IP guards on signup, magic-callback, classify/demo, search, contact, MCP, and key creation
• JWT secret: JWT_SECRET env var (must be ≥32 chars in production); used to sign the dev_session cookie

Key patterns

• Lifespan handler in app.py manages the asyncpg pool lifecycle
• NACE-derived systems (WZ, ÖNACE, NOGA) copy all NACE nodes and create 1:1 equivalence edges
• Next.js API proxy: client-side api.ts uses relative paths (/api/v1/...), next.config.ts rewrites them to the FastAPI backend
• Galaxy View: D3 force simulation in a React useEffect, with useTheme() for theme-aware rendering. Cleanup properly (stop simulation, cancel animation frame)
• IndustryMap: Links to /explore?q=<term>, Explore page reads useSearchParams() wrapped in <Suspense>

Wiki system (Karpathy LLM Wiki pattern)

Curated markdown files in wiki/ serve four channels from one source of truth:

1. Web pages at /guide/[slug] - server-rendered HTML with SEO metadata
2. MCP instructions - injected into the instructions field of the MCP initialize response
3. llms-full.txt - concatenated plain text for AI crawlers
4. Wiki API at GET /api/v1/wiki - JSON for developers and RAG pipelines

Key files:

• wiki/_meta.json - page metadata (slug, title, description, order)
• wiki/*.md - 10 curated guide pages
• world_of_taxonomy/wiki.py - Python wiki loader (load_wiki_meta, load_wiki_page, build_wiki_context, build_llms_full_txt)
• world_of_taxonomy/api/routers/wiki.py - Wiki API router
• frontend/src/lib/wiki.ts - Server-side wiki utilities for Next.js
• frontend/src/app/guide/ - Guide pages (index + [slug] dynamic route)
• scripts/build_llms_txt.py - Regenerates frontend/public/llms-full.txt from wiki content
• docs/diagrams/*.mmd - Mermaid diagram source files

When adding/modifying wiki content:

• Run python scripts/build_llms_txt.py to regenerate llms-full.txt
• Wiki files must not contain em-dashes (U+2014)
• Each page should have at least one ## heading and stay under 10K tokens

What's NOT done yet

• Node detail page ✓ done - /system/[id]/node/[code]/page.tsx with breadcrumb, typographic depth, children panel, cross-system equivalences, inline API endpoint
• Auth frontend pages (login, register, API key dashboard) - backend is complete
• Production deployment (Vercel for frontend, Fly.io/Railway for backend)
• CI/CD pipeline
• Domain-specific taxonomy extensions (ICD codes, crop taxonomies, etc.)