MITM Proxy

The MITM proxy is the network engine's HTTPS interception boundary. It intercepts traffic from the air-gapped guest VM, normalizes it into typed facts, hands a SecurityEvent to the security engine, and preserves allowed runtime bytes for upstream. Treat it as a system, not a collection of hacks -- every capability must be general-purpose.

Security boundary

Network code parses transport bytes, routes traffic, and emits typed SecurityEvent facts. It must not broker credentials, create credential refs, run CEL/security decisions, or sanitize ledger projections. Those belong to the security engine plugin rail. Every security plugin has the same data contract: it receives a SecurityEvent and returns a SecurityEvent; the plugin stage only controls ordering (preprocess, postprocess, or logging).

There are two materialization paths and they must never be collapsed:

• Runtime materialization prepares bytes for the real upstream. It may resolve a broker ref back to a real credential because the protocol needs it.
• Ledger materialization prepares the event stored in session.db, structured logs, route JSON, and UI stats. It must contain only broker refs, hashes, bounded previews, typed detections, and plugin execution evidence.

No credential logic belongs in HTTP header formatters, DB readers, frontend transforms, debug harnesses, or route adapters. If a future change needs capture, injection, redaction, threat intel, or PII handling, implement it as a security plugin stage over SecurityEvent -> SecurityEvent.

Pipeline

Guest curl -> iptables REDIRECT -> capsem-net-proxy (guest, port 10443)
  -> vsock port 5002 -> Host MITM proxy
  -> SNI parse -> network metadata capture
  -> TLS terminate (rustls, per-domain cert minted from Capsem CA)
  -> HTTP request parse (hyper)
  -> SecurityEvent -> SecurityRuleSet + plugin rail
  -> Forward to real upstream over TLS
  -> Record telemetry to session DB
  -> Stream response back to guest

Key source files

Cross-process network limits

config/data/runtime-limits.toml owns the supported concurrent network-client ceiling and its required multiples. capsem-proto/build.rs validates that file and generates the shared constants. DNS session capacity equals the client ceiling. HTTP descriptor-grant capacity multiplies it by traffic_grants_per_client because a pool can hold active and idle connections while a retiring connection is still closing. The process descriptor floor multiplies it by file_descriptors_per_client and must cover both ends of every overlapping grant. The ledger producer queue multiplies the client ceiling by producer_events_per_network_client; its admission batch must divide that queue exactly. admission_coalesce_wait_microseconds gives a lone audit event a bounded chance to join its immediate followers before the logger performs the commitment and admission IPC exchanges.

Never add a numeric connection, ledger queue, batch or coalescing ceiling to the proxy, service broker, VM owner, logger, core DNS session, or guest DNS forwarder. A limit duplicated across those processes can make one stage admit work that the next stage sheds, which previously produced intermittent HTTP 502 and DNS SERVFAIL responses under load. The guest relay, VM owner, service broker and confined proxy must call capsem_foundation::unix::process::ensure_open_file_limit with the generated floor before accepting traffic; raising only one process leaves the same cliff at the next hop. Change the config-owned values, exercise every consumer through the generated constants, and run the official HTTPS and DNS load benchmarks twice before and twice after the change. Any traffic error or capacity refusal fails the run. User overrides belong in the typed settings contract tracked by issue #349. Extend and run tests/citadel/test_runtime_limit_authority.py whenever a new consumer or derived limit is added.

The same file's [cache] table owns DNS answer count and TTLs plus the TLS leaf count. Use its generated capsem_proto::CACHE_* constants; do not place a second cache bound in the proxy or DNS modules.

Use just bench mitm-scale when a one-VM curve plateaus while the host still looks idle. The lane uses fresh 1/2/4/8-VM cohorts, keeps per-VM work fixed and records the mock upstream outside the Capsem process tree. Compare the curve to physical cores, not only logical CPUs: after the VM owners, confined proxies and ledgers consume the physical-core budget, SMT contention is expected to raise per-VM latency. A busy fixture, request error or hidden descriptor refusal invalidates the run; do not explain it as product saturation.

Content-Encoding: the systemic rule

The proxy MUST handle response decompression as a general capability. This is not optional, not per-feature.

1. Normalize Accept-Encoding in outgoing requests to only allow encodings we can decompress (gzip at minimum)
2. Transparently decompress response bodies before any parsing (SSE, body preview, telemetry)
3. Never strip encoding headers as a workaround -- that breaks upstream behavior

Why this matters: Failing to handle gzip on Anthropic SSE responses caused all model/token/cost metadata to be NULL. The SSE parser received compressed garbage. This went undetected because Google's API happened to not compress SSE in testing. The fix was general-purpose decompression, not an Anthropic-specific hack.

Serde optimization for ai_traffic parsers

The ai_traffic parsers (openai.rs, google.rs, request_parser.rs) deserialize LLM request/response bodies that can be megabytes. Never use serde_json::Value for struct fields that hold large unconstrained JSON (tool call args, function responses, model outputs). Use Box<serde_json::value::RawValue> for fields that are only stringified, and remove unused fields entirely. See /dev-rust-patterns for the full pattern and examples.

SSE parsing

AI provider APIs (Anthropic, OpenAI, Google) use Server-Sent Events for streaming responses. The proxy parses SSE to extract model names, token counts, and cost data for telemetry.

SSE parsing happens AFTER decompression. The body must be plaintext UTF-8 by the time the SSE parser sees it.

model_calls filtering

Only emit model_calls telemetry for actual LLM API paths (e.g., /v1/messages, /v1/chat/completions), not every request to an AI provider domain. Health checks, auth endpoints, and static assets should not create model_call rows.

Security evaluation order

1. Network mechanics parse and normalize SNI, HTTP, DNS, model, and process facts into a SecurityEvent.
2. Built-in default, settings, and corp rules compile into one SecurityRuleSet; built-in defaults are normal late-priority rules.
3. Security plugins run by stage over the same SecurityEvent object: preprocess, rule evaluation, postprocess, then logging before ledger handoff.
4. Runtime materialization forwards allowed bytes upstream. Ledger materialization writes the sanitized/enriched event to session.db, logs, routes, and UI stats.

Certificate authority

• Static CA keypair: crates/capsem-core/resources/ca/capsem-ca.key + crates/capsem-core/resources/ca/capsem-ca.crt (ECDSA P-256), compiled in via include_str! from net/cert_authority.rs
• Certs minted on-demand per domain, cached in RwLock<HashMap>
• CA baked into guest rootfs via update-ca-certificates + certifi patch + env vars
• No security value from the CA itself -- the guest is already fully sandboxed

Provider wire format references

Read these for the exact SSE format, request/response shapes, and telemetry extraction points:

• references/anthropic-wire.md -- Anthropic Messages API (event-typed SSE, gzip gotcha)
• references/openai-wire.md -- OpenAI Chat Completions + Responses API (data-only SSE, [DONE] sentinel)
• references/google-wire.md -- Google Gemini (complete JSON per event, no tool call IDs, camelCase)

Testing the proxy

• Unit tests: cargo test -p capsem-core net (policy evaluation, SNI parsing, cert minting)
• In-VM: just exec "capsem-doctor -k network" (TLS trust chain, port blocking, domain filtering)
• Telemetry: just exec "curl -s https://api.anthropic.com/", then inspect the typed timeline/security routes for the expected net_events evidence
• Adversarial: test with blocked domains, overlapping wildcards, malformed SNI, huge request bodies