CLAUDE.md

Guidance for Claude Code (and other AI assistants) working in this repository. See README.md for end-user/usage detail; this file is the agent-facing reference.

Project overview

Whisper Flow is a real-time, streaming speech-to-text service built on OpenAI Whisper and served via FastAPI. It splits incoming audio into temporal tumbling windows to emit partial transcripts at sub-500ms latency. Shipped as the Python package whisperflow (v1.4.0, Python 3.8+, developed/tested on 3.12).

Commands

Everything runs through run.sh. The -local target rebuilds .venv from scratch; the other targets assume the venv is already activated (source .venv/bin/activate).

Running a single test (venv activated): pytest tests/test_streaming.py -v

Quality gates — must pass

CI enforces these via Dockerfile.test. ./run.sh -test runs the same checks locally:

• black whisperflow tests — formatting (no diffs allowed)
• pylint --fail-under=9.9 whisperflow tests — near-perfect lint score required
• pytest --ignore=tests/benchmark --cov-fail-under=95 --cov whisperflow — 95% coverage

Implication: every code change needs accompanying tests to keep coverage ≥95%, and all code must be black-formatted and pass pylint at 9.9+. Run ./run.sh -test before considering work done.

Architecture & key files

• whisperflow/transcriber.py — Whisper model loading and inference: get_model() (returns a ModelPool), transcribe_pcm_chunks(), transcribe_pcm_chunks_async(). Never call one Whisper model from two threads at once: its decoder installs kv-cache hooks on the model, so concurrent calls corrupt each other. Borrow a replica with ModelPool.acquire() (transcribe_pcm_chunks does this for you).
• whisperflow/streaming.py — tumbling-window streaming logic, TranscribeSession, and the committed-prefix scheme (Segment.commit)
• whisperflow/fast_server.py — FastAPI app: WebSocket /ws, GET /health, GET /ready, POST /transcribe_pcm_chunk
• docs/protocol.md — wire contract; bump PROTOCOL_VERSION in whisperflow/__init__.py on any breaking change to it
• whisperflow/chat_room.py — multi-user chat
• whisperflow/audio/microphone.py — audio input
• whisperflow/models/ — bundled tiny.en.pt model
• tests/ — pytest suite. tests/benchmark/ and tests/audio/ are excluded from the coverage gate.

Environment gotchas

• Python 3.12 preferred — run.sh auto-selects python3.12 when available.
• setuptools<70 pin is required for openai-whisper compatibility (see run.sh).
• PortAudio system dependency is required for PyAudio: brew install portaudio (macOS) / apt-get install portaudio19-dev (Debian/Ubuntu).
• Audio format: 16 kHz, mono, 16-bit signed integer (int16) PCM.
• Ports: 8181 when run locally, 8888 in Docker.

Conventions

• Match the existing style; keep modules small and focused.
• Add tests with every code change (the 95% coverage gate enforces this).
• Always run ./run.sh -test and ensure it is green before finishing.