Speech Captions

Automatic speech-to-text and caption file generation. Use this skill when you need spoken content turned into subtitles — not when muxing or burning captions (see subtitles).

When to use

• This skill — transcribe dialogue, detect language, convert transcript text/segments to SRT or WebVTT.
• subtitles — convert/shift/extract/burn existing subtitle files after you have captions.
• audio — extract or normalize audio before transcription when the source is noisy or the wrong sample rate.
• inspect — confirm duration, audio codec, and that an audio stream exists before running whisper.

Gotchas

• First run downloads a model — faster-whisper caches models under ~/.cache/huggingface. tiny is fast but less accurate; use small or medium for production dialogue.
• CPU-only by default — scripts use device="cpu" and compute_type="int8". GPU setups can edit the script or run whisper elsewhere; these CLIs target portable agent use.
• Plain text → cues is approximate — to-srt.py / to-vtt.py split on sentence boundaries and spread cues evenly. For accurate timing, use transcribe.py or pass --segments-json from a real ASR pass.
• Music and silence — whisper may hallucinate text on instrumentals or long silence. Inspect output; trim leading/trailing silence with audio first if needed.
• Language auto-detection — detect-language.py samples the first 60 seconds. Short clips or multilingual content may misreport; pass an explicit language to whisper in a custom workflow if you know the locale.
• Video inputs — transcription extracts mono 16 kHz WAV via ffmpeg. Very long files take proportionally longer; there is no chunking in the default transcribe script.

Running scripts

This skill's scripts live in scripts/ alongside this file. Before running any script, resolve its absolute path from this skill's own directory — do not assume the shell cwd is the skill folder, the repo root, or the workspace. Stay in the workspace (so media paths resolve) and pass the absolute script path to uv run:

uv run <skill_directory>/scripts/transcribe.py --input "<path-to-media>" --model tiny

<skill_directory> is a placeholder. Never pass those angle brackets to the shell — the command will fail. Replace it with a real absolute directory before invoking uv run.

How to get that directory (do this yourself; do not ask the user first; do not skip the skill):

1. If your tools showed a filesystem path when loading this SKILL.md, strip SKILL.md and use that folder.
2. If you already ran another mediaskills script (for example .../audio/scripts/extract.py), this skill is a sibling of that folder: same parent, directory name speech-captions.
3. Otherwise search standard install locations and confirm the script exists:

NAME=speech-captions
for d in \
  "$HOME/.agents/skills/$NAME" \
  "$HOME/.cursor/skills/$NAME" \
  "$HOME/.codex/skills/$NAME" \
  "$HOME/.claude/skills/$NAME" \
  "$PWD/.agents/skills/$NAME" \
  "$PWD/.cursor/skills/$NAME" \
  "$PWD/skills/$NAME"
do
  if [ -f "$d/SKILL.md" ] && [ -e "$d/scripts/transcribe.py" ]; then
    SKILL_DIR=$(cd "$d" && pwd)
    break
  fi
done
[ -n "$SKILL_DIR" ] || { echo "mediaskills: $NAME not found" >&2; exit 1; }
uv run "$SKILL_DIR/scripts/transcribe.py" --input "<path-to-media>" --model tiny

Prefer absolute paths for --input / --output when cwd is uncertain.

Missing binaries vs permissions

Before running any script, treat these two failures differently:

• If a required binary (uv, ffmpeg) reports No such file or directory or command not found: this is a missing or unreachable binary, not a permissions issue. Escalating sandbox permissions will not fix it. Run command -v uv and command -v ffmpeg to see whether they resolve at all. If not, install or re-link them (install-media-tools doctor/install) rather than retrying with wider permissions.
• Only escalate sandbox permissions if the error is explicitly a permission denial (e.g. Operation not permitted, Permission denied).

Recipes

Transcribe a clip to SRT + JSON

uv run <skill_directory>/scripts/transcribe.py --input interview.mp4 --model tiny

Returns data.srt_path, data.json_path, full text, and per-segment timings in data.segments.

Detect language before choosing a model

uv run <skill_directory>/scripts/detect-language.py --input interview.mp4

Check data.language and data.probability before committing to a larger whisper model.

Build SRT from transcript text (no ASR)

uv run <skill_directory>/scripts/to-srt.py --text "Hello world. This is a test." --duration 6 --output captions.srt

Build VTT from transcribe JSON

uv run <skill_directory>/scripts/to-vtt.py --segments-json interview_transcript.json --output captions.vtt

The JSON file can be the json_path from transcribe.py (it includes a top-level segments array).

End-to-end caption pipeline

1. inspect — confirm audio stream and duration.
2. speech-captions transcribe — produce SRT + JSON.
3. subtitles shift — fix sync offset if captions lead/lag video.
4. subtitles burn — hardcode for players without subtitle support.

Troubleshooting

Available scripts

Acceptance checks (agent must pass before delivery)

1. Contract: exit 0, ok: true, SRT/JSON paths exist and are non-empty.
2. Skill gate: hand off to captions-compliance validate.py before broadcast export.
3. Spot-check: read first/last cues; language matches request; duration coverage roughly matches source audio (not a 2-second stub on a long file).
4. ASR is imperfect — never claim verbatim accuracy without human/agent review of critical lines.
5. On failure: re-transcribe with a clearer extract or escalate; do not deliver empty SRT.

Do not use for

• Burning subtitles into video (use subtitles/burn.py)
• Broadcast SCC/SMPTE-TT export (use captions-compliance)
• Video or audio transforms (use video-transformation / audio)

Related skills

• subtitles — format conversion, timing shift, extract embedded tracks, burn-in
• audio — extract/normalize audio before ASR
• inspect — probe source media before transcription
• install-media-tools — install ffmpeg