Add Captions: Animated Subtitles

You are helping the user add captions to their Remotion video.

Load the remotion-production skill for the captions-workflow rule (routes, presets and safe zones), and the official remotion-captions skill for transcription and display. For the Scribe fallback, also load the official ElevenLabs speech-to-text skill. The official skills win on any API detail.

Workflow

1. Identify the Audio Source

Find the voiceover or primary audio file(s):

ls public/ public/footage/ 2>/dev/null | grep -E '\.(mp3|wav|m4a|ogg|mp4|mov|webm|srt)$'

If the user already has an .srt file, use it (step 3). If no audio exists, suggest running /add-voiceover first. If the video has one voiceover per scene (vo-scene1.mp3, vo-scene2.mp3, …), caption each file separately.

Make sure Studio is already running (npx remotion studio, or --no-open with an in-app browser; open the URL it prints), so the user sees captions appear.

2. Choose Caption Style

Ask the user what style they want:

• TikTok style (default): bold, a few words at a time, the active word pops in a highlight color. Needs word-level.
• Karaoke style: full phrase on screen, words highlight as they're spoken. Needs word-level.
• YouTube style: bottom of screen, a sentence at a time. Also works with the user's SRT.
• Minimal: small, subtle, bottom-left. Also works with the user's SRT.

3. Transcribe

Both routes give word-level Caption JSON, so every style works. Save each file's result as public/<source>.captions.json (e.g. vo-scene1.captions.json).

Default: official whisper-webgpu. Follow official remotion-captions/transcribe-captions.md. It writes a Node script with @remotion/whisper-webgpu that outputs one Caption per word. Local and free. It needs a compatible GPU and doesn't run on Linux arm64. Use small.en for English; for other languages use a multilingual model and pass language.

Fallback: ElevenLabs Scribe. When the script's canUseWhisperWebGpu() check fails, transcribe with Scribe v2 and convert with elevenLabsTranscriptToCaptions() from @remotion/elevenlabs (npx remotion add @remotion/elevenlabs). The script is in the captions-workflow rule. It needs ELEVENLABS_API_KEY (if it isn't set, run npx -y @elevenlabs/cli@latest onboard init), costs credits and sends the audio to ElevenLabs, so tell the user before running it. It accepts video files directly.

User's own SRT. Put it in public/ and read it with parseSrt() from @remotion/captions, as official import-srt-captions.md shows. SRT is line-level: each caption is a whole subtitle line, so a word highlight can't work from it. If the user picked TikTok or karaoke, offer to transcribe the audio instead, or use the YouTube style with their SRT.

Keep each transcript's timestamps relative to its own source file.

4. Place the Official Caption Element

Follow official remotion-captions/display-captions.md:

• Install @remotion/captions with npx remotion add @remotion/captions if needed.
• Copy the element for the chosen style into the project from https://www.remotion.dev/elements/captions/<element>.md: Popping Word Captions for TikTok, Word Highlight Captions for karaoke, Basic Captions for YouTube and minimal.
• Put one element per captioned clip, inside that clip's own <Sequence> next to its <Audio>/<Video>. Don't add one global caption layer over the whole composition.
• Inline the caption array in the captions prop so the Studio Caption editor can write back.

5. Customize Appearance

Apply the preset from the captions-workflow rule, then offer these options. Change them through the element's width, style and combineTokensWithinMilliseconds props, or the style constants at the top of the copied file:

• Position: top (10%), center (50%), bottom (15-25% on vertical, to stay clear of platform UI). See the safe-zone table in the rule.
• Font size: Small (40px), Medium (56px), Large (72px) at 1080 wide
• Highlight color: Yellow (#FFD700), Cyan (#00FFFF), Green (#00FF88), custom
• Background: None, semi-transparent box
• Font: Inter, Poppins, or whatever the project uses (load it as official remotion-markup google-fonts.md shows)
• Words per page: combineTokensWithinMilliseconds (lower = fewer words per page)

Any extra animation uses inline interpolate() in style with the scale/translate properties. Never use CSS transition or animation; they don't render.

6. Check in the Preview

Tell the user to scrub the captions in the open Studio. Common adjustments:

• Text too small/large on mobile? → Adjust font size
• Words changing too fast? → Increase combineTokensWithinMilliseconds
• Wrong timing? → Check that the captions sit in the same sequence as their audio, and that timestamps weren't offset
• Overlapping with visuals or platform UI? → Change position (top vs bottom)
• Wording mistakes? → Fix them in the Studio Caption editor, or edit the inlined array

Don't render unless the user asks for an MP4.