Playwright Demo Video Skill

Generate end-to-end demo videos (MP4) from Playwright specs with on-screen captions, AI voiceover (ElevenLabs), freeze-frame timing, and background music.

When to Use

• User says "create a demo video", "generate a demo", "playwright demo"
• User wants to convert a Playwright spec into a narrated video
• User wants to add voiceover to an existing video recording
• User wants to auto-discover project features and generate a demo spec
• User wants to extract captions from a Playwright test file

Prerequisites

Quick Start

One command records the spec, captures exact timestamps, generates voiceover, and merges the final MP4:

node scripts/run-pipeline.mjs \
  --record \
  --spec path/to/demo.spec.ts \
  --grep "Exact Test Title Here" \
  --music path/to/background.mp3 \
  --output-dir ./demo-output \
  --project-dir /path/to/web-project

Two non-negotiables before running it:

1. The spec calls startTimestampRecording() as its first line (exact video-audio sync)
2. --grep matches exactly ONE test (see Multiple @demo Specs below)

Data-First Approach (MANDATORY)

You cannot narrate what doesn't exist. Before writing any demo script or narration, you MUST verify that the application has real data to show. A demo that says "here are our recipes" while the screen shows "No items found" is a failed demo.

Stage 0: Data Setup & Discovery

This stage runs BEFORE any recording or script writing. Full copy-paste specs for each step are in references/verification.md.

Step 1: Verify or create data. Run a data-check spec that visits the main pages, scans body text for empty-state indicators ("No items", "Create your first", ...), and counts visible content elements (cards, articles, list items).

If the app shows empty states:

1. Find and run the seed script (npm run seed, npx prisma db seed, npx tsx lib/db/seed.ts, etc.)
2. If no seed script exists, create minimal seed data via the app's API or direct DB inserts
3. Re-run the data check to confirm content now appears
4. Do NOT proceed to script writing until content is verified

Step 2: Explore what Playwright can see. Run an exploration spec that visits every route, checks for error states, catalogs interactive elements (buttons, links, forms, images, cards), and screenshots each page. Use the exploration output to write the demo script. Only narrate features the exploration confirmed exist — if a route shows 0 cards, don't write narration about "browsing our collection."

Step 3: Build the demo script incrementally. Do NOT write the entire demo spec at once. Write scene 1, record it, verify the output shows real content; then add scene 2 and verify both; continue until complete.

test('demo scene 1 - landing page @demo', async ({ page }) => {
  startTimestampRecording();  // REQUIRED first line
  await page.goto('http://localhost:3000');
  await page.waitForLoadState('networkidle');

  // ASSERT content is visible BEFORE writing any caption about it
  await expect(page.locator('h1, [data-testid="hero"]').first())
    .toBeVisible({ timeout: 10000 });

  // Only NOW describe what's actually on screen
  caption(page, 'Welcome to Recipe Discovery', 3000);
});

Rule: Assert, then narrate. Every caption must be preceded by an assertion that the content it describes is actually visible. The narration is a description of what IS on screen, not what SHOULD be on screen.

Demo Script Writing Rules

1. Always call startTimestampRecording() as the first line of the test. This is the single most important step for reliable video-audio sync. Without it, the pipeline uses heuristic timestamp estimation that drifts on demos >60s.
2. Never assume data exists. Always assert before narrating.
3. Never use .catch(() => {}). If an interaction fails, the demo should fail. For genuinely optional elements (cookie banners), use an explicit isVisible() conditional — see references/verification.md.
4. Match narration to visuals exactly. If the page shows 3 recipes, say "3 recipes" not "our collection of recipes."
5. Test each scene in isolation first. Don't chain 10 scenes together and hope they all work.
6. Use specific, observable descriptions. Instead of "powerful search capabilities," say "searching for pasta dishes" while the search box shows "pasta" and results are filtered.
7. Size caption hold times to text length. The voiceover takes longer than you think:

   holdMs = max(textLength * 80, 3000)

   Example: a 72-char caption → caption(page, text, 5760) not caption(page, text, 3000). Short captions (< 40 chars) can use 3000ms.
8. Add a closing pause. The last scene must include await page.waitForTimeout(5000) AFTER the final caption so the voiceover finishes before the Playwright recording ends. The merge script adds a tail freeze for safety, but the spec should still provide enough buffer.
9. Act before you narrate when UI state is ephemeral. caption() blocks for seconds — live dashboards, "running now" cards, toasts, and progress banners can complete and disappear mid-caption, so a click queued after the caption targets an element that no longer exists. Assert the ephemeral element, click through to a stable destination page immediately, and put the captions there. (Real case: a job card that lived ~7s vanished during an 8s caption; the follow-up "Watch live" click timed out.)
10. Parameterize expensive real actions via env. If the demo triggers something slow or costly (a real LLM run, a deploy, a paid API), select it with an env var (const BACKEND = process.env.DEMO_BACKEND ?? 'dryrun') so scene iteration runs against the cheap path and only the final take pays. Gate the captions/assertions that only apply to the real path behind the same flag.

Visual Verification (MANDATORY)

Demo videos are worthless if the screen shows errors while the narration describes features. Every demo recording MUST pass verification at three stages — skipping these is not acceptable; a demo of an error page is worse than no demo at all. Full code for each stage is in references/verification.md.

Auto-discover-generated specs must additionally include navigation assertions, content visibility waits, a pre-flight test block, and no try/catch suppression — see references/verification.md.

Common Failure Modes

Known issues that cause demos to record error pages or blank screens:

When any of these are detected during the pre-recording health check, fix the underlying issue before proceeding. Never record a demo of a broken application.

Critical: Multiple @demo Specs

The --spec flag does NOT filter which Playwright tests run. It only controls caption extraction. The --grep flag (default: @demo) controls test execution. If multiple spec files contain @demo in their test name, ALL of them run during --record — markers from every test mix into one log, captions.json gets entries from all of them, and the auto-found video is the LAST test's, not yours.

Prevention: always pass a grep pattern that matches exactly one test:

# BAD: matches ALL @demo specs
node run-pipeline.mjs --record --spec demo/auth-demo.spec.ts --grep @demo

# GOOD: matches only the specific test by title
node run-pipeline.mjs --record --spec demo/auth-demo.spec.ts \
  --grep "Five Authentication Strategies"

# GOOD: use a unique tag per spec (@auth-demo vs @exec-demo)
node run-pipeline.mjs --record --spec demo/auth-demo.spec.ts \
  --grep @auth-demo

Recovery after a bad recording (find the per-test video, trim captions.json, re-merge with --skip-voice — no ElevenLabs credits needed): see references/troubleshooting.md.

Two Modes

Mode 1: Guided (User Provides Spec)

User has a Playwright spec with showCaption()/caption() calls.

Steps:

1. Copy templates into the target project:
   • templates/caption-overlay.ts -- caption CSS + functions + timestamp recording
   • templates/demo-helpers.ts -- pause/scroll/viewport/dragAndDrop helpers
   • templates/playwright.video.config.ts -- video recording config
2. IMPORTANT: Ensure the spec calls startTimestampRecording() as the first line of the test:

   import { startTimestampRecording, showCaption, caption, hideCaption } from './caption-overlay';
   
   test('My Demo @demo', async ({ page }) => {
     startTimestampRecording();  // <-- REQUIRED for exact timestamps
     await page.goto('/');
     // ... rest of demo
   });

3. Recommended (one command): Run the pipeline with --record (see Quick Start above). This runs Playwright, captures timestamp markers from stdout, auto-finds the video, and produces the final MP4 with exact audio-video sync.
4. Alternative (separate steps): Record manually, then run pipeline:

   # Record and capture output
   npx playwright test --config=playwright.video.config.ts --grep @demo 2>&1 | tee demo-output/recording.log
   # Run pipeline with captured log
   node scripts/run-pipeline.mjs \
     --spec path/to/demo.spec.ts \
     --video path/to/recording.webm \
     --music path/to/background.mp3 \
     --output-dir ./demo-output

   If recording.log exists in the output dir, extract-captions will auto-detect __CAPTION_TS__ markers.

Mode 2: Auto-Discover (Generate Spec from Project)

Scan a project's source code and generate a demo spec automatically. See references/auto-discover.md for detailed patterns.

Steps:

1. Run auto-discover to generate spec and feature inventory:

   node scripts/auto-discover.mjs <project-dir> [--focus <feature>] [--dry-run]

   (Full options in references/scripts-reference.md.)
2. Review the generated spec -- adjust captions and timing as needed
3. Copy templates into the target project (helpers.ts, caption-overlay.ts, playwright.video.config.ts)
4. Record the video: npx playwright test --config=playwright.video.config.ts --grep @auto-demo
5. Run the pipeline on generated spec + recorded video

What auto-discover detects:

Guided mode: Pass --focus <feature> to generate a spec focused on a single feature (e.g., --focus kanban, --focus "dark mode", --focus charts). The focus string matches against feature category, label, and description. If no match is found, falls back to full discovery with a warning.

Pipeline Architecture

Project source code
  |  auto-discover       Scan framework, routes, features, test IDs
  v                      Generate Playwright spec with captions
Playwright spec (*.spec.ts)
  |  record (optional)   Run Playwright test, capture __CAPTION_TS__ markers
  v                      from startTimestampRecording() -> recording.log
  |  extract-captions    Parse markers for exact timestamps (preferred)
  v                      OR estimate from code heuristic (fallback)
  |  generate-voice      Per-caption ElevenLabs API -> caption_NN.mp3 (cached)
  v
  |  merge-video         Video + audio -> freeze-frame merge -> MP4
  v                      Zero audio overlaps guaranteed
  |  add-music           Voiced MP4 + music -> final MP4 (15% volume, looped)
  v
Final demo.mp4

Timestamp Modes

Always prefer real timestamps. The heuristic mode exists as a fallback for specs that don't use startTimestampRecording(), but it will drift on demos longer than ~60s with many navigation/network operations.

Pipeline Scripts

All scripts live in scripts/ and use Node.js builtins only (zero npm dependencies). Full CLI options, timestamp estimation constants, and error-handling behavior are in references/scripts-reference.md.

Templates

Copy these into your target project.

• templates/caption-overlay.ts — Caption CSS + showCaption/hideCaption/caption functions + startTimestampRecording(). When recording starts, each caption function emits __CAPTION_TS__ markers to stdout that the pipeline parses for exact video-audio sync. extract-captions.mjs parses these function calls, so the naming convention matters.
• templates/demo-helpers.ts — Demo pacing utilities: pause, scenicPause, quickPause, smoothScroll (CSS selectors only), scrollToLocator (any Playwright Locator), setViewport, naturalType, dragAndDrop. Each has documented internal timing that extract-captions.mjs uses for timestamp estimation.
• templates/playwright.video.config.ts — Playwright config optimized for video recording: headless mode, 1280x800 viewport, generous timeouts, sequential execution, auto-start dev server.

Reference Files

• Verification specs (full code): references/verification.md -- data-check, route exploration, pre-flight, during-recording assertions, post-recording frame validation
• Scripts CLI reference: references/scripts-reference.md -- all options per script, timestamp estimation constants, error handling
• Troubleshooting: references/troubleshooting.md -- multiple @demo specs deep dive, recovery after bad recordings, --skip-voice, expanded failure checklist
• Auto-discover patterns: references/auto-discover.md -- framework detection, feature scanning, spec generation
• Pipeline patterns: references/pipeline-patterns.md -- ffmpeg commands, ElevenLabs API, freeze-frame algorithm
• Full research findings: references/RESEARCH.md -- AST vs regex analysis, prototype results, algorithm pseudocode

Success Criteria

• Caption extraction matches all showCaption/caption calls in the spec
• With startTimestampRecording(): timestamps are exact (within ~1s absolute, 0 relative drift)
• Without: heuristic timestamps within +/-1s for short demos, may drift on long demos
• JSON manifest produced and editable
• Auto-discover identifies major features from project structure
• Pipeline produces working MP4 with synchronized voice and captions
• Audio clips never overlap in the final video

On Failure