Meeting to Documentation

Turn a recorded feature/project walkthrough into a polished markdown document with embedded screenshots.

Inputs

• VIDEO — path to the .mp4 recording.
• TRANSCRIPT — path to a .vtt, .srt, or .txt transcript.
• OUTPUT_DIR — target folder for the generated doc and screenshots.

Preferred launch form:

/meeting-to-documentation VIDEO=/absolute/path/demo.mp4 TRANSCRIPT=/absolute/path/demo.vtt OUTPUT_DIR=/absolute/path/output

If one or more required inputs are missing, ask for all missing values in one message before starting the workflow. Do not ask for one path at a time. If only .txt is provided (no timestamps), screenshot correlation degrades to "best effort" — warn the user.

Input Collection

• If the VS Code askQuestions or ask-questions tool is available in the current environment, use it first to collect any missing required inputs in one integrated prompt.
• Collect VIDEO, TRANSCRIPT, and OUTPUT_DIR together rather than asking separate follow-up questions.
• If the tool is not available, ask a single consolidated chat question listing every missing value.
• Do not start probing files, running scripts, or asking secondary workflow questions until the required inputs are resolved.

When to Use

• User supplies a meeting/demo recording and wants a written doc.
• Needs screenshots extracted from the video embedded in the output.
• One-shot generation, not iterative editing.

Prerequisites

• ffmpeg and ffprobe installed (which ffmpeg). If missing, tell the user to install (brew install ffmpeg on macOS) and stop.
• Python 3 available for the transcript parser.

Model Guidance

This workflow mixes tool use, image inspection, transcript-to-structure synthesis, and screenshot curation — use the most capable model available. For clean, well-structured recordings a mid-tier model is sufficient; for noisy transcripts, heavy speaker overlap, or visually ambiguous UI states, prefer a higher-capability model.

Procedure

1. Parse the transcript

Run parse_transcript.py:

python3 ./scripts/parse_transcript.py "$TRANSCRIPT" > "$OUTPUT_DIR/transcript.json"

Output is a JSON array of {start, end, speaker, text} segments (seconds as floats). Read it end-to-end to understand the meeting content.

2. Determine the crop region (required for meeting recordings)

Meeting recordings almost always contain content that must not end up in the doc:

• The macOS menu bar (top) or Windows taskbar (bottom).
• Speaker video tiles (Teams/Zoom/Meet put these on the right, top, or bottom).
• Browser chrome you may or may not want to keep.

Run the extractor in probe mode to get the video dimensions and one sample frame:

./scripts/extract_frames.sh --video "$VIDEO" --output-dir "$OUTPUT_DIR/screenshots_candidates" --probe --probe-at 00:00:30

Pick a --probe-at timestamp where the app is clearly visible (not the intro). Then use the view_image tool to inspect probe_sample.jpg and derive the crop rectangle that keeps only the shared application window. Express it as W:H:X:Y (pixels):

• W = crop width, H = crop height
• X = left offset from the top-left corner, Y = top offset

If unsure, ask the user to confirm the crop. Store the chosen value in CROP.

3. Extract candidate screenshots

Run extract_frames.sh in scene-detect mode, passing the crop and a settle delay so frames are captured AFTER the scene change — this avoids half-rendered dialogs, the macOS app switcher, window transitions, and fades (the common failure modes):

./scripts/extract_frames.sh \
  --video "$VIDEO" \
  --output-dir "$OUTPUT_DIR/screenshots_candidates" \
  --scene-threshold 0.3 \
  --settle-delay 1.0 \
  --crop "$CROP"

Produces candidates.tsv (columns: filename, seconds, HH:MM:SS) plus the cropped jpg frames.

Tuning:

• Too many near-duplicates or transition blur → raise --scene-threshold (e.g. 0.4) and/or raise --settle-delay (e.g. 1.5–2.0).
• Missing moments → lower --scene-threshold (e.g. 0.2) or grab specific times with --timestamps (pass --crop again).
• Dialog/popup still animating in captured frame → raise --settle-delay.
• Frame shows transition to next scene instead of the one just entered → lower --settle-delay (e.g. 0.5).

See references/ffmpeg-tips.md.

4. Correlate frames with narration

For each candidate frame timestamp, find the transcript segment(s) whose [start, end] covers it (or the closest segment within ±5s). Build a table: frame → timestamp → surrounding narration (±15s window) → speakers.

Apply the heuristics in references/transcript-correlation.md to score relevance. Key signals: deictic references ("here", "this", "click", "if I go to", "you can see"), UI-state-change verbs, speaker is the presenter (not asking a question).

5. Infer the document structure

Read the full transcript and pick a structure that fits the content — do NOT force a template. Examples in references/doc-patterns.md. Common patterns: feature walkthrough, architecture review, workflow demo, Q&A digest. The structure should mirror how the presenter actually organized the talk.

6. Curate the screenshot set

From the candidate pool:

• Drop near-duplicates (consecutive frames showing the same screen).
• Drop transitional / blurry frames (app-switcher overlays, fades — these should be rare now thanks to --settle-delay, but review anyway).
• Drop frames where a dialog/popup is still animating in (partial opacity, sliding). If many survive, re-run with a larger --settle-delay.
• Prefer frames where the speaker explicitly references the UI at that moment.
• Aim for 5–15 screenshots total unless the content clearly needs more.

If a key topic has no nearby candidate frame, re-run the extractor in explicit-timestamp mode for that moment (pass the same --crop):

./scripts/extract_frames.sh --video "$VIDEO" --output-dir "$OUTPUT_DIR/screenshots_candidates" \
  --timestamps "00:05:23,00:12:47" --crop "$CROP"

7. Write the document

1. Create $OUTPUT_DIR/screenshots/ and copy the curated frames with descriptive filenames (e.g. 01-dashboard-overview.jpg, 02-create-form.jpg).
2. Write $OUTPUT_DIR/documentation.md:
   • Synthesize prose from the transcript — do NOT paste verbatim quotes unless a quote is particularly illustrative.
   • Remove filler words, false starts, and off-topic side chatter.
   • Write as a neutral technical author, not as a recorder of a session. The document must read as if a developer or technical writer documented the feature or project directly. Never mention that the content comes from a recording, session, walkthrough, or transcript.
   • Forbidden phrases and patterns — never use anything that reveals the source material:
     • "the presenter", "the speaker", "the demo", "the walkthrough", "the session", "the recording"
     • "as shown in the recording", "discussed during the session", "mentioned in the walkthrough"
     • "the first/second half of the session", "the narration", "the recorded explanation"
     • "the presenter showed/explained/noted/pointed out/called out/highlighted"
     • Passive constructions that still imply a live narrator: "was shown", "was demonstrated", "was walked through"
   • Use neutral, documentation-style language instead: "The feature works as follows", "Users can…", "The system…", "This field controls…", "An important caveat is…".
   • Do NOT attribute statements to speakers. If a caveat or note is important, present it as a factual note in the documentation.
   • Embed screenshots with captions: ![Creating a new report](./screenshots/02-create-form.jpg).
   • Place each screenshot near the section of prose it illustrates.

8. Clean up intermediate artifacts

The extraction pipeline produces several intermediate files and folders that must NOT be left in $OUTPUT_DIR. The final deliverable is just:

• $OUTPUT_DIR/documentation.md
• $OUTPUT_DIR/screenshots/ (only the curated, renamed frames)

Before finishing, delete every intermediate artifact:

rm -rf "$OUTPUT_DIR/screenshots_candidates"
rm -f  "$OUTPUT_DIR/transcript.json"
rm -f  "$OUTPUT_DIR/candidates.tsv"
rm -f  "$OUTPUT_DIR/probe_sample.jpg"

Also remove any other scratch files the workflow created (e.g. temporary correlation tables, uncurated frames, .DS_Store). Verify the final tree with ls -la "$OUTPUT_DIR" and ls "$OUTPUT_DIR/screenshots" before reporting completion — only the two expected entries above should remain.

If the user explicitly asks to keep intermediates for debugging, skip this step and mention it in the summary.

9. Summarize

Print a final summary: crop region used, settle delay used, total candidates, frames kept, frames dropped (with reason), inferred doc structure, the output path, and confirmation that intermediate artifacts were cleaned up.

References

• Transcript correlation heuristics
• Documentation patterns
• ffmpeg tuning tips

Anti-patterns

• Dumping the raw transcript into the doc.
• Including every candidate frame (bloat).
• Forcing a fixed template that doesn't match the talk.
• Skipping screenshot curation because "the agent picked them" — always review.
• Treating Q&A banter as primary content.
• Skipping the probe/crop step and shipping screenshots that show the macOS menu bar, Windows taskbar, or speaker video tiles.
• Capturing frames at the exact scene-change timestamp — the dialog or new window is almost always still animating. Use --settle-delay.
• Leaking the recording as the source — any phrase that reveals the document was generated from a session recording ("the presenter", "the walkthrough", "during the session", "the recorded explanation", etc.). The output must be indistinguishable from hand-written feature documentation.
• Leaving intermediate artifacts behind — shipping screenshots_candidates/, transcript.json, candidates.tsv, probe_sample.jpg, or any uncurated frames in $OUTPUT_DIR. The final output must contain only documentation.md and the curated screenshots/ folder.