Shots

Detect hard cuts (shot boundaries) in video and extract a representative still from the middle of each shot.

Shots vs program-master

Use shots when you need every edit point or shot-level stills for vision analysis. Use program-master when the user asks to label segments, find commercial breaks, or name episodes/parts in a broadcast master. Do not use shot detection for TV structure — a single "shot" may span an entire scene, and program breaks are often fades, not hard cuts.

When to use

• Cataloging a program into shots for review or QC
• Feeding shot-level stills into vision analysis or OCR
• Building on-screen text reports at shot granularity

Prefer shot detection over fixed-interval frame sampling when the unit of analysis is a shot, not a time grid. For uniform time sampling (e.g. every 1s for forced-narrative sweeps), extract frames at fixed intervals instead.

Output paths

Shot manifests and extracted stills write to the workspace root .mediaskills/generated/ (not inside this skill folder). Override with $MEDIASKILLS_DATA_DIR/generated/ when set.

Scripts

Running scripts

This skill's scripts live in scripts/ alongside this file. Before running any script, resolve its absolute path from this skill's own directory — do not assume the shell cwd is the skill folder, the repo root, or the workspace. Stay in the workspace (so media paths resolve) and pass the absolute script path to uv run:

uv run <skill_directory>/scripts/detect.py --input "<path-to-media>"

<skill_directory> is a placeholder. Never pass those angle brackets to the shell — the command will fail. Replace it with a real absolute directory before invoking uv run.

How to get that directory (do this yourself; do not ask the user first; do not skip the skill):

1. If your tools showed a filesystem path when loading this SKILL.md, strip SKILL.md and use that folder.
2. If you already ran another mediaskills script (for example .../audio/scripts/extract.py), this skill is a sibling of that folder: same parent, directory name shots.
3. Otherwise search standard install locations and confirm the script exists:

NAME=shots
for d in \
  "$HOME/.agents/skills/$NAME" \
  "$HOME/.cursor/skills/$NAME" \
  "$HOME/.codex/skills/$NAME" \
  "$HOME/.claude/skills/$NAME" \
  "$PWD/.agents/skills/$NAME" \
  "$PWD/.cursor/skills/$NAME" \
  "$PWD/skills/$NAME"
do
  if [ -f "$d/SKILL.md" ] && [ -e "$d/scripts/detect.py" ]; then
    SKILL_DIR=$(cd "$d" && pwd)
    break
  fi
done
[ -n "$SKILL_DIR" ] || { echo "mediaskills: $NAME not found" >&2; exit 1; }
uv run "$SKILL_DIR/scripts/detect.py" --input "<path-to-media>"

Prefer absolute paths for --input / --output when cwd is uncertain.

Missing binaries vs permissions

Before running any script, treat these two failures differently:

• If a required binary (uv, ffmpeg, ffprobe) reports No such file or directory or command not found: this is a missing or unreachable binary, not a permissions issue. Escalating sandbox permissions will not fix it. Run command -v uv, command -v ffmpeg, and command -v ffprobe to see whether they resolve at all. If not, install or re-link them (install-media-tools doctor/install) rather than retrying with wider permissions.
• Only escalate sandbox permissions if the error is explicitly a permission denial (e.g. Operation not permitted, Permission denied).

Recipes

Shot catalog (cuts only)

uv run <skill_directory>/scripts/detect.py --input /path/to/clip.mp4

Read shot_count and shots from the JSON result. The manifest path is in manifest_path (auto-generated *_shots.json under .mediaskills/generated/).

Shot stills for vision analysis

uv run <skill_directory>/scripts/detect.py --input clip.mp4
uv run <skill_directory>/scripts/extract-frames.py --shots-path .mediaskills/generated/clip_shots_*.json

Or detect inline:

uv run <skill_directory>/scripts/extract-frames.py --input clip.mp4

The updated manifest lists every frame_path with per-frame technical metadata (width/height). JPGs land in .mediaskills/generated/<stem>_shot_frames/.

Threshold tuning

• Default --threshold is 0.35 (ffmpeg scene score 0–1). Lower (e.g. 0.25) finds more cuts; higher (e.g. 0.5) is stricter.
• --min-shot-seconds (default 0.5) merges shots shorter than that into neighbors — reduces flicker from flash frames.
• Hard cuts are reliable; slow dissolves/fades may be missed (ffmpeg scene filter limitation).

Outputs

• detect.py → JSON manifest (*_shots.json) with shots[] array: index, start/end seconds, timecodes, scene_score
• extract-frames.py → updates the manifest with frames[] and writes JPG stills to frames_dir

Always use the manifest_path / shots_path from the JSON result for downstream steps — do not invent filenames.

Acceptance checks (agent must pass before delivery)

1. Contract: exit 0, ok: true, manifest and any frames_dir exist.
2. Spot-check: shot_count is plausible for the content; open 2–3 midpoint JPGs and confirm they fall inside distinct shots when cuts are expected.
3. On hard-cut masters, assert known cut inventory when available; dissolves/fades may be missed — say so.
4. On failure: retune --threshold / --min-shot-seconds or escalate; do not invent shot boundaries.

Do not use for

• Full program structure / act breaks (use program-master)
• Vision-based scene understanding (use vision-analysis)

Related skills

• image — ocr.py on a single extracted still
• program-master — black/silence TV structure (different segmentation axis)
• video-transformation — extract_frame.py for a single timestamp
• timecode — frame-accurate timecode math