analyze-video
Skill16 starsChanged 4 months agoMITView on GitHub
Read a video two ways — Gemini 3.5 Flash (frames + audio) for the beat plan, and Whisper (OpenRouter) for exact word timing. Writes analysis/beat-plan.md + analysis/words.json. Use at the start of a run.
- Reads credentials
7 other projects use a version of this.
Adopted by
Most recently taken up by
- evillollive/Analyze-Video-Skilltheir own version, first seen Nov 11, 2026
- tim-osterhus/video-analysis-kittheir own version, first seen Oct 20, 2026
- barefootford/buttercuttheir own version, first seen Sep 14, 2026
- jacksoswag/ObsidAgenttheir own version, first seen Aug 10, 2026
- novoads/agent-skillstheir own version, first seen Aug 8, 2026
- flintfromthebasement/skillstheir own version, first seen Jul 6, 2026
Other versions7 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 7
- barefootford/buttercut★ 625same name
- novoads/agent-skills★ 30same name
- evillollive/Analyze-Video-Skill★ 4same name
- flintfromthebasement/skills★ 1same name
- jacksoswag/ObsidAgent★ 1same name
- breethomas/bette-os★ 0same name
- tim-osterhus/video-analysis-kit★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository5 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- assembleSkill
- craft-promptSkill
- graphic-designSkill
- omni-vfxSkill
The file
Files
analyze-video
Paths —
input/,analysis/,assets/,output/are relative to the active projectprojects/<slug>/(the video being edited), not the repo root.
Turns the raw clip in input/ into analysis/beat-plan.md + analysis/words.json — the source of truth for the whole run. There are two reads, and they do different jobs: Gemini finds what/where/why; Whisper gives exact when.
Run it
- Read #1 — semantic beats (Gemini).
python3 scripts/analyze.py input/<clip>.mp4 > analysis/beats.json- Audio-aware, one key: Gemini 3.5 Flash via OpenRouter (
OPENROUTER_API_KEY). The clip is downscaled + base64'd into the request, so it sees frames AND hears the audio — beats anchor to what's actually said. No native Gemini key, no file upload. - Big videos (>~4 min): the base64 gets large. Either downscale harder / chunk into segments, or use
scripts/analyze_native.sh(native Gemini File API, needsGEMINI_API_KEY, handles up to ~1hr). - No-audio ultralight fallback:
python3 scripts/analyze_frames.py input/<clip>.mp4 2(frames only, OpenRouter).
- Audio-aware, one key: Gemini 3.5 Flash via OpenRouter (
- Read #2 — exact word timing (Whisper).
python3 scripts/transcribe.py input/<clip>.mp4→analysis/words.json- OpenRouter Whisper large-v3 (
OPENROUTER_API_KEY), word-level timestamps to ~10ms. This is where the correct timing comes from — Gemini's ±1s timestamps drift and cut off sentences; Whisper's word-times let you place seams in the pauses between words and write exact-seconds prompts later. - Whisper mis-hears proper nouns ("Omni"→"only", "Claude"→"cloud") — ignore the spelling, the timing is what we use.
- OpenRouter Whisper large-v3 (
- Convert + write the plan. Each Gemini beat comes back as
{start:"MM:SS", end:"MM:SS", beat_type, reason, suggestion}. Convert MM:SS → seconds, then snap each in/out to the nearest word boundary inwords.jsonso no cut lands mid-word. Writeanalysis/beat-plan.mdas the table below. - Also record the input aspect:
ffprobe -v error -select_streams v:0 -show_entries stream=width,height -of csv=p=0 input/<clip>.mp4→ note16:9or9:16(map anything else to nearest for Omni).
beat-plan.md format
Group edits into full ≤10s SEGMENTS (2–3 edits each), one row per segment — never one row/gen per single effect, and never cut a segment mid-action (see the segmentation rule in the root CLAUDE.md). Each segment = one Omni generation.
# Beat plan — <clip> (aspect: 16:9 · duration: 12:34)
| seg | start | end | type | edits in this segment (2–3) | why | prompt | status |
|-----|-------|-----|------|-----------------------------|-----|--------|--------|
| 1 | 00:00 | 00:08 | vfx | (a) brand can swap · (b) 3-stat card floats in · (c) glowing wordmark | hook + product beat | (craft-prompt fills) | 🔲 plan |
| 2 | 00:12 | 00:20 | vfx | (a) set change · (b) mascot walks in | co-pilot beat | (craft-prompt fills) | 🔲 plan |Each edit tagged with its own type (vfx | graphic) in the cell if they differ. Status ladder: 🔲 plan → ✍️ prompted → 🎬 generating → 👀 review → ✅ placed.
Reality (be honest)
- Gemini samples ~1 FPS → its timestamps are ±1s — good for marking beats, useless for exact cuts. Always take the real in/out from
words.json(Whisper), not from Gemini. - Long videos (>~50 min): the script fits default res; if it errors on context, add
media_resolution: low. - Both reads are cheap (cents). The expensive step is generation — that's gated later.
Then
Show beat-plan.md to the user — GATE 1. They edit/cut/add beats. Only after approval does craft-prompt run (it pulls its exact timecodes from words.json).
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

