music-to-video
Skill18 starsChanged 49 days agoMITView on GitHub
Turn a music track (an audio file, a video to pull audio from, or a track generated from a mood brief) into a beat-synced video — lyric video, slideshow, or kinetic promo. The music drives all pacing; any user-supplied images/videos are cut onto the same beat grid, and a complete video needs zero assets. Narrated pieces → the input-matched workflow (see /hyperframes). Unclear → /hyperframes.
- Reads credentials
- Installs packages
20 other projects use a version of this, 18 of them edited.
Adopted by
Most recently taken up by
- Vujavujavuja/Stillmotion-onbrandedited, first seen Nov 16, 2026
- calesthio/OpenMontageedited, first seen Nov 9, 2026
- OpenMontageInc/OpenMontageedited, first seen Nov 8, 2026
- edenfunf/reelmimicword for word, first seen Nov 4, 2026
- vikneshwaran16032005-alt/claude-skillsedited, first seen Nov 1, 2026
- langchain-ai/design-agentedited, first seen Oct 25, 2026
Used with
Projects with it also use website-to-video, bfl-api and 7 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
jaron-goh-centrSkillwebsite-to-video
Capture a general website/URL and turn it into a video OF the site — tour, showcase, or social clip built from captured screenshots and the site's own brand assets. Use for portfolio / blog / docs / landing-page showcases. Not a product launch or promo, even from a URL (/product-launch-video). Unclear → /hyperframes.
Also in 7 projects that use this
LinHao-citySkillbfl-api
BFL FLUX API integration guide covering endpoints, async polling patterns, rate limiting, error handling, webhooks, and regional endpoints with Python and TypeScript code examples.
Also in 3 projects that use this
LinHao-citySkillflux-best-practices
Comprehensive guide for BFL FLUX image generation models. Covers prompting, T2I, I2I, structured JSON, hex colors, typography, multi-reference editing, and model-specific best practices for FLUX.2 and FLUX.1 families.
Also in 3 projects that use this
- +6 more from LinHao-city
Other versions20 projects use a version of this.
Word for word · 1
- edenfunf/reelmimic★ 2k100% same
Edited · 18
- calesthio/OpenMontage★ 81k84% same
- OpenMontageInc/OpenMontage★ 81k84% same
- EvovexAI/EvoFlow★ 63684% same
- langchain-ai/design-agent★ 61995% same
- duyet/clickhouse-monitoring★ 30584% same
- LinHao-city/StoryMind★ 1084% same
- Samin12/skills★ 898% same
- vikneshwaran16032005-alt/claude-skills★ 395% same
- Vujavujavuja/Stillmotion-onbrand★ 296% same
- ElyzeSolutions/skills★ 198% same
8 more not listed
Wrote their own version · 1
- STARDUSTLC666/dsh-hyperframes★ 35same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository37 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- anime-celSkill
- blender-product-filmSkill
- captions-overlaySkill
- crayon-storybookSkill
- cut-the-curveSkill
- embedded-captionsSkill
- faceless-explainerSkill
- ffmpegSkill
- figmaSkill
- frontend-designSkill
- general-videoSkill
- hyperframes-animationSkill
- hyperframes-audioSkill
- hyperframes-cliSkill
- hyperframes-coreSkill
- hyperframes-creativeSkill
- hyperframes-keyframesSkill
- hyperframes-registrySkill
- hyperframesSkill
- hyperframes-studioSkill
- media-useSkill
- motion-doctrineSkill
- motion-graphicsSkill
- oversized-cursorSkill
- painted-animationSkill
- paper-cutoutSkill
- pixel-artSkill
- product-launch-videoSkill
- pr-to-videoSkill
- remotion-to-hyperframesSkill
- seam-craftSkill
- slideshowSkill
- talking-head-recutSkill
- video-cloneSkill
- whiteboardSkill
The file
We split this skill into the rules it gives an agent
5 rules. 0 are common, 1 is distinctive, and 4 are very specific to this project.
0
Common
Most of the 17 edited versions we read keep these.
1
Distinctive
This author's own choices. Worth a look.
4
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 33
Collaborative: wait for sign-in or an explicit choice to continue offline with the local provider.
- Very specific: line 91
Read references/planning.md, storyboard-format.md, template-catalog.md, motion-primitive-catalog.md, and
- Very specific: line 111
Create compositions/frames/.
- Very specific: line 128
Gate: every frame has its compositions/frames/NN-*.html on disk.
- Very specific: line 153
Run the CLI on the assembled project — that's the correct unit (the per-frame workers couldn't run it).
“Common” is measured: kept by at least 6 in 10 of the 17 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
First, keep this skill fresh — confirm with the user before running:
npx hyperframes skills update music-to-video. A fast no-op when everything is current; otherwise it refreshes this skill plus the core domain skills it depends on before you rely on them.
music-to-video — one music-grounded, beat-synced video workflow
Use this skill to turn a music track into a beat-synced HyperFrames video. You analyze the track once, lay out the frames, fill in a per-frame plan, and build each frame as a composition. The input is a music track plus optional user images or videos — there is no narration and no website capture. Typography and templates are the floor (a complete video needs zero assets); any media the user supplies is cut in on the same beat grid.
You are the orchestrator. Work in videos/<project>/. Run the steps in order and pass each Gate before moving on. Two steps need the user: Step 3 (plan approval) and Step 6 (render approval) — both are checkpoint gates per ../hyperframes/references/brief-contract.md (read it before Step 0): in autonomous mode, post the summary as a heads-up and proceed instead of waiting. Do every step yourself except Step 4, where you dispatch one sub-agent per frame. Keep design and motion rules out of this file — they live in references/ and the frame-worker sub-agent.
SKILL_DIR = this skill directory. PROJECT_DIR = videos/<project-name>/.
Workflow: Step 0 setup → hyperframes.json + assets/bgm.mp3; Step 1 analyze → audiomap.json; Step 2 skeleton → STORYBOARD.md (frames, groups TBD); Step 3 plan → complete STORYBOARD.md + frame.md; Step 4 build → compositions/frames/NN-*.html; Step 5 assemble → index.html; Step 6 render → renders/video.mp4.
Two ideas that shape everything
- One analyzer, and you trust it.
analyze-beatgrid.pyis the only beat analyzer — never re-measure beats with another tool or by ear. Its energy / density / rolls / onsets / silences are always reliable. Itsbpmandbeats_secare reliable only when the music is genuinely rhythmic; on calm music the grid is a metronome the tracker imposed, so pace by phrases and energy instead and never hard-cut to it. Deciding which case you're in is each frame'spacing(Step 2). - One frame = one file; groups live inside. Step 2 cuts the track into frames, and each frame becomes one composition file
compositions/frames/NN-<frame_id>.html, built by one frame-worker. A frame can subdivide into groups (each a template or a motion-primitives combo). Extra density goes inside a group, so frame count tracks distinct treatments, not beats — a fast track does not blow up the number of sub-agents.
Step 0: Setup, BGM, and inputs
Goal: Establish the music source, create the HyperFrames project, and note any user-supplied media.
The brief starts at the intent layer. Opening rule, in order: (1) BRIEF.md exists → read it and ask nothing it answers — its flow/storyboard derive the mode (brief contract § 1). (2) No BRIEF.md but the project exists → resume from what's on disk; never re-interrogate. (3) A fresh creation request that arrived here directly → read /hyperframes and run its intent layer (references/intent-interview.md): it confirms this route's must-haves (the music source, destination → aspect — ../hyperframes/references/routes/music-to-video.md) and announces what stays deferred — brand and genre are chosen at Step 3 by design. Write BRIEF.md immediately after init (never before — init refuses a non-empty directory) and record the preference-backed answers (brief-format.md). Edit requests skip all of this.
The music is the spine — establish one track before anything else. This skill is tuned for fast, high-energy BGM: a strong beat grid drives the cuts (calm tracks work, but pace by phrase rather than beat). If the user supplied audio — a music file, or a video to pull audio from — use it. Otherwise choose the mood from the request and generate a track through /media-use (references/bgm.md). Before the first authenticated provider action, run npx hyperframes auth status and relay its output verbatim. If signed out, apply one branch:
- Collaborative: wait for sign-in or an explicit choice to continue offline with the local provider.
- Autonomous: state the status and continue through the available local provider.
If no offline provider can satisfy the required music capability, surface the blocker. Never write keys into a per-repo .env. Auth ownership and offline fallbacks live in /media-use references/setup-providers.md § Providers. The resulting track lands at assets/bgm.mp3. Stage supplied images or videos so frames can use them on the beat grid; otherwise typography carries the video.
Lyric videos: for lyrics synced to the vocals, get word/line timing by transcribing the track via /media-use, or ask the user for the lyrics text and place lines on the beat grid.
Initialize only if hyperframes.json is missing. Name <project> from the brief in kebab-case, such as midnight-drive-loop — never a timestamp. init checks the installed skills against the latest on GitHub and updates the global set if any are out of date.
npx hyperframes init "videos/<project>" --non-interactive --example=blank --skill=music-to-video
mkdir -p "$PROJECT_DIR/assets" "$PROJECT_DIR/renders"
cp "<user-music>" "$PROJECT_DIR/assets/bgm.mp3" # extract from a video first if needed
# only if the user gave you images/videos:
node <SKILL_DIR>/scripts/stage-assets.mjs --from <dir> --hyperframes "$PROJECT_DIR" --into publicThe brand (font + palette) is chosen at Step 3, not here. Don't pick a genre or a track type up front — assets are just an optional ingredient, and the genre emerges from the per-frame choices.
Gate: hyperframes.json + assets/bgm.mp3 exist; aspect / length / fps and (if any) the asset inventory are noted.
Step 1: Analyze the music
Goal: Produce the one canonical timing analysis the whole video is built on.
analyze-beatgrid.py is the only beat analyzer — never re-measure beats with another tool or by ear. It reads the track once and writes audiomap.json: energy phases (level / density / feel), onsets + onset_rate, rolls, silences, hard_stops, key_moments, phrases, tempo / grid, and audio.duration_sec. It's deterministic — the same file always gives the same map. Most fields are reliable on any music; bpm and beats_sec are reliable only when the music is genuinely rhythmic, and judging that is the call you make at Step 2.
Prerequisites: Python 3 with librosa, numpy, and soundfile available. If import fails, install them into the active Python environment before running the analyzer:
python3 -m pip install librosa numpy soundfilepython3 <SKILL_DIR>/scripts/analyze-beatgrid.py "$PROJECT_DIR/assets/bgm.mp3" \
-o "$PROJECT_DIR/audiomap.json" --printGate: audiomap.json exists; audio.duration_sec is known.
Step 2: Frame skeleton (structure only)
Goal: Read the music and lay out the frames — the skeleton of STORYBOARD.md.
Read references/frame-skeleton.md. Turn audiomap.json into the skeleton of STORYBOARD.md yourself — there is no intermediate JSON. Cut the track into frames at real musical changes (hard_stops, SURGE / DROP key_moments, the edges of a roll, a stretch with no onsets, a big energy jump), snapping every boundary to an audiomap anchor. For each frame set span_sec, pacing (the verdict from Step 1's trust call — beat_cut when the grid is real, phrase_flow when it's a metronome imposed on calm music), mood, and a one-line feel (the plain music situation Step 3 matches a template against). Only classify and lay out here: leave every frame's ### Groups as TBD (Step 3) and the frontmatter style blank — no templates, copy, color, or fonts. Expect ~1–6 frames.
Gate: frames tile the track (first at 0, last at duration_s); each carries span_sec + pacing + mood + feel; every ### Groups is TBD; no content anywhere.
Step 3: Fill the plan (user-gated)
Goal: Turn the skeleton into an approved, complete STORYBOARD.md.
Read references/planning.md, storyboard-format.md, template-catalog.md, motion-primitive-catalog.md, and montage.md (only if the user supplied assets). Editing the same file in place, do two things:
- Pick the brand. Choose one preset from
../hyperframes-creative/frame-presets/using the table in../hyperframes-creative/references/design-spec.md(match the track's mood; only its fonts and colors matter — templates own composition). Copy it intoframe.mdunmodified and fill the frontmatterstyle(font + a ≤4–6 swatch palette) from it. - Fill every frame. Decide its groups and give each a treatment: a matched template from the catalog (with bound params and real audiomap anchors), a free-compose from the primitive catalog, or an asset treatment that obeys
pacing. Before you free-compose a named look, search the live catalog for it: for every look, effect, treatment or transition the user asked for — "CRT scanlines", "glitch", "film grain", "shimmer sweep" — runnpx hyperframes catalog --query "<the look, in plain English>" --jsonand read the top results.template-catalog.mdandmotion-primitive-catalog.mdlist only this skill's own local materials; the search ranks the whole hosted registry (~400 blocks and components) and needs nothing installed — no project, no prioradd, no account. Free-compose a look only after a search for it came back with nothing that fits. Write the copy. You own WHAT (template / primitives + content + anchors); the frame-worker owns HOW — never write millisecond tweens into the storyboard.
node <SKILL_DIR>/scripts/validate-plan.mjs --storyboard "$PROJECT_DIR/STORYBOARD.md" \
--audiomap "$PROJECT_DIR/audiomap.json" --templates <SKILL_DIR>/references/templatesFix every ✗ (hard errors: duration mismatch, frames not tiling the track, a missing src); warnings are best-effort. Then present the frame-by-frame summary in chat as a proposal (../hyperframes/references/review-loop.md § 1) and iterate on the user's replies until they approve; for storyboard: yes, also write it as storyboard.html (../hyperframes-creative/references/storyboard-recipe.md § 3) for them to open. In autonomous mode this is a checkpoint gate: post the summary as a heads-up and proceed (the validate-plan.mjs pass is a quality gate and still blocks).
Gate: frame.md is a verbatim preset copy; validate-plan.mjs exits 0; the user approved the plan (autonomous: the summary was posted as a heads-up).
Step 4: Build frames from the plan
Goal: Build every frame as a self-contained composition file.
Create compositions/frames/. Read sub-agents/frame-worker.md and ../hyperframes/references/subagent-dispatch.md. Dispatch one frame-worker per frame, in parallel where possible (otherwise in waves). Each worker gets exactly one frame and this context:
PROJECT_DIR: <abs path>
frame_id: <NN-frame_id> # = the frame file stem, e.g. 02-f2; the composition id
Your block: the `## Frame N — <frame_id>` block in PROJECT_DIR/STORYBOARD.md
audiomap: PROJECT_DIR/audiomap.json
frame.md: PROJECT_DIR/frame.md
Materials: for each group, <SKILL_DIR>/references/templates/<id>/index.html (templates) and
<SKILL_DIR>/references/motion-primitives/<id>/ (free); staged assets/ (asset groups)
Contracts: ../hyperframes-core/references/sub-compositions.md + determinism-rules.md
Canvas: <w>×<h> Pacing: <beat_cut|phrase_flow>
Write to: PROJECT_DIR/compositions/frames/<frame_id>.htmlThe worker forks the cited materials, converts every anchor to frame-local seconds (local_t = track_t − span_sec[0]), gates its groups with 0ms cuts, and writes one seek-safe frame file. The worker never runs the hyperframes CLI — those commands operate on the assembled project, which doesn't exist yet, so they'd report on the wrong files. The worker just writes to the contract and stops; you verify after assembly (Step 6). As each worker returns, you can confirm its file landed on disk.
Gate: every frame has its compositions/frames/NN-*.html on disk.
Step 5: Assemble
Goal: Wire the built frames + BGM into the playable index.html.
assemble-index.mjs is deterministic — no subagent, no judgment. It references each frame file at its cumulative data-start, mounts assets/bgm.mp3 on track 11, and hard-cuts frame → frame (frames tile the track with no gaps, so there is no transition injector).
node <SKILL_DIR>/scripts/assemble-index.mjs --storyboard "$PROJECT_DIR/STORYBOARD.md" \
--hyperframes "$PROJECT_DIR" --audiomap "$PROJECT_DIR/audiomap.json"Fix any ✗ it reports — a missing or blank frame file means that worker wrote a partial file; re-dispatch it (Step 4) and re-assemble.
Gate: index.html exists; total duration == audiomap.audio.duration_sec.
Step 6: Verify and render
Goal: Verify the assembled video, get user approval, and render the final MP4.
Run the CLI on the assembled project — that's the correct unit (the per-frame workers couldn't run it). check runs structural lint and the headless-browser runtime, layout, motion, and contrast gate in one pass; --snapshots also emits the review frames.
( cd "$PROJECT_DIR" && npx hyperframes check . --snapshots )Inspect at t=0, each frame start, the strongest DROP / SURGE, every hard_stops[].t, and the final frame. On failure, make the cheapest safe fix yourself: edit the offending compositions/frames/NN-*.html. Never change duration or audio timing to hide a sync issue. Once the gates pass, pause for user review, then render only on approval (autonomous mode: ask the one kept question — "preview first, or render?" — then deliver the MP4 with the contact sheet):
( cd "$PROJECT_DIR" && npx hyperframes render . --skill=music-to-video -q draft -o renders/video.mp4 --fps 30 )Gate: check passed and the snapshots were inspected; the user approved (autonomous: checks passed and the delivery includes the contact sheet); renders/video.mp4 exists with audio, duration == audiomap.audio.duration_sec. The final reply states the MP4 path and duration.
Resume table
| You have | Continue from |
assets/bgm.mp3 only | Step 1 |
audiomap.json | Step 2 |
STORYBOARD.md (skeleton) | Step 3 |
STORYBOARD.md (complete) | Step 4 |
| all frame files | Step 5 |
index.html | Step 6 |
Quick Reference
Formats: landscape 1920x1080 by default; portrait 1080x1920; square 1080x1080. Set the canvas once in the storyboard frontmatter (canvas: { w, h, fps }).
Scripts under scripts/: analyze-beatgrid.py (the one analyzer), validate-plan.mjs (plan check), assemble-index.mjs (index assembly), stage-assets.mjs (stage user media), lib/storyboard.mjs (vendored parser). Everything else is the hyperframes CLI.
| Read | When |
references/frame-skeleton.md | Step 2: read the music, lay out the frames, set pacing |
references/planning.md · storyboard-format.md | Step 3: pick the brand, fill each frame, write the plan |
references/template-catalog.md | Step 3: pick a template per group |
references/motion-primitive-catalog.md | Step 3/4: L0 recipes for free-compose |
references/montage.md | Step 3/4: asset treatments (beat-cut / ken-burns) |
sub-agents/frame-worker.md | Step 4: dispatch + build one frame |
../hyperframes/references/subagent-dispatch.md | Step 4: dispatch sub-agents safely |
../hyperframes-creative/references/design-spec.md | Step 3: pick the preset (the brand) |
Directory layout
music-to-video/
SKILL.md
references/ frame-skeleton.md · planning.md · storyboard-format.md
template-catalog.md · motion-primitive-catalog.md · montage.md
templates/<id>/ { index.html (+ assets/ · program.json) } ← L1 catalog impls
motion-primitives/<id>/ { index.html (mounts the scene), scene.html (the sub-composition) } (+ ../assets/gsap.min.js shared by recipes) ← L0 catalog impls
scripts/ analyze-beatgrid.py · assemble-index.mjs · validate-plan.mjs · stage-assets.mjs · lib/storyboard.mjs
sub-agents/ frame-worker.md ← the one subagent (one per frame)Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

