For the opt-in video.model_profile route, read PROFILED-VIDEO.md. It uses exact retained frame spans and full-master audio; legacy options are not automatically compatible, and no global LTX-2.5 default is promoted.

Music Video Skill

Author a YAML spec → run one command → get a full music video.

The loop

Nine-step workflow with two quality-check gates where the pipeline stops and waits for a human (or reviewing agent) to confirm before the next expensive phase. A bad song variant or a broken base anchor will ruin every downstream scene — don't skip the gates unless you're sure.

1. init <slug> — creates ~/.openclaw/workspace/<slug>/ with a skeleton song.yaml. Use --theme "<text>" to inject context for the lyrics/style fields. Example: music_video.py init new-beginnings --theme "fresh start"
2. Fill song.yaml — title, style (producer-style brief — see ~/.openclaw/skills/suno-mcp/references/style-guide.md), and lyrics ([Verse]/[Chorus]/[Bridge] tags, one line per phrase, no mid-line punctuation — see ~/.openclaw/skills/suno-mcp/references/lyrics-guide.md). Leave scenes empty for now.
3. song <spec> — generates spec.suno.runs * 2 suno variants (song.mp3, song_v2.mp3, song_v3.mp3, ...). Restart-safe: re-runs only top up missing variants.
4. QUALITY GATE 1 — pick the best variant. Listen to every song*.mp3, pick the one you want, back up or delete song.mp3, then rename your chosen variant to song.mp3. Keep the others as song_vN.mp3 — they become parallel final_vN.mp4 renders during assembly.

5. MANDATORY — transcribe the chosen song for word-level timestamps. You cannot eyeball scene start_sec/duration_sec. Whisper STT against the actual audio is the source of truth. Run:

   # comfy_graph.py uploads the audio to ComfyUI's input dir, runs
   # 'Apply Whisper' + 'Save SRT', and downloads three artefacts to the
   # spec dir: <prefix>.txt (plain transcript), <prefix>_segments.srt
   # (line-level cues), <prefix>_words.srt (per-word cues — use this for
   # lipsync alignment).
   comfy_graph.py stt --audio song.mp3 --prefix <slug> \
     --output-dir <project-dir> --language English

   Use the segment SRT cues to set start_sec/duration_sec so each scene starts on a vocal phrase boundary and ends just before the next one. The orchestrator will slice the song audio per scene at exactly these boundaries, so misalignment shows up as words cut in half across scene cuts.

   If you have separated stems (e.g. from Suno's stem-export tool, or comfy_graph.py stems on the full mix), transcribe the relevant stems individually and merge. Lead vocals, backing vocals, and the full mix can each surface lines the others miss:

   • Lead-vocal stem alone may drop hooks/answer-phrases that live only in the backing-vocals layer (e.g. a refrain "Stable altitude" answered only by the backing vocal).
   • Whisper on the full mix can hallucinate text where backing-vocal bleed sounds like a vocal phrase. Run STT on each stem with a different --prefix, diff the segment SRTs, and reconcile timing conflicts in favour of the cleanest stem for each line. Don't trust a single pass.

   Defensive note: if stt fails with HTTP 413, your ComfyUI server (or the proxy in front of it) has a body-size limit on /upload/image below your audio file's size. Either fix the proxy or re-encode the audio for the STT pass only: ffmpeg -i song.mp3 -c:a libmp3lame -b:a 24k -ar 16000 -ac 1 song_stt.mp3 — that re-encoded copy is fine for transcription but do NOT use it as the ia2v audio source (LTX's audio VAE wants the full-quality input).

   If Suno gave you a stem pack (zip of .wav + .mid per stem), unpack it first so the MIDI is available to the bar-grid analyser:

   unpack_suno_pack.py <Title>.zip --out <project-dir>
   # → <project>/stems/{vocals,drums,bass,guitar,synth,fx,backing_vocals}.wav
   # → <project>/midi/<same>.mid

   Skipping this step costs you the drum tempo map and the per-stem MIDI phrase boundaries — both crucial for the next step.

5b. OPTIONAL but recommended — combine MIDI bar grid + vocal phrases + Whisper STT into a ranked scene-boundary proposal. This catches musically-wrong cuts that the segment SRT alone won't:

```bash
analyze_midi.py  <project-dir>                    # report tempo / bars
analyze_song.py  <project-dir> \
    --whisper-srt <project>/<prefix>_segments.srt \
    --scenes <N> --min-spacing 4
```

The output is a table of `[start, dur, lyric]` per scene, snapped to
bar boundaries where possible, that you can drop into `scenes:`.
Mostly: the song's BPM dictates a bar length (e.g. 123 BPM → 1.944s);
every scene end that doesn't land within ~100ms of a bar will feel
musically off in playback. Use this output as the starting point even
if you adjust some boundaries by hand.

6. Add the scene list to song.yaml — each scene: label, start_sec, duration_sec, prompt, image (@anchor | @last | path). Optionally give a scene an anchor: block to pre-render a flux2 key frame for it. Lift start_sec/duration_sec straight from analyze_song.py's output (step 5b) when you have a MIDI, or from <prefix>_segments.srt (step 5) otherwise. Total duration must equal the song length (probe with ffmpeg -i song.mp3 if unknown).

Validate every boundary against <prefix>_words.srt before rendering. Segment-level SRT marks line endings, but vocalists routinely sustain final phonemes 0.3-1s past the line's nominal end — if your scene ends inside that sustain, LTX cuts the operator's mouth in the middle of "do" or "say" and the next scene's overlay audio carries the lingering vowel. Open _words.srt, find the line you're cutting on, and ensure your start_sec + duration_sec lands on the END of the last word in the line (or in a clear silent gap), not the "end" reported by the segment SRT. A 30-line audit script: python3 -c "import re; ..." over _words.srt flagging boundaries that fall inside word spans is faster than reviewing the final video. 7. anchors <spec> — flux2 renders the top-level anchor_image (from anchor_prompt, or fallback title + style) plus any per-scene anchors. Idempotent — skips files already on disk. 8. QUALITY GATE 2 — review the anchors. Open every PNG under <project>/ and <project>/scenes/. If anything is wrong (wrong face, wrong mood, bad composition), DELETE that PNG and re-run anchors <spec> — tweak the scene's anchor.prompt or top-level anchor_prompt first. A bad base anchor propagates into every LTX scene that chains off it. 9. scenes <spec> — renders every scene (LTX ia2v with the song slice + resolved image). Restart-safe. Iterate one at a time with scene N <spec>. 10. assemble <spec> — concat all scene MP4s, overlay the clean suno audio → final.mp4 (plus final_vN.mp4 for each extra song variant).

Running the whole thing

music_video.py all <spec> runs steps 3→10 in order. It STOPS automatically at gates 1 and 2 (controlled by gate_confirm_song / gate_confirm_anchors in the YAML, default both true). To run fully unattended, either flip both flags to false in song.yaml or pass --no-gate:

music_video.py all <spec>              # stops at gate 1, then gate 2
music_video.py all --no-gate <spec>    # skip both gates, full autopilot

Restart-safe throughout: anything already on disk is skipped. Iterate individual scenes with scene N song.yaml.

Day-1 decisions (lock these before step 7)

A few choices propagate into every downstream prompt and every render. Flipping them mid-pipeline forces you to throw away anchors and redo them. Settle these up front, when only song.yaml exists and no GPU time has been spent:

Anything else (per-scene prompts, camera LoRAs, transitions on/off, individual anchor tweaks) is cheap to iterate on later — those only re-render one scene at a time and don't propagate.

Command index

Every step of the pipeline maps to one CLI subcommand in scripts/music_video.py. The table below cross-references each subcommand to its handler in code so authors can read the exact behaviour, and lists every spec field the step consumes.

Quality gates — cmd_all stops after song (gate 1) and after anchors (gate 2) by default. Override per-spec via gate_confirm_song: false / gate_confirm_anchors: false, or per-invocation via --no-gate.

Workflow-step column (1, 3, 7, 9, 10) — corresponds to the numbered phases in The loop above. Steps 2/4/5/6/8 are human review/decisions (variant pick, STT, scene authoring, anchor review), not CLI commands.

Using with openclaw agents

Any agent with music-video in its agents.list[<id>].skills array can run this skill end-to-end. The gates make it safe to delegate: the agent runs all, waits at gate 1, and posts the variants back for a human review. After renaming, the agent is prompted again and runs all a second time to pass gate 2, and so on.

Example — asking Athena (char6166r) to start a project:

openclaw agents run char6166r --message "Start a music video project: create a song about <theme>, init it in your workspace, and launch the suno generation."

Athena will init, fill the YAML, run song, then report the variants back (gate 1). After a human renames the chosen variant, Athena resumes with scene authoring → anchors → gate 2 → scenes → assemble.

Long renders MUST go through sessions_spawn — never block the agent

scenes, transitions, all for any non-trivial spec take 30 minutes to several hours. The agent invoking those commands must NOT exec them directly and tail stdout — that locks the agent's turn for hours, floods the chat with [18:21:43] transition 2→3 done lines, and prevents any further user message from being handled.

The right pattern is OpenClaw's sub-agent mechanism: sessions_spawn (or /subagents spawn) creates a background child run with push-based completion. The parent returns to the user within seconds; the child runs the long subprocess; when the child finishes, OpenClaw wakes the parent with the result. Do not poll sessions_list or /subagents list — the announce is automatic.

Skeleton the agent should follow (pseudocode for the parent's tool call):

sessions_spawn({
  agentId: "<self>",          // same agent, isolated child session
  context: "isolated",        // child gets the brief only — no transcript fork
  task: """
    Run the music-video scenes phase for /workspace/<slug>:
      uv run --script /home/sandbox/.openclaw/skills/music-video/scripts/music_video.py \\
        scenes /workspace/<slug>/song.yaml
    Stream nothing back during the run. When complete, report:
      - exit code
      - the count of files in <slug>/scenes/*.mp4
      - the path to <slug>/run.log
  """,
  runTimeoutSeconds: 21600,   // 6 h ceiling — adjust per spec size
})

The parent immediately tells the user "Render started in the background, I'll let you know when it's done" and yields its turn. The next visible parent turn is the announce when the child finishes (success or fail).

Same pattern for transitions, all, and assemble when the spec is large. The short-running commands (init, plan, song, anchors, single scene N) stay inline — they finish within an agent turn budget.

For raw subprocess detachment without spawning a child agent (you don't need it here, but it's the same shape), see OpenClaw's background tasks — nohup, setsid, or disown on a wrapped command, with the agent later inspecting the run.log to surface status.

Script paths (absolute; no ~)

The canonical install path is ~/.openclaw/skills/music-video/scripts/music_video.py. ~/.openclaw/… works in an interactive shell but NOT always under exec, so when invoking from another agent or process pass an absolute path:

• OpenClaw sandbox: /home/sandbox/.openclaw/skills/music-video/scripts/music_video.py
• Host install: $HOME/.openclaw/skills/music-video/scripts/music_video.py
• Fresh clone: <repo>/music-video/scripts/music_video.py

The script uses a uv run --script shebang with inline deps (pyyaml, imageio-ffmpeg). Invoke it via uv run --script so deps auto-install:

uv run --script /path/to/music-video/scripts/music_video.py <cmd> ...

Plain python3 music_video.py … will fail with ModuleNotFoundError: yaml unless you've already installed pyyaml in your env. Use uv instead — it's the designed path.

Commands

# Prefix every command below with the uv invocation above.
music_video.py init <slug> [--theme=<text>] [--force]   # create project skeleton under ~/.openclaw/workspace/<slug>/ (sandboxed agents get /workspace/<slug>/)
music_video.py plan     <spec.yaml>       # validate + show scene breakdown
music_video.py song     <spec.yaml>       # suno only — N*2 variants (spec.suno.runs)
music_video.py anchors  <spec.yaml>       # flux2 top-level + per-scene anchors (idempotent)
music_video.py scene N  <spec.yaml>       # one scene's ia2v (1-3 min)
music_video.py scenes   <spec.yaml>       # every scene in sequence
music_video.py transitions <spec.yaml>    # LTX transition clips per boundary (runs if video.transitions.enabled)
music_video.py assemble <spec.yaml>       # concat + overlay song audio (splices in transitions)
music_video.py all      <spec.yaml> [--no-gate]   # end-to-end, stops at the 2 gates by default
music_video.py status   <spec.yaml>       # what's already generated

Typical for a 3-scene 60-second video: ~6 minutes wall time (song ~4 min, scenes ~45s each sequential, assembly a few sec).

YAML spec

Minimum viable:

title: Glass Harbour
style: "folk noir, coastal Americana, 74 BPM, fingerpicked acoustic guitar, bowed upright bass, sparse brushed snare, foghorn field recording, salt wind ambience, weathered male vocal, close-mic room reverb, mournful tone"
lyrics: |
  [Verse]
  Salt in the air and rust on the crane,
  the fishermen leave before the morning train.
  [Chorus]
  Glass harbour, glass harbour,
  where the cold light bends.

video:
  fps: 24
  resolution: [1024, 576]
  negative: "pc game, cartoon, modern tech, text, ugly"
  crossfade: 0.5                      # optional; ffmpeg xfade between scenes
  camera_lora_strength: 0.8           # default applied per-scene if camera_lora is set
  fast: true                          # optional global default for scene[].fast (iteration shortcut; flip off for final)
  # lipsync_audio: song_lipsync.mp3    # optional — vocal-forward remix used for ia2v
  #                                      conditioning only. song.mp3 stays canonical
  #                                      for assembly. Hard-fails if set but missing.
  #                                      Per-scene override: scene[].lipsync_audio
  #                                      (use null to force song.mp3 on instrumental
  #                                      / guitar-riff scenes where the vocal-forward
  #                                      mix would mute the music dynamics).

anchor_image: anchor.png                # first scene uses this; omit → t2v
# anchor_prompt: "..."                  # optional — drives `anchors <spec>` for the
#                                         top-level PNG. Falls back to title + style.

scenes:
  - label: establishing
    start_sec: 0
    duration_sec: 8
    prompt: "wide fog-shrouded harbour at dawn, lonely crane silhouette, salt spray, muted palette"
    image: "@anchor"
  - label: fisherman
    start_sec: 8
    duration_sec: 10
    prompt: "weathered fisherman hauling nets, breath visible, early light"
    image: "@last"                      # chains from prev scene's last frame
  - label: lighthouse
    start_sec: 18
    duration_sec: 8
    camera_lora: static                 # optional: dolly-in/out/left/right, jib-up/down, static
    prompt: "lighthouse blinks through fog, beam sweeps across waves"
    image: "@last"

Complete schema reference

Every key the orchestrator reads from a spec, cross-referenced to the function in scripts/music_video.py that consumes it. Anything not listed is silently ignored.

Top-level

video.*

video.transitions.*

scenes[] items

scenes[].anchor.*

When anchor.prompt is set, _generate_anchor flux2-pre-renders a PNG per scene and uses it as that scene's first frame, overriding image.

Anchor-type cheatsheet (full discussion in the storyboard skill):

scenes[].guides[] items (multi-guide)

Resolved via storyboard.lib.guides.resolve_guides.

scenes[].transition_from_prev.*

Goes on the INCOMING scene (B-side), not scene A. Read by cmd_transitions.

The ic_lora dict shape:

transition_from_prev:
  duration: 4.0
  ic_lora:
    file: ltx-2.3-22b-ic-lora-union-control-ref0.5.safetensors
    strength: 1.0                           # LoRA model strength
    reference_video: cond/22-23-spin.mp4    # or reference: <png>
    reference_strength: 1.0                 # AddGuide weight (0..1)
    frame_idx: 24                           # where the cond starts in the latent

reference_video is auto-trimmed by cmd_transitions if it would overflow (length_frames - frame_idx) / fps — the assertion latent_idx + guide_latent.shape[2] <= latent_length in LTXAddVideoICLoRAGuide.execute is enforced upstream. A trimmed copy is written next to the transition mp4 as …-ictrim.mp4.

Ingredients mode (project-level character/prop/location consistency)

When a music video must keep the same character, props and locations consistent across every scene, condition each scene on ONE project reference sheet (black-bg element panels) via the LTX-2.3 ingredients IC-LoRA, instead of authoring a per-scene flux2 anchor.

1. Build the sheet once with the storyboard skill:

   storyboard/scripts/generate_reference_sheet.py --out sheet.png \
     --character cast.png --character-desc "…" \
     --prop "x=…" --location "…" \
     --width 1024 --height 576        # = video.resolution (aspect MUST match)

2. Point the project at it and flag the scenes:

   reference_sheet: sheet.png          # project-level (or per-scene override)
   video: { resolution: [1024, 576], fps: 24 }
   scenes:
     - label: chorus-1
       start_sec: 41.0
       duration_sec: 5                  # ~121f @ 24fps is the trained bucket
       prompt: "the singer on the neon rooftop, holding the mic, rain"
       mode: ingredients                # render via the reference sheet

mode: ingredients is checked first in the scene dispatch (authoritative over image/guides). The render is silent by design — the song is muxed over the whole timeline at assemble time and assemble.py strips per-clip audio anyway, so no audio slice is conditioned. The scene's generic video.negative is dropped in favour of the ingredients LoRA's tuned negative (set a per-scene negative: to override). Tune with ingredients_lora_strength (def 1.4) / ingredients_reference_strength (def 1.0). Trained buckets are landscape (768×448, 960×544); portrait is off-bucket — evaluate. Compose freely with camera LoRAs and transitions. See the comfyui skill's ingredients command and storyboard's "Reference sheets" section.

Camera LoRAs

Seven cinematic motion LoRAs are installed on the comfy server and can be applied per-scene via camera_lora: <name>:

Defaults to no LoRA if unset. Strength via camera_lora_strength (top-level default 0.8; overridable per-scene). Pair the LoRA name with a matching prose description of the same move in the prompt — the LoRA reinforces the intent.

Scene transitions (LTX, not ffmpeg)

Boundaries between @last-chained scenes usually land with a visible seam — ia2v conditioning drifts enough that adjacent scenes don't quite meet. The transitions stage renders a short LTX clip per boundary that morphs from scene A's tail into scene B's head, using real video guides from both sides (not still frames) so the motion continues across the cut.

Enable + tune in video.transitions:

video:
  transitions:
    enabled: true
    duration: 4.0            # total transition length (seconds)
    guide_sec: 1.0           # real-video guide on each side; middle = dur - 2*guide_sec
    fps: 24
    default_b_sparse: "96"   # "96" = 1 single B-anchor frame at the tail (default)
                             # "72,80,88,96" = 4 sparse B-anchors (smoother into singing)

The resulting shape for a 4.0s / 1.0s boundary is 1s A-guide → 2s masked morph (audio drives the invention) → 1s that ends on scene B's head. Assemble overlaps the transition with scene A's trailing duration/2s and scene B's leading duration/2s so the final timeline stays bit-exact with the song.

Per-boundary override goes on the INCOMING scene (not scene A) via transition_from_prev:

scenes:
  - label: chorus_in           # boundary is prev_scene → this_scene
    transition_from_prev:
      duration: 4.0
      b_sparse: "72,80,88,96"  # 4f — use INTO lipsync / singing scenes
      prompt: "…optional override for this boundary's morph…"

Hard cut: set duration: 0 to skip the LTX transition for one boundary. The assemble step butt-cuts straight from prev scene's tail to this scene's head — useful on drum-hit / chorus-crash entries where you want the impact, not a smooth morph. Pair with a punchy visual change (palette flip, location jump) for maximum bite.

scenes:
  - label: chorus_crash
    transition_from_prev:
      duration: 0              # ← hard cut, no LTX transition rendered

Defaults that work: 4s duration, 1s guide, default_b_sparse: "96" (1f). 4f ("72,80,88,96") was empirically the sweet spot for boundaries into singing shots — more anchors prevent identity drift when the character immediately has to sing. Contiguous multi-frame B-blocks (e.g. 16 frames at positions 80-96) caused a mid-transition freeze at snap-in; always keep the B-side sparse (every 8 latent frames).

b_sparse positions that map to BEFORE scene B starts are dropped, not clamped. A 4s transition between c2_breath (ends 130.02s) and tunnel_drop (starts 132.0s) puts the boundary centre at 130.02 → transition spans [128.02, 132.02]. Latent positions 72/80/88 then map to song times 131.02 / 131.35 / 131.69 — all BEFORE scene B's start at 132.0 → negative src_frame in next_video. The old max(0, x) clamp silently locked all four anchors to scene B's frame 0, killing the morph's freedom in the empty middle (every output snapped to scene B's static head by latent 72). The current behaviour skips those positions with a stderr warning and proceeds with whatever anchors remain. For boundaries where scene B starts in the back half of the transition window, override b_sparse: "96" explicitly to keep just the tail anchor.

Multiguide stitching (the canonical seamless-transition pattern)

When a morph should bridge two scenes with continuous structural motion through the empty middle (not just smooth-pixel-interpolate from A's tail to B's head), use the multiguide stitch: A-side LTXVAddGuide at frame_idx=0 + IC-LoRA cond at frame_idx=24 + single B-side anchor at latent 96. The IC-LoRA's structural reference video carries continuous geometry through the morph's empty middle, the morph LoRA blends the endpoints, and the single tail anchor lands scene B cleanly without yanking the latent.

Recipe:

1. Render a time-aligned cond for the transition's song-time window via midi2canny.py --polyfield-only (or tunnel_3d.py, etc.). For a boundary spanning song time [T_a, T_b]:

   midi2canny.py --polyfield-only \
     --drums midi/drums_audio_aligned.mid \
     --vocals midi/vocals_audio_aligned.mid \
     --guitar midi/guitar_audio_aligned.mid \
     --start-sec T_a --duration (T_b - T_a) \
     --fps 24 --width <w> --height <h> \
     --poly-count 24 --poly-rings 3 \
     --output cond/N-N+1-trans.mp4

   Audio-aligned MIDIs are critical — the polyfield's drum-hit scale pulses and vocal-pitch rotV impulses then line up with what the LoRA will be conditioning the audio against. A cond rendered for the wrong song-time window produces an output that "looks the same every render" because the cond's temporal arc doesn't reinforce the audio's arc.
2. Wire the cond into the per-boundary transition_from_prev.ic_lora block on scene B with frame_idx: 24 (= 1.0s @ 24fps = the empty middle's start). The non-zero frame_idx is critical: a guide at frame_idx=0 collides with the A-side LTXVAddGuide's keyframe time-start, which makes comfy_extras.nodes_lt.LTXVCropGuides undercount num_keyframes (it uses torch.unique(time_starts)) and pass-2 refines the un-stripped guide frames as content — 5-9s glitched output instead of clean 4s. frame_idx: 24 puts the cond past the A-side block.
3. Override b_sparse: "96" if scene B starts in the back half of the transition window (see warning above). Single tail anchor only — gives the morph LoRA the full empty middle to follow the cond.
4. Match scene B's IC-LoRA cond to its own song-time window. Scene 23 spans 132-139s, so its hdr_lora.reference_video should be a cond rendered with --start-sec 132 --duration 7, NOT the transition's cond (which covers 128-132s). The output looks the same every render if the cond's temporal arc doesn't match the audio's arc.
5. If scene B's intro is static, scene B has to be rerendered with an action prompt. The morph LoRA interpolates A → B; if B's first frames are a held shot, the morph lands on a held shot. Rewriting scene B's prompt to "action from frame one" + bumping its IC-LoRA reference_strength is what unlocks "polygons whipping past the face" instead of "face holding still until the drum hit at +2.8s".

Worked example for the stable-altitude v9 c2_breath → tunnel_drop boundary:

- label: tunnel_drop
  start_sec: 132.0
  duration_sec: 7.0
  transition_from_prev:
    duration: 4.0
    b_sparse: "96"                          # single tail anchor — see warning above
    ic_lora:
      file: ltx-2.3-22b-ic-lora-union-control-ref0.5.safetensors
      strength: 1.0
      reference_video: cond/22-23-spin.mp4   # midi2canny --polyfield-only for 128-132s window
      reference_strength: 1.0
      frame_idx: 24                          # past A-side AddGuide block
  hdr_lora:
    file: ltx-2.3-22b-ic-lora-union-control-ref0.5.safetensors
    strength: 1.0
    reference_video: cond/23-tunnel_drop-v2.mp4   # midi2canny for 132-139s — TIME-ALIGNED to scene
    reference_strength: 1.0
  prompt: >
    {operator}'s face flickering at the center of an EXPLODING geometric
    mandala of neon-cyan + sodium-orange polygons that spin and shatter
    outward from frame one, polygons whipping past the face, … continuous
    violent motion through the drum slam at 2.8s into the explosion peak.

Multi-reference IC-LoRA (multiple LTXAddVideoICLoRAGuide nodes chained under different cond videos) is NOT supported today. Upstream comfy_extras.nodes_lt.LTXVCropGuides.get_keyframe_idxs counts num_keyframes = unique(keyframe_idxs[:, 0, :, 0]) — when 2+ guides share any pixel-frame time-start, the unique-count undercounts and pass-2 refines the extras as content. Workaround: only one IC-LoRA per transition. The transition_from_prev.loras (list) schema is reserved but rejected at yaml-load if used.

Multi-guide scenes (guides: field)

A scene whose first frame can't/shouldn't show the character — e.g. opens on an empty landscape or a prop before the character enters — needs a mid-scene character anchor or LTX will invent a random identity. Add a guides: list with pre-rendered flux anchors at specific times:

scenes:
  - label: meets_hare
    start_sec: 98.110
    duration_sec: 13.630
    image: "sheets/mid_anchors/hare_candle_first_00001_.png"   # candle only
    prompt: "Wide dusk-meadow — a candle on a stone; {hare} cartwheels around it; {snakebird} shields the flame…"
    guides:
      - image: "sheets/mid_anchors/hare_character_00001_.png"
        at_relative: 0.5         # halfway through the shot
        strength: 1.0

Each guide entry: image (literal path, @anchor, or @last) + one of at_sec / at_relative (0..1) / at_frame + optional strength (default 1.0). Frame indices snap to multiples of 8 (LTX latent quantization). For long shots (13s+), add two guides (e.g. at 0.5 and 0.9) to hold identity through the whole shot — a single mid-anchor drifts by the tail.

When guides: is present, the scene renders via LTX multiguide (chained LTXVAddGuide). The shared implementation lives in the storyboard skill (lib/guides.py) so drama-video and one-off renders use the same resolver.

A fuller reference with all optional fields is at references/example.yaml. Format details in references/song-format.md.

Folder layout (auto-created)

<project>/
  song.yaml
  song.mp3
  song_meta.json
  song_slices/
    001-establishing.mp3
    002-fisherman.mp3
    003-lighthouse.mp3
  scenes/
    001-establishing.mp4
    001-establishing-last.png
    002-fisherman.mp4
    002-fisherman-last.png
    003-lighthouse.mp4
    003-lighthouse-last.png
  final.mp4
  run.log

<project> = the directory containing song.yaml. The script writes everything there.

Prompting recipe (for agents filling the YAML)

1. Style brief is a producer-style text-to-music prompt. See ~/.openclaw/skills/suno-mcp/references/style-guide.md for the full pattern (genre, BPM, instruments, production, vocal, mood, narrative sentence). Avoid keyword lists.
2. Lyrics use [Verse]/[Chorus]/[Bridge]/[Instrumental] tags. One line per phrase; no mid-line punctuation. See ~/.openclaw/skills/suno-mcp/references/lyrics-guide.md.
3. Scenes should map to lyric structure. For a ~3-minute song, prefer shorter, more numerous scenes (~6-8s each, 20-30 total) over a small number of long ones. Long scenes (15s+) are at LTX's edge and benefit less from re-rolls. Short scenes give you variety per chorus and let you place dancer close-ups / cutaway beats between operator-lipsync passes. Align start_sec/duration_sec with the actual song structure — duration accuracy matters for audio-reactive feel.
4. Scene prompts should describe what's on screen — not tell a story. LTX-2.3 handles short camera moves well (dolly, pan, push-in) and struggles with cuts. Keep each scene as a continuous shot.
5. Image chain with @last for continuity between adjacent scenes; use @anchor or a literal path to reset to a different look (verse/chorus transitions).
6. Resolution: 1024×576 is a good default for music videos. 768×512 is faster. Higher eats GPU. (LTX's 2× upscaler then doubles to the final output — see video.anchor_scale.)
7. Negative prompt: use the video.negative field to blacklist "ugly, pc game, cartoon, text" etc. Keeps LTX-2.3 from drifting.
8. Cold-open / first appearance. Don't put the protagonist in scene 1 by default. A short atmospheric t2i / i2i (curtain, empty stage, exterior establishing) for the instrumental intro, then cut to the protagonist's first appearance exactly when a defining instrument enters (typically the drums — use analyze_midi.py to find the first drum hit). The cut feels earned because the protagonist's entrance lands with the beat that just kicked in. The cold-open boundary is almost never the first vocal — it's the first drum hit, which is usually 1-3s earlier.
9. Variety beats inside each chorus. A chorus rendered as ONE 15s scene of the protagonist lipsyncing reads as static. Split it into 3 shorter (4-7s) scenes: wide-stage establishing, close-up of a different character (a dancer mouthing the answer hook), push-in on the protagonist's face. Same lyric, three different camera languages. This is the single biggest "professional music video" lever short of using transitions.
10. Featured-character cameos (call-and-response, duets, eye-contact moments). When the lyric is about engagement — call-and-response, "I wanna hear you say", a partner's answer — having the operator literally turn and meet eyes with another person makes the song's emotional argument visible. Two patterns:
    • i2i2 against [operator, group_setting] picks the partner from the recurring cast (a chorus dancer). LTX renders an interchangeable face — fine for chorus formations, bad for "the singular partner" moment.
    • i2iN against [operator, setting, featured_singer.png] uses a dedicated reference image for the duet partner so they're visually distinct from the recurring cast. Make featured_singer.png a t2i upfront with explicit "different from any of the dancers" language (different hair, different outfit, different posture). Reuse this ref across every eye-contact / duet scene to keep her identity consistent.
11. Hand anatomy guidance for crowd shots. flux2 will hallucinate extra hands the moment multiple people are mid-gesture in the same frame ("operator with mic raised + dancers with arms up" → often produces a third hand on the operator). Write the anchor prompt with explicit positional language per character (e.g. "operator's right hand thrusting the chrome mic forward, his left hand at his side; dancers behind with both arms straight up overhead in symmetric V") and add anatomically correct hands to the prompt tail. Same fix for "operator + featured singer sharing a mic" — say which hand holds the mic.
12. Avoid long sustains at scene cuts. Scenes that end mid-sustain (held "ever", held "do", held "say") cut LTX's lipsync mid-phoneme. Move the boundary to after the sustain ends — check _words.srt for the exact word-end time, not the segment-end time. This is the single most common source of "the lipsync looks weird at the cut" reports.

Troubleshooting

"song.mp3 missing" when running scene → run song first, or use all.

"scene N: @last requested but ...-last.png does not exist" → previous scene hasn't been generated yet; run them in order or use all.

Final video's audio feels off → each scene is conditioned on its audio slice, but the final pass overlays the clean suno track. If a scene's video is mis-synced, regenerate that scene (scene N spec.yaml) — suno audio is not re-generated, just the video.

Suno login required → mcporter call suno.suno_login --config ~/.openclaw/config/mcporter.json first.

Scenes visually inconsistent → pin @anchor on the first scene and chain @last through the rest. Add specific character/place descriptors in every scene prompt, not just the first.

UnicodeEncodeError on Windows → the script prints unicode glyphs (→ ✓ ⚠) and writes them to run.log. As of the latest version, music_video.py reconfigures stdout/stderr/log file to utf-8 at startup, so this should "just work" on PowerShell / cmd.exe. If you're on an older copy and still see 'charmap' codec can't encode character '→', prefix the invocation with PYTHONIOENCODING=utf-8 or upgrade the skill.

Lipsync feels off / words cut at scene boundaries → your start_sec/duration_sec are landing inside word spans rather than at word ends. Don't trust segment SRT timings for this — vocalists sustain final phonemes past line-end. Open <prefix>_words.srt, find what word is being sung at each scene's end_sec, and shift the boundary to after that word's end. Re-render only the affected scene + the one starting at the new boundary; everything else is preserved by the restart-safe scene cache. See "Avoid long sustains at scene cuts" in the prompting recipe.

Cuts feel "off the beat" → your boundaries are not bar-aligned. Run analyze_midi.py <project> to get the bar grid (e.g. 123.4 BPM → 1.944s/bar), then nudge each end_sec to the nearest bar boundary that also respects a word-end (the two usually coincide within ~150ms for well-recorded vocals).

mido.KeySignatureError: Could not decode key with 16 sharps → Suno's stem-pack midi exports occasionally emit key signature meta events outside the standard MIDI spec. The current analyze_midi.py / analyze_song.py pre-populate mido's decode table with safe fallbacks so the file loads — if you're on an older copy, monkey-patch mido.midifiles.meta._key_signature_decode to add fallback entries before the first MidiFile() call.

Advanced (not in v1)

• Beat detection & waveform-linked light effects → not implemented. If needed, add a separate post-process pass with ffmpeg's showwaves / asetnsamples + blend modes, fed by a beat track from librosa.onset.onset_strength.
• Lyric subtitles → not generated. Add an SRT track separately.