Video Generation

Workflow Overview

1. Phase 1: Initial → Gather requirements, STOP for user confirmation
2. Phase 2: Global Definitions → Define style, characters, voices, BGM (text only, no images)
3. Phase 3: Clip & BGM Planning → Segment into clips, plan each clip, and create the final BGM blueprint
4. Phase 4: Reference Images → Generate reference images (MANDATORY before Phase 5)
5. Phase 5: Execution → Purely execute the blueprint: generate keyframes, videos, and all audio assets

Critical Rules (MUST Follow)

Before starting, memorize these non-negotiable rules:

1. [PHASE 1 STOP] If the user has not explicitly specified certain details, MUST ask questions to gather the missing information. DO NOT assume or guess—always ask the user for clarification. Never proceed without explicit user confirmation.
2. [DETAILED VIDEO PROMPT] Video prompts must include detailed transition_description (2-4 sentences). One-line prompts are insufficient.
3. [KEYFRAME DIFFERENCE] Last keyframe must show interpolatable change from first keyframe: subject position/pose, subject state (open/close, appear/disappear), or composition change. Subtle-only changes (lighting, background) while subject stays static cause unnatural video motion.
4. [PHASE 4 MANDATORY] MUST generate reference images before keyframes. Never skip Phase 4.
5. [ASPECT RATIO] ALL keyframes must use 16:9 or 9:16, and must be upright (not rotated). Never generate 1:1 or other ratios.
6. [NO TTS FOR ON-SCREEN] Never use TTS for on-screen dialogue or singing. Video model generates audio with lip sync.
7. [NARRATION PER SPAN] Generate TTS per narration span, not all at once. For audio-visual sync scenes, prefer one narration per clip; for continuous commentary over B-roll or montage, a span may cover multiple consecutive clips.
8. [AUDIO MIXING] When combining audio tracks (video audio, narration, BGM), preserve ALL tracks—overlay, never replace. Narration must be clearly audible and maintain consistent volume across all clips.

Image Generation Tools

Phase 1: Initial

Gather Information

Five-Dimension Expert Framework

Use these perspectives to guide your questions:

Ask questions across all dimensions. Prioritize based on user's initial description.

[MANDATORY STOP - DO NOT PROCEED WITHOUT USER CONFIRMATION] Summarize gathered information and wait for user confirmation before Phase 2.

Phase 2: Global Definitions (Text Only)

Visual Style Specification

Define these 4 dimensions (applied to primary reference images in Phase 4):

Example specification:

Sub-genre: Cyberpunk anime
Rendering + Line: 2D digital painting, thin glowing outlines
Color + Lighting: High saturation neon (pink, cyan, purple), dark backgrounds, rim lighting
Detail density: Highly detailed backgrounds, moderate character detail

Recurring Elements

For each character/object:

Voice Profiles

• On-screen: From character definitions (dialogue/singing)
• Off-screen narrator: name, gender, tone, pace, language

BGM Source Decision

If Separate, define global BGM properties (consistent across all clips):

• Genre & style
• Tempo & rhythm (BPM) — default locked; vary only if arrangement specifically requires tempo change
• Key & scale — default locked; vary only if arrangement specifically requires modulation
• Core instrumentation — the full sonic palette available for the track (e.g., acoustic guitar, ukulele, piano, brushed drums); not all instruments must play in every section
• Soundscape/ambiance
• Production quality

Phase 3: Clip & BGM Planning

Segmentation Rules

• Clips: 4, 6, or 8 seconds only
• Each clip: one action, one scene

Per-Clip Specification

Field Dependencies

• inter_clip_boundary = continuous → next clip's first_keyframe_reuse = yes
• first_keyframe_reuse = yes → previous clip must have last_keyframe_required = yes

Keyframe Difference Requirement

When planning last_keyframe_visible_content, ensure interpolatable change from first_keyframe_visible_content:

• Subject position/pose change (movement, rotation, action)
• Subject state change (open/close, appear/disappear, expression)
• Composition change from camera movement (zoom, pan result)

[WARNING] Avoid last keyframes with only lighting or background changes while subject remains static—this causes unnatural video motion.

Decision: last_keyframe_edit_from_first

transition_description Requirements

This field directly becomes part of the video prompt. The more detailed, the better.

Must include:

1. Subject appearance: Key visual features that must remain consistent throughout
2. Movement trajectory: How subject/camera moves through space and time
3. State changes: How objects/environment change over the duration
4. Existence statements: What is present throughout (prevents pop-in/pop-out)

Length guideline: 2-4 sentences minimum. One-line descriptions are insufficient.

transition_description Examples

Physical Consistency Check

Common Mistakes:

Narration Planning

Goal: Design narration text that fits comfortably within the video timeline and sounds natural at TTS output speed.

• Too much text → rushed, unnaturally fast delivery.
• Too little text → sparse, disconnected feel.
• No gaps between segments → suffocating, no breathing room.
• Text length is the primary lever at planning stage; speaking rate and pauses are fine-tuned later via SSML.

Step 1: Plan narration segments

Based on the narrative arc and all clips' content, determine how many narration segments the video needs and what each segment conveys.

Then assign each segment to clip(s), considering:

• Descriptive narration (narration directly describes on-screen action) → align the segment to a single clip for tight audio-visual sync.
• Commentary narration (narration provides context or explanation independent of specific on-screen action) → the segment may span multiple consecutive clips if the thought is semantically unbreakable.

Some clips may have no narration at all. When spanning multiple clips, write the full text on the first clip's narration_cue and mark "continues" on subsequent clips.

Step 2: Calculate narration budget

narration_budget = total spanned clip duration (in seconds)

This is the maximum allowed TTS audio duration for this narration span. Write it in the narration_budget field of the first clip in the span.

Step 3: Write narration text within budget

Estimate the text's natural speaking duration to ensure it fits within the budget. Rough reference for sanity-checking:

• CJK languages (Chinese, Japanese, Korean): ~4 characters/sec
• Alphabetic languages (English, Spanish, French, etc.): ~2.5 words/sec
• When unsure about a specific language, err on the side of shorter text — a slightly short narration is always better than a rushed one.

[MANDATORY] Reference Image Requirements

After all clips planned, list required reference images:

[WARNING] Only generate what clips actually need. Do NOT generate all angles by default.

[MANDATORY FINAL STEP] BGM Emotional Arc Blueprint

After all clips have been planned, and ONLY if bgm_source was set to separate, you MUST perform the following summary task. This creates the definitive blueprint for the music.

• Action: Read the full clip plan you just created.
• Task: Create a precise, second-by-second Emotional Arc List in a markdown table. For each clip, calculate the start and end time based on its duration. Extract the bgm_cue for that time segment. Merge consecutive clips into a single row if they share identical bgm_cue values (same mood/emotion, arrangement state, and density & brightness).
• Output Example:

This concludes Phase 3. You now have a complete blueprint for the video and its BGM.

[MANDATORY] Phase 4: Reference Image Generation

MANDATORY. Do not skip to Phase 5.

Generation Order

Step 1: Primary reference (visual anchor)

• Tool: generate_image (no references)
• Prompt MUST include: Full Visual Style Specification from Phase 2 + element description
• White background
• Ends with "no text, no watermarks, no logos, no labels, no annotations"

Step 2: Additional angles/shots

• Tool: generate_image_variation
• References: [primary_ref, other_refs...]
• Prompt: "Edit this image: [changes only]"
• White background
• Ends with "no text, no watermarks, no logos, no labels, no annotations"

[WARNING] Never generate additional refs without using primary ref as reference.

Phase 5: Execution

Global Rules

[CRITICAL] ALL keyframes: aspect ratio from Phase 1 (16:9 or 9:16). Never 1:1.

First Keyframe

first_keyframe_reuse = yes → Use previous clip's last keyframe (no generation)
first_keyframe_reuse = no  → Generate new keyframe

If generating first keyframe:

• Tool: generate_image
• References: Appropriate Phase 4 images
• Aspect ratio: 16:9 or 9:16
• Prompt includes:
  • Visual style (sub-genre + key characteristics, brief)
  • Scene environment
  • Framing (shot size + angle + lens)
  • Visible content
  • Subject appearance + outfit
• Prompt ends with: "no text, no watermarks, no logos, no annotations"

Last Keyframe

last_keyframe_required = no  → Skip
last_keyframe_required = yes:
  last_keyframe_edit_from_first = yes → Edit mode
  last_keyframe_edit_from_first = no  → Generate mode

If EDIT mode:

• Tool: generate_image_variation
• References: [first_keyframe, Phase 4 refs...]
• Prompt: "Edit this image: [changes only]"
• Do NOT repeat unchanged elements

If GENERATE mode:

• Tool: generate_image
• References: [first_keyframe (scene ref), Phase 4 refs...]
• Aspect ratio: 16:9 or 9:16
• Prompt includes:
  • Visual style (brief)
  • Last keyframe framing + visible content
  • Subject appearance and end state
  • "Same location/environment as reference"
• Prompt ends with: "no text, no watermarks, no logos, no annotations"

Consistency Checklist (Easily Overlooked)

When generating last keyframe, verify:

• Interpolatable change: Clear difference in subject position/pose, state, or composition (not just lighting/background)
• Same lighting direction and shadows as first keyframe
• Same color temperature (warm/cool)
• Same depth of field
• Same outfit, facial features, body proportions
• Environment details consistent

Video Generation

Video prompt should be detailed. Even with keyframes, video models may drift during generation.

Prompt includes:

• Visual style (brief)
• Pacing (slow / moderate / fast)
• transition_description from Phase 3 (detailed, 2-4 sentences)
• Subject appearance (key features for consistency)
• Scene environment (brief)
• Audio (see below)

Audio in prompt:

Prompt ending by bgm_source:

• embedded → (no ending, music described in prompt body)
• separate/none → End with "No background music."

Example (music video with embedded BGM):

Hatsune Miku center stage, singing in Japanese with sweet electronic voice: 
"ラララ、光の中で踊り出す", energetic J-pop at 140 BPM with synthesizer, 
crowd cheering, concert atmosphere

[CRITICAL] Never use TTS for on-screen dialogue/singing. Video model generates audio with lip sync.

BGM Generation (if bgm_source = separate)

[CRITICAL] The BGM blueprint (Emotional Arc List) was already created at the end of Phase 3. Your task here is pure execution.

Step 1: Retrieve the Blueprint

• Action: Locate the Emotional Arc List you created at the end of Phase 3.

Step 2: Generate the BGM Prompt

• Action: Read the music-prompter skill to apply its 9-dimension framework and syntax.
• Task: Use the Emotional Arc List as the SOLE SOURCE OF TRUTH for the arrangement breakdown section of the prompt. Translate each row of the table into a timestamped section in the prompt.
• Process:
  1. Start with the Global Directives: Instrumental only, no vocals. Create a [Total Duration]s track at [BPM] BPM. (Get Total Duration and BPM from the plan).
  2. Write the Core DNA: Use the global BGM properties defined in Phase 2.
  3. Write the Arrangement Breakdown: Translate each row of the Emotional Arc List into a [mm:ss - mm:ss] section.

[MANDATORY] The final BGM prompt MUST reflect the exact timing and emotional cues from the Emotional Arc List. Do NOT use a generic template.

Step 3: Generate the Audio

• Action: Use the generate_music tool with the prompt created in the previous step.
• Task: For duration and multi-clip strategy, follow the rules in the music-prompter skill. Do not guess.

[WARNING] DO NOT use Python to generate music.

Narration Generation (if narration exists)

[WARNING] Generate per narration span, not all at once. A narration span may cover one or multiple consecutive clips.

• [MANDATORY] Read the tts-skill for SSML construction and tool usage guidance.
• Same voice profile across all clips.

Audio Summary

Audio Mixing (Final Assembly)

When combining multiple audio sources:

[CRITICAL] Mixing rules:

• Preserve ALL audio tracks—overlay, never replace one with another
• Narration must be clearly audible—not drowned out by other tracks
• Narration volume must be consistent across all clips