MiniMax H3 — prompting and ComfyUI setup

MiniMax H3 is an open-weight omni-modal video model: it generates video and native stereo audio in a single forward pass, at 24 fps, with a trained clip length of roughly 5–15 s. It ships as two checkpoints — fl2va (frame-conditioned) and ref2va (reference-conditioned) — which are different weights, not modes of one model.

This skill follows MiniMax's own prompt-writing guides and adds the failure modes those guides do not cover.

• references/prompting.md — the official output format for T2VA / I2VA / FL2VA / L2VA
• references/reference-mode.md — the official six-section format for full reference (Ref2VA)
• references/templates.md — fill-in templates for every mode
• references/troubleshooting.md — symptom → cause → fix, from real failures
• references/comfyui.md — checkpoints, quants, VRAM, node-by-node settings
• references/reel-to-prompt.md — rebuilding a reference clip: measure its cuts, read its frames, write the prompt from what is there
• references/reel-modes.md — ControlNet / Motion Strip / Hybrid selection, source dialogue and mode-specific readiness

When the user supplies a reference clip or a link to one, do not describe it from memory. Run tools/reel_shots.py first — it downloads the clip, detects every cut, writes frames at each shot's head, middle and tail, and lists the valid 17k+5 lengths bracketing the source duration. Then read those frames. Beat timings taken from a measured cut list are the difference between a rebuild that feels right and one that floats, because H3 reads beat duration literally as event speed. Also report which shots need which reference photos — front, three-quarter, back, close-up — since that list is what the user has to supply and cannot guess.

0. Before writing a prompt

Use the current two-mode Reel Maker for reference-based reels. Read skills/minimax-h3-reel-maker/SKILL.md and follow its workflow-specific instructions. With a source clip to reconstruct, select Ref + pose/depth ControlNet + mandatory TS Pose Keypoint Smoother. Without a source clip, select Ref Only: photos and text, without requiring source footage, pose, depth, ControlNet, TS smoothing or source audio. One relevant photo may be sufficient in Ref Only. Do not ask a mandatory mode question when the supplied assets already select the branch. This is automatic preparation routing, not permission to install dependencies, rent compute or submit a render.

An explicit Motion Strip, Hybrid or legacy two-image ControlNet choice overrides these defaults; for those advanced alternatives read references/reel-modes.md. These remain preparation workflows within Ref2VA, not additional checkpoints. Source-inspection and source-transcription requirements below apply only when a source video actually exists. The dedicated skill's instructions take precedence for its two bundled graphs; the general H3 material below remains applicable to other generation modes.

Require a description for each source video. First read any attached text or matching sidecar (source.txt, description.txt, описание видео*.txt, or a clearly mapped message). If it already explains that video's intended meaning, do not ask again. Otherwise ask for a short description for that specific video before writing its final prompt: what happens, why the moment matters, required details and intended changes. For several videos, request a filename/ID-to-description mapping; do not reuse one description for unrelated clips. Source inspection may continue meanwhile. Treat this description as guidance for intent and emphasis, and the footage as evidence of visible/ audible facts. Resolve material contradictions explicitly instead of ignoring the text or turning an interpretation into an observed fact. If the user explicitly declines a description and asks to proceed from the video alone, honour that choice and record the limitation.

Inspect the soundtrack for every source reel in every workflow. When speech exists, extract a timestamped transcript and include the actual dialogue in the complete H3 prompt automatically; no separate transcription request is needed. Identify who speaks in-frame and offscreen, preserve the original words/language, and distinguish music lyrics from conversation. Mark unclear words and verification limits rather than guessing or claiming listening that did not occur. The source descriptions, wardrobe, scene, props and relevant facial performance must be checked against the footage in every mode.

H3 prompts are long and expensive to iterate, and the wrong mode wastes the whole render. Ask rather than guess when any of these is unclear — one question up front is cheaper than a bad eight-second generation:

• Which mode? If the user has an image, establish whether it is a frame (I2VA/FL2VA/L2VA) or a reference (Ref2VA). This is the single most consequential fork and users rarely state it. "Do you want the video to literally start from this photo, or just to feature this person?" settles it.
• Which assets exist? How many reference images, whether there is a reference video or audio, and what each is supposed to contribute. A reference with no assigned role is the top Ref2VA failure.
• Duration. Frame count snaps to a grid and the timeline must match it; a prompt written for 10 s rendered at 124 frames is crushed into 5.
• What must not change — identity anchors, wardrobe, location, grain.
• Whether a real camera move is required, since that is the weakest axis and may need a reference video or a different mode.

State a recommendation rather than only listing options, and say plainly when a request will not work as asked — for example a back view demanded from a frontal close-up through I2VA. Flag the trade-off, propose the mode that does work, and proceed.

Split long or multi-cut sequences into separate generations. One generation per shot, joined afterwards in an editor, beats one long generation with cuts written into it. Every extra shot inside a single render costs coherence, and frames drive VRAM and time directly, so a 15-second multi-cut sequence is both the most expensive and the least reliable way to ask. Prefer splitting whenever the sequence has more than about two cuts, whenever it runs past roughly eight seconds, or whenever a beat needs a length very different from its neighbours. A locked-off camera makes the joins invisible, so a static sequence can be cut into as many pieces as convenient at no cost. Where the framing must match exactly across a join, generate the next clip as I2VA from the previous clip's final frame. Black frames, hard cuts and any transition are cheaper and cleaner added in the editor than generated.

Commit to one exact length. Never offer a range. This is an automated pipeline: the user should paste a number, not weigh options. Forbidden: about 3–4 seconds, roughly 5 s, either 124 or 141, 3.5 s or so. Required: a single grid value with the duration it actually yields — length 90 (3.75 s).

The seconds are derived, not chosen: frames ÷ 24, to two decimals. Do not round them to a tidier number — 90 frames is 3.75 s, never "about 3.8" and never "3.5". When two grid values are both defensible, pick one, state it, and give the reason in a clause; do not hand the choice back.

Always state a recommended length when the user has not given one. Put it after the prompt block, as frames and seconds — length 192 (8.00 s). Derive it rather than guessing: budget each beat its real-world duration, add a second of settle at the end, then round up to the nearest 17k+5 value. Duration is read literally as event speed, so an over-long clip does not give the model room — it gives you slow motion. The table in references/prompting.md lists realistic durations for common events; the short version is 124 for a single action on a static camera, 158 with one camera move, 192 for an entrance or approach, 209 for action → reaction → settle, and 243+ once there are cuts. For dialogue, count words at roughly 2.7 per second. Mention the cost when it matters: frames drive VRAM and render time directly, and past 362 the model is out of distribution.

For ControlNet or Hybrid, resolve frame alignment before applying the duration defaults below. The maps and target must contain the same actual frame count; upward rounding does not create control frames. In Hybrid, build the strip from separate phases of that same selected interval and use its FPS, crop and target count for both controls. Follow references/reel-modes.md for a selected crop or agreed extension policy and preserve the user's choice.

A measured reference video sets the length, overriding the defaults above. Its duration, rounded up to the nearest 17k+5 value and capped at 362 frames (15.08 s), is the length — because a reference video longer than the target is truncated to the target. A 14.90-second motion reference asked for at 124 frames delivers only its first 5.17 s of choreography; the remainder is absent, not compressed. Detect the cuts before choosing anything: with no cuts and a source inside 15.08 s it is one clip at the bracketing grid value; with cuts, ask whether the user wants one render per cut or a single full pass (possible only under 15.08 s total); past 15.08 s, one render cannot hold the take, so ask whether to split it into consecutive segments or keep one section. When clips cover segments of one source, trim the reference video to each segment before wiring it — truncation keeps the head, so an untrimmed file makes every clip copy the opening. And when the job is to reproduce a reference's choreography, write the description as a beat every one to two seconds across the whole length; a summary like "a few quick adjustments" leaves the model to invent the rest, and explicit text outweighs a video reference.

Label every prompt in a split sequence. When a sequence is delivered as several prompts, put a header line above each code block naming the prompt number, the reference images that prompt needs, and its length — Prompt 2 — <Picture 1>, <Picture 3> — length 90 (3.75 s). The user is wiring different references per clip and setting a different length each time; making them reconstruct that from the prompt body is where mistakes happen.

Always emit the complete prompt, in one code block, ready to paste as-is — including on follow-ups that change a single word. Never reply with only the edited section, never "replace the beats block with this", never [rest unchanged] or ... standing in for text already written. The user copies and pastes; a fragment forces a manual merge and invites typos. Put nothing inside the block but the prompt itself: the alignment instruction line where the mode requires one, then every section in order, complete. Explanation of what changed goes after the block, in prose, as brief as the change deserves. Length is never a reason to abbreviate.

Terminology. Use MiniMax's names — T2VA, I2VA, FL2VA, L2VA, Ref2VA — when talking about modes, and the weight-family names fl2va / ref2va only when talking about checkpoint files. The aliases T2V / I2V / FLF2V / R2V are common in the wild; recognise them, but do not emit them.

Provenance. SOURCES.md tags every claim in this skill as Official, Implementation, Empirical or Community. When a user leans on a number, check which it is — several widely repeated figures (2K output cap, 7000-character prompt field, 12-file reference limit) are community reports with no primary source, and the empirical rules are tendencies rather than guarantees. Say which kind you are relying on when it matters to the decision.

1. Pick the mode

fl2va = first/last frame → video+audio. ref2va = reference → video+audio.

Decision rule: if the value is in the picture — its room, light, grain, composition — use fl2va. If the value is in who or what is in it, use ref2va. fl2va is animation: it takes the frame and moves it. ref2va is casting: it takes the subject and shoots a new scene.

A shot needing a viewpoint that does not exist in the source photo (back view, wide, different room) is a full-reference job. Forcing it through I2VA makes the model hallucinate the missing body while rotating, which is where identity collapses.

There is no video-to-video, and therefore no frame-preserving character swap. Every H3 node in ComfyUI starts sampling from an empty latent; keyframe and reference latents are conditioning re-injected each step and never denoised. No frame of a reference video survives into the output. The video editing and video continuation task types are not thereby proven inert locally — the plumbing is identical for every task type and only the summary prefix differs — but no source frame is preserved whichever prefix you write. Whether the model responds differently to them under local inference is untested.

But do not over-correct — the useful version of that request is the tool's main purpose. "A new video in which the person resembles my reference photo and moves the way this clip does" is exactly reference generation, and it works. Expect recognisable likeness rather than face-swap identity: H3 is a generative video model, not an identity adapter. Distinguish the two explicitly when a user says "swap", because they usually mean the second one and will be wrongly discouraged by a flat no.

Likeness improves, in order of payoff: two to four reference photos of the same person from different angles, all merged into one <Subject N>; identity in <Subject N> rather than a standalone <Picture N>; ref_image_size: max on the final render; short camera travel, so less unseen geometry has to be invented; a higher-precision text encoder if VRAM allows, since identity flows through it in reference mode.

2. Output format

T2VA / I2VA / FL2VA / L2VA — an alignment instruction line (except T2VA), one blank line, then three fields:

integrated_multimodal_description: [Shot 1] ...

overall_soundscape: ...

non_diegetic_music: ...

Full-reference — six sections, in this order:

subject_definitions:   what each referenced item is and what it contributes
summary:               [task type] one paragraph
retention_analysis:    per-label fidelity markers
detailed_description:  shot-by-shot body, normally 350–500 words
overall_soundscape:    ambience and physical sound
non_diegetic_music:    audience-only score, or N/A

Shots: [Shot 1] carries no timestamp. Later shots open with a strictly increasing cut time — [Shot 2] At 00:03.500, the camera cuts to …. Open [Shot 1] with the style: [Shot 1] Live-action, cinematic, a medium-wide shot frames …

Number dialogue speakers by first actual vocal event. The first voice is (S1), even offscreen; each later new voice receives the next number. Subject IDs do not decide speaker IDs. In detailed_description, write speaker, visibility and delivery outside <d>[Language] exact spoken words.</d>. Keep speaker IDs out of retention_analysis. Run python tools/check_prompt_dialogue.py <prompt-file> for a file-backed dialogue prompt; it checks formatting/numbering, not transcription accuracy.

3. Reference labels

Four labels, and picking the wrong one is the most common structural mistake.

Identity lives in <Subject N>, never in a standalone <Picture N>. The guide is explicit: if an image only defines a character, scene, costume or style, do not give it its own picture entry — cite it inside the subject definition.

One subject may draw on several assets, and that is how you resolve reference conflicts:

<Subject 1> is the woman whose appearance comes from <Picture 1> and whose walking
motion comes from <Video 1>.

That single line is the official answer to "take the motion from the video and the face from the photo". You do not forbid the video from contributing a face — you define one subject and state what each source supplies. Anything reused as visible content from a video belongs to <Subject N>; <Video N> only names the asset or its structure.

Motion follows the selected reel workflow. In ControlNet, the source supplies aligned pose/depth IMAGE batches to H3 ControlNet Apply, separately from the Ref2VA image references. The default image inputs are <Picture 1> face and <Picture 2> body; there is no required strip, composition keyframe or <Picture 3>. Describe wardrobe, scene, props, complete motion and facial acting explicitly. A control map is not a Ref2VA <Video N>; see references/reel-modes.md for compatibility and alignment. In Motion Strip, selected source frames become a chronological image reference; that mode needs no ControlNet weights or preprocessors. Use a <Video N> label only when a clip actually occupies a Ref2VA video slot.

In Hybrid, ControlNet pose/depth maps patch the same Ref2VA ref2va MODEL branch, while the motion strip is ordinary Ref2VA image conditioning. Its default images are <Picture 1> face/identity, <Picture 2> body proportions and <Picture 3> motion strip; <Picture 4> is an optional explicitly chosen look/composition frame. This is not a new checkpoint, an automatic <Video 1>, or a claim of better quality. The maps control the complete selected sequence; the strip supplies separate chronological phases of that same interval, while the written timeline controls pace rather than strip panel count.

The three-slot reference convention (Motion Strip only). This workflow's default is three Load Image nodes: <Picture 1> the identity photo (passport-style — frontal, evenly lit, plain background, face large), <Picture 2> the look frame (a composed still of the subject as she appears in this video: wardrobe, hair, location, framing), <Picture 3> the motion strip (a horizontal contact sheet of the reference clip's frames, chronological left to right). Labels follow slot index, and no <Video N> exists unless a clip is wired into a video slot — writing one then points at a label that is not in context. Cite <Picture 1> inside <Subject 1>; cite <Picture 2> for wardrobe and location and give it its own entry as the composition anchor; cite <Picture 3> inside a subject that defines the action progression, scoped weak_reference. A strip carries poses and their order but no timing at all, so the written timeline is the only thing setting pace — take its numbers from a measurement of the source clip anyway. Choose enough distinct, readable phase panels for the action; at ten panels an 1800-pixel strip leaves ~180 px per pose.

Identify before you describe — the zoom pass. A contact sheet resolves pose and trajectory and nothing else; at six panels across an 1800-pixel strip each frame is about 300 px wide. Before naming any object the subject holds or touches, crop it from the full-size frame and look — ffmpeg -ss T -i src.mp4 -frames:v 1 -vf "crop=W:H:X:Y,scale=2*W:2*H" zoom.png. A plausible guess is what gets rendered: a hair video makes "comb" plausible when the object is a makeup pencil held up like a plumb line, and the meaning of the gesture goes with it. Zoom on the same pass for marks on the reference actor that must be excluded — tattoos, jewellery, a watch — since you cannot exclude what you never saw. If it is still unidentifiable after zooming, say so and ask; a confident wrong noun becomes a rendered prop.

Find the through-line before writing the panels. A strip is usually one continuous movement sampled at intervals, not six independent poses. State what travels and in which direction, then write the panels as waypoints on that path — six equal-weight poses can be reordered into any activity, a stated trajectory cannot. Naming the single travelling element and stating that the rest of the body stays quiet blocks the wrong activity by construction.

Write a motion strip panel by panel, and count your motion words. Enumerate the panels in order, one sentence each, using ordinals — she begins … next … then … finally — because the model cannot count and prose second-counts schedule nothing. And weigh the vocabulary: repeated ambient movement (bouncing, shifting weight, moving to the beat, mouthing along) outcompetes a key action named once at the end, and the render comes back as dancing. Name incidental motion once in a subordinate clause; give the key action a sentence in every panel it appears in. Crop burned-in captions off the strip rather than forbidding them in retention_analysis — repeated across every panel, they are its most prominent content.

Name the distinguishing details — the picture does not describe itself. Small, low-contrast, off-centre features are the first thing flattened toward an average face: a face tattoo, freckles, a mole, a scar, a piercing, an unusual iris colour. Whatever is not named in subject_definitions can come back generic. Name each one by body landmark and side (a small black cross tattooed on her right cheekbone just below the outer corner of her eye), measure rather than compare (hair falling to her collarbone, not long hair), restate the same features in retention_analysis behind the fidelity marker, and copy the wording verbatim into every shot and every separate render. Scale sets the ceiling — a centimetre-wide mark is a few pixels in a wide shot and will not survive it, so frame close where it has to read. Ask which details must survive when a reference first appears; users rarely volunteer them and always notice when they are gone. Full treatment in references/reference-mode.md.

4. Camera motion

Official vocabulary. Write it as a natural English action inside the shot, never as labels stacked at the end.

Type — Zoom In / Zoom Out (focal length, body still) · Push In / Pull Out (body moves) · Pan Left / Right (pivot horizontally) · Truck Left / Right (translate horizontally) · Tilt Up / Down (pivot vertically) · Pedestal Up / Down (whole camera rises or drops) · Arc Shot (arc around the subject) · Tracking Shot · Static Shot · Shake Slightly / Strongly · POV · Roll Clockwise / Counterclockwise

Amplitude — with small amplitude, with large amplitude. Speed — at slow speed, at fast speed. Medium amplitude and normal speed are simply omitted.

The camera pushes in with small amplitude at slow speed toward the folded letter in her hands.
The camera arcs around her with large amplitude at slow speed as the lamp sweeps across frame.

Prefer camera motion over a cut when only distance or angle changes. A cut should introduce new information about subject, space, state, viewpoint or time.

5. Hard rules

These are empirical — none of them appear in MiniMax's guides.

Structure beats instruction. Anything you can make impossible by construction should be made impossible by construction rather than forbidden in words. A second shot is prevented by a lowered muzzle and drifting smoke, not by no second shot. A subject spinning instead of the camera is prevented by describing background parallax, not by she does not turn. A reference video bleeding its actor into your output is prevented by a merged <Subject N> definition and a weak_reference marker in retention_analysis, not by do not take the person from <Video 1> in the body.

This is not "never exclude anything" — it is about where the exclusion lives. A fidelity marker is a fixed value in MiniMax's documented format, which is a stronger convention than an ad-hoc sentence; but ComfyUI parses nothing, so it arrives as ordinary prompt tokens either way. That scoping there beats the same words in the body is empirical, not mechanical. What is solid is the general observation: a ban is text competing against a data signal, and the data usually wins.

The stock BasicGuider template has no negative prompt. It uses one conditioning input, CFG effectively 1. Prefer positive descriptions of the desired state. A custom ControlNet workflow may use a different guider with negative conditioning: inspect its actual graph instead of asserting that all H3 workflows lack that input. Changing the guider is a separate tuning decision; do not silently replace the user's working configuration.

The model cannot count, and bans amplify what they ban. exactly one shot is a token sequence, not a constraint, and no second shot puts second shot into the conditioning with no negative channel to subtract it. Name an event once, in one shot, with no prohibition attached, then block repetition through scene state.

Never write camera as a noun the subject interacts with. H3 renders she holds the camera as a prop. Use the official camera vocabulary for the movement, describe the subject's arms separately, and keep the two in different sentences — joined by a verb of possession they resolve into an object. When the device is the viewpoint, use POV and never mention it at all; a visible outstretched arm is what sells the grip.

Impossible poses produce body horror. Camera directly behind + body not rotating + eye contact needs a 170° neck twist, and the model resolves it by inverting the head or blending front and back anatomy. Stop at three-quarter, let the shoulders rotate with the head, and lock anatomy explicitly.

Shot and beat duration is read literally as event speed. A fall given 1.5 s renders as a 1.5-second fall — weightless, moon gravity. Real falls take about half a second. Budget the real duration, spend the remaining time on the aftermath, and remember that weight comes from the stop rather than the drop.

Big physical events need intermediate poses and room in the frame. "She falls" is an outcome the model smooths away. Give the trajectory, and make sure the framing actually contains the ground.

One primary camera move per shot. A secondary tilt or pan that keeps the subject framed during that move is fine — a pedestal up with a compensating tilt down is one operation, not two. What collapses into mush is several independent, equal-weight moves competing in the same shot.

Iterate one variable at a time, at a fixed seed.

6. Limits

Verified: 24 fps, the 17k+5 frame grid, a trained range of roughly 124–362 frames. Community-reported and not found in the primary sources: output up to 2K with a 1440 short edge, and a 7000-character prompt field — do not present either as documented. detailed_description runs 350–500 English words for generation tasks, with documented exceptions: dialogue-dense content departs from the range to fit the spoken timeline, and editing descriptions scale with the source. Full-reference slots in ComfyUI: 9 reference images, 3 reference videos, 3 same-index video soundtracks, 3 standalone audio clips. Standalone audio is accepted on its own — the node processes it unconditionally, despite community write-ups claiming otherwise. The reference-video tooltip recommends 2–15 s, but the code enforces only a 5-frame minimum, and a clip longer than the aligned target frame count is truncated to that, then trimmed down until its own frame count also satisfies 17k+5. No 12-file total or combined-duration cap exists in the source.

Frame count snaps to a 17k+5 grid, and anything off it is rounded up silently — multiples of 4 are not the rule. The node accepts 5 to 3600; the minimum is five frames, about a fifth of a second, not five seconds. Valid values at 24 fps:

5 = 0.21 s · 22 = 0.92 s · 39 = 1.63 s · 56 = 2.33 s · 73 = 3.04 s · 90 = 3.75 s · 107 = 4.46 s · 124 = 5.17 s · 141 = 5.88 s · 158 = 6.58 s · 175 = 7.29 s · 192 = 8.00 s · 209 = 8.71 s · 226 = 9.42 s · 243 = 10.13 s · 260 = 10.83 s · 277 = 11.54 s · 294 = 12.25 s · 311 = 12.96 s · 328 = 13.67 s · 345 = 14.38 s · 362 = 15.08 s

The tooltip calls ~124–362 the trained range and marks longer as untested. It says nothing against shorter: values below 124 are accepted and simply unproven, not forbidden. A one- or two-second beat is a legitimate choice — do not tell the user 124 is a floor.

7. What ComfyUI does with your prompt

There is no rewriter. MiniMax's guides describe the output of their own rewriting model, but ComfyUI tokenizes your text verbatim — not chat-templated, no special tokens. So the documented structure has to be written by hand if you want it at all. Whether reproducing it outperforms an equivalent plain-prose prompt has not been measured here; this skill uses it as the documented default, not as a proven win.

The alignment instruction is not emitted for you. ComfyUI prepends only "<Picture 1>: " and the image. The For the target video, at 0.00 seconds… and How the reference pictures align… lines from the guide are your text — type them as the first line of the prompt box, followed by a blank line.

ComfyUI injects the reference labels itself, before your prompt, in a fixed category order: all images, then videos — each video's soundtrack label placed immediately before its own <Video k> — then standalone audio. Ordinals are 1-based per type and follow slot number, not the order you happened to wire things. Two consequences: a video's soundtrack takes <Audio 1> ahead of any standalone audio, and moving an image from ref_image_2 to ref_image_0 renumbers it without touching your prompt. When you write <Picture 1> you are pointing at a label that already exists in the context.

Slot maxima on the node: 9 images, 3 videos, 3 video soundtracks, 3 standalone audio. The sockets auto-grow, so a fresh node showing three image inputs is not the limit.

Reference audio never reaches the text encoder — only its label does; the waveform goes to the DiT separately. Reference video reaches the encoder at 2 fps, so a 5-second clip arrives as roughly ten frames.

See references/comfyui.md for the exact ref_image_size formulas, the native canvas, and the undocumented MiniMaxH3SigmaShift node.