ComfyUI Skill

Direct ComfyUI REST API access for image and video generation.

Python dependency provisioning

The scripts require Python >=3.11 and the local creative-comfy-graph 0.1.0 package. It contains graph primitives plus the existing Flux2, TTS, graph-only post and LTX-2.3 builders; uploads, transport, CLI orchestration and local clip staging remain here. The library uses only the Python standard library. Install it explicitly into the interpreter that executes the scripts; imports never auto-install dependencies or search sibling/installed skill copies. For checkout development (uv and Python 3.11 or 3.12 required):

ROOT=/path/to/creative-skills
ENV=/tmp/comfy-env
uv venv --python 3.11 "$ENV"
uv pip install --python "$ENV/bin/python" --link-mode=copy -e "$ROOT/comfy-graph"

Editable installs need the checkout to remain available. For copied skills, create the environment as above, copy the tracked skill scripts together and build/install the local wheel instead (no package source copy required):

DIST=/tmp/comfy-dist
SCRIPTS=/path/to/copied/comfyui/scripts
uv build "$ROOT/comfy-graph" --out-dir "$DIST"
uv pip install --python "$ENV/bin/python" --link-mode=copy --no-deps \
  "$DIST/creative_comfy_graph-0.1.0-py3-none-any.whl"
export PATH="$ENV/bin:$PATH"
python3 -c 'import creative_comfy_graph as p; print(p.__version__, p.__file__)'
python3 "$SCRIPTS/comfy_graph.py"

The no-argument CLI prints usage and exits 1, which is expected. Missing package installation fails before transport work with creative-comfy-graph is required; install it into the Python environment running this script. Copy mode is scoped to each install, avoiding cross-filesystem cache hardlink fallback without global configuration. Keep that PATH in the parent and inherit it into every bare python3 child, including children of a uv parent:

uv run --no-project --python "$ENV/bin/python" python -c \
  'import subprocess; subprocess.run(["python3", "-c", "import creative_comfy_graph as p; print(p.__file__)"], check=True)'

Interpreter selection or a shebang alone does not fix child environments. Do not use PYTHONPATH or rely on installed-skill fallback. This local package is not yet a published immutable dependency pin. comfy-graph/README.md documents the all-family wheel/sdist-rebuild acceptance on actual Python 3.11/3.12, including copied-script startup, plain-Python child routing and missing-package checks. Those offline gates do not claim live GPU/model validation, LTX-2.5 support, consumer migration or deployment. Existing clock/seed/environment and audio/crop behavior is preserved, not corrected. Arbitrary legacy help/dump paths are not guaranteed side-effect-free; only guarded test fixtures are exercised offline.

Native H3 And Queue Admission

Canonical graph code lives in this repository's comfy-graph/; install it into the bot's interpreter before using these adapters. h3-t2v, h3-i2v, and h3-r2v retain their native generated-audio recipes. h3-ia2v --image LOCAL.png --audio LOCAL.mp3 --prompt '...' --frames 124 adds FL8 first-frame/audio conditioning and preserves the normalized source soundtrack. It requires Pillow and ffmpeg/ffprobe; audio must cover frames/24 seconds and be at most 32 MiB. All files/options validate before any upload. dump h3-ia2v is local only; audio support is graph-contract-tested, not live lip-sync evidence. Read references/minimax-h3-prompt-guide.md before authoring actual prompts; short CLI examples are transport fixtures, not model-specific prompt guidance. Exact recipes/evidence are in the canonical repository's comfy-graph/H3-PROFILE.md.

Both core.py and standalone comfy_run.py require a fresh, empty, well-formed backend queue before POST. Busy queues wait within the deadline; malformed queue responses fail closed. POSTs are never automatically retried. Include queue_admission.py, h3.py, and audio_input.py when copying skill adapters. Queue inspection is cooperative admission, not an exclusive server lock.

Server URL configuration

The scripts read three environment variables, in priority order:

Defaults: a fresh clone with no env vars set assumes a ComfyUI server at http://localhost:8188 — the standard port when you start ComfyUI manually (python main.py) or via Docker.

ComfyUI Desktop: the desktop app binds to http://localhost:8000 instead. If you're using Desktop, set COMFY_URL=http://localhost:8000 (or persist it in your shell rc).

Two-server topology (e.g. separate Flux + LTX boxes for VRAM headroom): set both COMFY_URL_FLUX and COMFY_URL_VIDEO. The CLI auto-routes per command — agents don't have to track which server.

OpenClaw sandbox: agents in an agentic-media sandbox have these vars injected at boot via the per-sandbox credentials.env propagation; agent prompts and skill code don't need to mention them.

Output directory configuration

Precedence: OPENCLAW_MEDIA_DIR > /workspace/media/outbound default in sandbox > --output-dir CLI arg (i.e. the env var, when set, overrides the CLI arg for sandbox writes — set the env var per-sandbox, leave --output-dir alone). Mirrors suno-mcp's SUNO_OUTPUT_DIR pattern.

Why it exists: commons-tier OpenClaw sandboxes bind-mount /workspace read-only, so the previous hardcoded /workspace/media/outbound raised PermissionError on mkdir. Set OPENCLAW_MEDIA_DIR to a writable path (e.g. /home/sandbox/outbound) for those sandboxes.

Script paths (use absolute; tilde expansion is unreliable under exec)

The canonical install location is ~/.openclaw/skills/comfyui/scripts/…. In OpenClaw sandboxes that resolves to /home/sandbox/.openclaw/skills/comfyui/scripts/…; on a host install it depends on the user account that ran the install. ~/.openclaw/… works in an interactive shell but NOT always under exec — pass an absolute path when invoking from another agent or process.

LTX-2.3 Video Models (Required)

To use LTX-2.3 video generation, download these models to your ComfyUI installation:

# From your ComfyUI models directory, run:
mkdir -p checkpoints text_encoders loras latent_upscale_models

wget -P checkpoints https://huggingface.co/Lightricks/LTX-2.3-fp8/resolve/main/ltx-2.3-22b-dev-fp8.safetensors
wget -P text_encoders https://huggingface.co/Comfy-Org/ltx-2/resolve/main/split_files/text_encoders/gemma_3_12B_it_fp8_e4m3fn.safetensors
wget -P loras https://huggingface.co/Lightricks/LTX-2.3/resolve/main/ltx-2.3-22b-distilled-lora-384.safetensors

One-liner for copy-paste:

cd /path/to/ComfyUI/models && mkdir -p checkpoints text_encoders loras && wget -P checkpoints https://huggingface.co/Lightricks/LTX-2.3-fp8/resolve/main/ltx-2.3-22b-dev-fp8.safetensors && wget -P text_encoders https://huggingface.co/Comfy-Org/ltx-2/resolve/main/split_files/text_encoders/gemma_3_12B_it_fp8_e4m3fn.safetensors && wget -P loras https://huggingface.co/Lightricks/LTX-2.3/resolve/main/ltx-2.3-22b-distilled-lora-384.safetensors

Alternative - distilled checkpoint (smaller, faster):

wget -P checkpoints https://huggingface.co/Lightricks/LTX-2.3-fp8/resolve/main/ltx-2.3-22b-distilled-fp8.safetensors

Alternative text encoder (fp8 - faster):

wget -P text_encoders https://huggingface.co/Comfy-Org/ltx-2/resolve/main/split_files/text_encoders/gemma_3_12B_it_fp8_e4m3fn.safetensors

LTX-2.3 Model Files Summary

Text Encoder Path

⚠️ Important: LTXAVTextEncoderLoader must use a model name offered by the server. This skill defaults to gemma_3_12B_it_fp8_e4m3fn.safetensors, downloaded directly to text_encoders/.

The skill defaults to gemma_3_12B_it_fp8_e4m3fn.safetensors (fp8 variant) for faster inference.

Quickstart examples

# Image generation (text-to-image)
python3 /home/sandbox/.openclaw/skills/comfyui/scripts/comfy_graph.py t2i \
  --prompt "<your-character> in a forest"

# Image-to-image with reference
python3 /home/sandbox/.openclaw/skills/comfyui/scripts/comfy_graph.py i2i \
  --image <reference.png> \
  --prompt "<your-character> riding a white horse"

# Text-to-video
python3 /home/sandbox/.openclaw/skills/comfyui/scripts/comfy_graph.py t2v \
  --prompt "<your-character> walking through ancient ruins" \
  --seconds 5

# Query server state
python3 /home/sandbox/.openclaw/skills/comfyui/scripts/comfy_query.py stats

# Run workflow from JSON
python3 /home/sandbox/.openclaw/skills/comfyui/scripts/comfy_run.py workflow.json --output-dir /tmp/imgs

SAM 3D Body: video to animated human mesh

Use comfy_graph.py sam3d --video person.mp4 --seconds 5 to extract an animated full-body GLB and a mesh-overlay MP4. Local videos upload automatically; server-side input/ names also work. Routes to COMFY_URL_VIDEO and preserves the source FPS and audio. Requires native SAM3 / SAM3DBody / MoGe nodes and four models; see SAM 3D Body setup and testing.

Start with --seconds 0.25 --batch-size 1 for a smoke test. Defaults enable tracking, RT-DETR, MoGe camera estimation, face expressions, hand refinement, and temporal smoothing. For limited VRAM, reduce the batch size first; only disable tracking on single-person clips. dump sam3d performs no uploads.

If OpenClaw rejects the GLB with "local attachment allowlist", do not regenerate the mesh; see the gateway compatibility fix in SAM 3D Body setup.

Guide video generation with authored WebGL spaces

For camera paths through a 3D environment, render registered RGB and depth from one WebGL scene, then use LTX Union IC-LoRA control (the LTX route for ControlNet-style depth/edge guidance), not merely an RGB first-frame reference. Read the WebGL-to-LTX recipe before authoring or submitting: it includes scene construction, depth calibration, lossless exports, the executed two-pass node wiring and verification gates. For stitching, use first-frame conditioning 1.0 in both applicable passes, not soft 0.7 guidance, and verify the actual shared boundary before dropping any generated frame. The next integration harness is Easel Client storyboards and WebGL exports.

The worked staircase is user-accepted as an illustrative example, not exact geometry/camera compliance, optimal strength, or a seamless generated loop. The approved faster-Canny carried-props and projected-depth sculpture pilots have verified local outputs, including a controlled video-strength 0.6 vs 1.0 comparison; exact speed/topology, perfect looping and universal improvement are not certified. The reference separates video-control, image-reference and LoRA strengths, documents source-audio accounting and audits embedded MP4 graph metadata as untrusted data. At source 394142a, the curated LTX-2.5 CLI/profile has no Union control-video input. Use the reviewed explicit graph only within authorised prototype scope; do not invent flags, silently switch profiles, auto-execute attached graphs or imply Easel API/client guide support.

Working with named character references

For storyworld-style workflows where you have a named character with multiple reference images (cataloged on HuggingFace, with structured yaml metadata), see the companion skill:

https://github.com/venetanji/polyu-storyworld/tree/main/skills/storyworld-references

That skill handles HuggingFace dataset downloads, character-yaml lookup, and reference-image selection; it calls into this comfyui skill for the actual t2i / i2i / video generation. They are intentionally separate so this skill stays generic.

Script Inventory

comfy_graph.py — CLI entry point (most common)

Builds workflows and submits them. All commands support --output-dir, --timeout, --seed.

Sandboxed-agent pattern: --input-json (no wrapper scripts)

Sandboxed agents (commons, per-role lordships) cannot reliably pass long prompts via --prompt "..." on the shell command line — the exec preflight rejects "complex interpreter invocation" when prompts contain quotes, newlines, or shell-special characters. Do NOT write a per-invocation python wrapper file; instead use the two-step write+exec pattern:

1. Write a JSON spec via the write tool (no shell parsing):

   // /workspace/.comfy/job.json
   {
     "prompt": "an elephant playing chess in a smoky 1920s salon, ...",
     "seconds": 8,
     "seed": 42,
     "notify_target": "discord:channel:1234"
   }

2. Run comfy_graph.py with a single argument — no shell quoting of user content:

   python3 /agentic-media/creative-skills/comfyui/scripts/comfy_graph.py \
     t2v --input-json /workspace/.comfy/job.json

JSON keys map 1:1 to the long-form CLI flag names with dashes converted to underscores (notify_target ↔ --notify-target). Any subset is allowed. CLI flags after --input-json still override individual values, so an ad-hoc re-run with a tweaked seed is one flag:

python3 comfy_graph.py t2v --input-json job.json --seed 99

The same pattern works for every subcommand (t2i, i2i, t2v, i2v, ia2v, flf2v, transition, multiguide, multiprompt, i2i2multi, i2iNmulti, tts, …).

t2i          Text-to-image (Flux2)
i2i          Single-reference image edit (Flux2, 1 ref + prompt)
i2i2         Two-reference image edit (Flux2)
i2iN         N-reference image edit (Flux2) — pass --images a.png,b.png,c.png
multiprompt  Many prompts × one reference  — one submission, N outputs
i2i2multi    Many prompts × two references — one submission, N outputs
i2iNmulti    Many prompts × N references   — one submission, N outputs
t2v          Text-to-video (LTX-2.3)
i2v          Image-to-video (LTX-2.3, first-frame)
ia2v         Image + audio → video (LTX-2.3)
ingredients  Reference-sheet → video (LTX-2.3 IC-LoRA; consistent characters/props/location)
flf2v        First + last frame → video (LTX-2.3, converges to the last frame)
transition   Song-aligned cross-scene morph (prev_video + next_video guides + masked middle)
multiguide   Chained LTXVAddGuide — N anchors at N latent positions within one clip
tts          Text-to-speech (Qwen3)
dump         Print workflow JSON only, no execution
last_frame   Extract last frame from a server-side video file

⚡ When the user asks for "variants" / "multiple versions" / "N angles" / "variations"

USE multiprompt or i2iNmulti — NEVER a shell for-loop. Each of these batches all prompts into ONE comfy submission (one model load, one queue slot, same reference reused) and outputs N PNGs with the conventional _00001_.._NNNNN_ suffixes. This is dramatically faster and the sandbox preflight blocks for-loops / && chained && commands anyway.

# 10 variants of the SAME subject from one reference image — ONE submission:
python3 /home/sandbox/.openclaw/skills/comfyui/scripts/comfy_graph.py multiprompt \
    --image /workspace/outputs/char.png \
    --prompts "$(printf 'variant 1 description\nvariant 2 description\n…\nvariant 10 description')" \
    --append ". Consistent style: <global style brief>. Preserve same character." \
    --width 896 --height 1664 \
    --prefix char_variants \
    --output-dir /workspace/outputs/variants/
# → char_variants_00001_.png, char_variants_00002_.png, … char_variants_00010_.png

# 8 composite scenes of the SAME two characters — ONE submission:
python3 /home/sandbox/.openclaw/skills/comfyui/scripts/comfy_graph.py i2i2multi \
    --image1 /workspace/outputs/snakebird.png \
    --image2 /workspace/outputs/owl.png \
    --prompts "$(printf 'scene A\nscene B\nscene C\n…')" \
    --append ". Preserve exact appearance of both characters. <style brief>"

Prompting each line (edit-style): every prompt line should be an edit instruction — "Take the character from image 1 … place them in … preserve same features …" — not a standalone scene description. Flux-2 Klein is an image-editing model when given refs; verbose edit instructions outperform terse scene descriptions.

comfy_query.py — Server diagnostics

The first thing to reach for when a render fails, the queue is wedged, or you need to know what's installed on the server. All commands honour COMFY_URL.

stats                          # GPU/RAM usage, comfy version, python/torch versions
queue                          # currently-running prompt + pending list
loras                          # available LoRAs
models [<type>]                # files in a category (loras|diffusion_models|checkpoints|vae|text_encoders|upscale_models|…)
node <ClassName>               # the node's input schema + dropdown enum values
history [<prompt_id>]          # status, executed nodes, errors for a specific run
history --limit N              # the last N runs

Canonical workflow when a render fails:

# 1. Pull the failed prompt_id from your render output. It's printed at the start:
#    prompt_id: 7690667a-f37f-497c-b57e-827b1c1d6f49

# 2. Inspect the failure
COMFY_URL=https://media-relay.tail74c072.ts.net:8189 \
  python3 comfy_query.py history 7690667a-f37f-497c-b57e-827b1c1d6f49

# Output shows status (running / error / success), executed nodes, and on error
# the failing node + the upstream node_type + the exception_message.

# 3. If `status: error`, the exception_message is the actual problem.
#    Common patterns:
#      "Conditioning frames exceed the length of the latent sequence"
#         → cond+pass-1-output-length math is off. Common when re-applying
#           IC-LoRA on a latent that wasn't cropped first; or when the
#           cond video is shorter than the requested duration.
#      "Adding guide to a combined AV latent is not supported"
#         → an LTXVAddGuide / LTXAddVideoICLoRAGuide ran AFTER ConcatAV.
#           Always Add{Video}Guide BEFORE ConcatAV in the pass-1 chain.
#      "HTTP 413"
#         → ComfyUI server proxy body-size limit; re-encode the audio
#           (only when running STT, not when ia2v needs full quality).

# 4. If `status: running` for too long, check queue / server stats:
COMFY_URL=… python3 comfy_query.py queue
COMFY_URL=… python3 comfy_query.py stats     # VRAM exhaustion shows up here

# 5. If a node-type validation error suggests an enum/dropdown mismatch,
#    look up the node's allowed values:
COMFY_URL=… python3 comfy_query.py node LTXICLoRALoaderModelOnly
# returns the input schema + the list of available .safetensors filenames

Recovering an output after a client-side timeout:

comfy_graph.py polls /history/<prompt_id> every 2s with a 15s per-request timeout. If the bridge hiccups or the server restarts mid-render, the client raises TimeoutError even though the file may have been saved. Don't immediately re-render — use comfy_query to check first:

# Did the prompt finish? (history is keyed by prompt_id printed at submit time)
COMFY_URL=… python3 comfy_query.py history <prompt_id>

# Anything else running / queued?
COMFY_URL=… python3 comfy_query.py queue

# Pull the file directly if the server still has it (filename in history outputs):
curl -O "https://media-relay.tail74c072.ts.net:8189/view?filename=<name>&type=output"

If history is empty AND queue is empty AND the file is not at /view, the server was restarted and the render genuinely needs to be re-run.

dump — print the workflow JSON without submitting

comfy_graph.py dump <op> <args> builds the exact workflow that <op> would submit and prints it to stdout. Use it to:

• See what the script will send to ComfyUI before running it (sanity-check params).
• Pipe into comfy_run.py for fine-grained control (e.g. custom output dir).
• Diff against the official Lightricks workflows at https://github.com/Lightricks/ComfyUI-LTXVideo/tree/master/example_workflows/2.3 when investigating an output that looks wrong — that's how the 2026-05-16 "glitchy squares" bug was found (our _upsample_between was wiring LTXVLatentUpsampler from the pre-crop latent; the official lipdub-2-stage wires from cropped[2]).

# Sanity check what would be submitted
COMFY_URL=… python3 comfy_graph.py dump ia2v \
  --image anchor.png --audio slice.mp3 --prompt "..." --seconds 5 \
  --ic_loras "..." --ic_lora_reference_video cond.mp4 \
  > /tmp/wf.json

# Count IC-LoRA nodes, samplers, etc.
python3 -c "import json; d=json.load(open('/tmp/wf.json'));
from collections import Counter;
print(Counter(n['class_type'] for n in d.values()))"

comfy_run.py — Workflow runner

Submit a workflow JSON and wait for results. Accepts stdin (-) or file path.

# Pipe from dump command
python3 /home/sandbox/.openclaw/skills/comfyui/scripts/comfy_graph.py dump t2i --prompt "a cat" | \
  python3 /home/sandbox/.openclaw/skills/comfyui/scripts/comfy_run.py - --output-dir /tmp/imgs

# Or from a saved JSON
python3 /home/sandbox/.openclaw/skills/comfyui/scripts/comfy_run.py workflow.json --output-dir /tmp/imgs --timeout 300

video_join.py — ffmpeg helper (concat / trim / extract frames)

Standalone script with a #!/usr/bin/env -S uv run --script shebang and PEP 723 inline metadata; uv auto-installs imageio-ffmpeg (static ffmpeg binary) on first run. Works identically on host and inside the sandbox — no apt install needed.

# Concatenate clips (stream-copy when codecs/sizes match → fast; otherwise re-encode)
/home/sandbox/.openclaw/skills/comfyui/scripts/video_join.py concat \
  --inputs scene_a.mp4,scene_b.mp4,scene_c.mp4 \
  --output full_scene.mp4
# or from a list file: --inputs @clips.txt  (one path per line)

# Trim a clip
/home/sandbox/.openclaw/skills/comfyui/scripts/video_join.py trim \
  --input v.mp4 --start 0.5 --duration 2.0 --output v_trim.mp4

# Extract first or last frame (for FLF2V chaining)
/home/sandbox/.openclaw/skills/comfyui/scripts/video_join.py last-frame \
  --input clip1.mp4 --output last.png

make_reference_sheet.py — tile panels into an ingredients reference sheet

Standalone uv run --script (auto-installs pillow). Lays already-rendered element panels (one per character/prop/location) onto one black sheet, no text for the ingredients command. Contain-fits each panel (no crop/distort) in an auto-grid sized for the canvas aspect. Emit the sheet at the project's render WxH — the ingredients graph stretches the sheet to the output size, so a mismatched aspect distorts the reference.

# Landscape (trained bucket)
make_reference_sheet.py --out sheet.png --width 768 --height 448 \
  face.png turnaround.png trident.png soda.png beach.png
# Portrait music-video (off trained bucket — evaluate): --width 896 --height 1536
# Force a layout: --grid 2x3  (or --cols / --rows); --pad / --margin / --bg r,g,b

Importable: from make_reference_sheet import build_sheet. To generate the panels and tile them from a single character ref, use the storyboard skill's generate_reference_sheet.py (it composes flux2 panels, then calls this).

FLF2V (first-last-frame-to-video) / scene continuation pattern

1. Generate clip A with i2v from an anchor image.
2. Extract the last frame of clip A via video_join.py last-frame.
3. Generate clip B with i2v from that last frame (same prompt or next-scene prompt).
4. Concatenate A + B via video_join.py concat → continuous scene.

Works the same for longer chains (A → B → C → …). For audio-reactive continuations, use ia2v for each segment and slice the audio to match.

test_all_workflows.py — Regression test suite

Run all workflow builders end-to-end. Results logged to test_results.json.

python3 /home/sandbox/.openclaw/skills/comfyui/scripts/test_all_workflows.py

Covers: Flux2 t2i, t2i+LoRA, i2i, angles, TTS, LTX-2.3 t2v, LTX-2.3 i2v.

CLI Reference

Global options

• --notify-target <target> — OpenClaw notification target (e.g. discord:1486985676066000957)
• --output-dir <path> — local output directory (default: outputs/)
• --timeout <seconds> — generation timeout
• --seed <int> — random seed for reproducibility
• --fast (any video command: t2v/i2v/ia2v/flf2v/continuation/multiguide/transition) — skip the refine pass. About half the wall time, half the output resolution (no 2× upsample), rougher detail. Use for prompt iteration; leave off for final output.

Environment variables

• COMFY_URL_FLUX / COMFY_URL_VIDEO — separate endpoints for the flux (image/TTS/audio) and video servers when you run them on different hosts. In an OpenClaw sandbox these are pre-set via per-sandbox credentials.env, so agents can just run comfy_graph.py t2i … and the right server is picked automatically.
• COMFY_URL — single-server fallback (used as default when the per-class env var is unset, and by comfy_run.py / comfy_query.py which talk to one server at a time). Default http://localhost:8188. Set http://localhost:8000 for ComfyUI Desktop.
• OPENCLAW_MEDIA_DIR — override the sandbox output directory (final asset writes + outbound copy). Full path, used as-is. Falls back to /workspace/media/outbound in a sandbox, or ~/.openclaw/workspace/media/outbound on host. Required on commons-tier sandboxes where /workspace is read-only.
• OPENCLAW_NOTIFY_TARGET — default notification target

Discord attachments — MEDIA: directive in your reply

Every successful comfy_graph call automatically copies the outputs to /workspace/media/outbound/<filename> (sandboxed agents) or ~/.openclaw/workspace/media/outbound/<filename> (host install). In an OpenClaw sandbox those are the same file via the workspace bind mount.

To attach in a Discord reply, add a line starting with MEDIA: (only leading whitespace allowed before it) pointing at the host-side path — the reply parser runs host-side and resolves literal paths only:

Here's your image!
MEDIA:$HOME/.openclaw/workspace/media/outbound/flux2_t2i_00012_.png

Inline forms ("... image: MEDIA:/path ...") are NOT parsed. One MEDIA: line per file. Do NOT cp or mv files into random workspace subdirs — core.py already places them in the right spot.

Architecture

Flux2 (images)

• UNET: flux-2-klein-9b-fp8.safetensors (9B full quality) or flux-2-klein-4b-fp8.safetensors
• CLIP: qwen_3_8b_fp8mixed.safetensors with type="flux2" ⚠️ NOT gemma!
  • gemma causes ValueError: Input img and txt tensors must have 3 dimensions
• VAE: flux2-vae.safetensors
• Workflow: RandomNoise → CFGGuider → SamplerCustomAdvanced → Flux2Scheduler → VAEDecode → SaveImage
  • ⚠️ No LatentAddNoise — that node is NOT installed on this server

LTX-2.3 (video) — two-pass refine pattern

• Checkpoint: ltx-2.3-22b-dev-fp8.safetensors
• Text Encoder: gemma_3_12B_it_fp8_e4m3fn.safetensors via LTXAVTextEncoderLoader
• Audio VAE: via LTXVAudioVAELoader (uses same ckpt_name)
• Upscaler (between passes): ltx-2.3-spatial-upscaler-x2-1.1.safetensors via LatentUpscaleModelLoader
• LoRA: ltx-2.3-22b-distilled-lora-384.safetensors at strength 0.6 — applied on ALL flows (t2v/i2v/ia2v/flf2v). This reproduces the distilled checkpoint behaviour on top of the dev-fp8 checkpoint we have installed; without it the 8-step schedule under-denoises and output is badly blurred.
• Pass 1 (coarse, 9 steps): euler_ancestral_cfg_pp + ManualSigmas "1.0, 0.99375, 0.9875, 0.98125, 0.975, 0.909375, 0.725, 0.421875, 0.0", raw LTXVConditioning
• Between passes: Separate AV → LTXVCropGuides on the post-pass-1 video latent → LTXVLatentUpsampler on the cropped latent (NOT on sep[0] — see the gotcha below) → re-apply image via LTXVImgToVideoInplace(strength=refine_guide_strength, default 0.3) (i2v/ia2v only) → re-concat with audio. The cropped conditioning (LTXVCropGuides output) is what feeds pass-2's CFGGuider.
  • ⚠️ Gotcha — upsample from CROPPED latent, not separated. When pass-1 added conditioning frames via LTXVAddGuide / LTXAddVideoICLoRAGuide (IC-LoRA / id_branch / multiguide / flf2v), the post-pass-1 latent's temporal dim is requested_length + cond_frames (e.g. 32 frames for a 16-frame request with 16 cond frames). LTXVCropGuides strips the cond frames from BOTH the cond's pos/neg metadata AND the latent (it slices latent_image[:, :, :-num_keyframes]). If LTXVLatentUpsampler is wired from the pre-crop sep[0] instead of cropped[2], pass-2 refines a 2x-temporal latent → 2x-duration glitched output. The basic ia2v 2-stage workflow (no AddGuide) gets away with sep[0] because it has no cond frames to strip; the official lipdub-2-stage IC-LoRA workflow correctly wires from cropped[2]. Fixed in ltx2._upsample_between 2026-05-16 after the stable-altitude v8 "glitchy squares" arc.
• Pass 2 (refine, 4 steps): euler_cfg_pp + ManualSigmas "0.85, 0.7250, 0.4219, 0.0", cropped conditioning
• Always AV-concat: All three variants (t2v/i2v/ia2v) use LTXVConcatAVLatent/LTXVSeparateAVLatent. For non-audio variants, LTXVEmptyLatentAudio provides a blank audio side. This keeps one graph shape.
• Length: (seconds × fps) rounded to ≡ 1 (mod 8)
• Audio for ia2v: LoadAudio → TrimAudioDuration → LTXVAudioVAEEncode → SetLatentNoiseMask (with zero-valued SolidMask)
• flf2v (first-last frame): adapted from the Comfy-Org/workflow_templates flf2v template into the same two-pass shape as the other flows. Pass-1 uses chained LTXVAddGuide (frame_idx=0 + frame_idx=-1 at strength 0.7) on the base latent; pass-2 re-injects both guides at strength 1.0 on the upsampled latent. Post-decode LTXVCropGuides strips the injected guide frames so the output doesn't show the raw input images as first/last frames (use --fast to keep the single-pass shape the template originally had). Default fps=25 here (every other video command defaults to 24); pass --fps 24 for concat-friendly output. Transition LoRA is off by default (opt-in with --use_transition_lora) — the inverse convention from multiguide, where it is on by default and disabled via --no_transition_lora 1.
• multiguide (N-anchor): chained LTXVAddGuide calls, one per guide, each at its specified latent frame_idx (snapped to 8 upstream). Used by music-video's guides: scene field to anchor identity mid-shot when the first frame doesn't show the character, and for scenes ≥13s where single-anchor ia2v drifts by the tail. Pass --no_transition_lora 1 when the output is a scene (not a transition) — the transition LoRA pulls hard toward endpoint convergence, which is wrong for a normal ia2v-style shot.
• transition (song-aligned morph between two clips): pass-1 injects real video frames from prev_video (A-guide, latent positions 0..N-1, strength 1.0) and next_video (B-guide, single frame at position 96 by default — see b_sparse_latent_positions). The masked middle is driven by the audio slice via LTXVAudioVideoMask. use_addguide=1 is the mode used by music-video — LTXVAddGuide natively handles multi-frame IMAGE batches; the older LTXVImgToVideoInplaceKJ with num_images=2 only picks the first frame of each batch and produces a frozen middle. Final decode MUST pass strip_guides_cond to remove the injected guide tokens from the latent tail, or the clip will run guide_frames longer than requested. Keep B-side guides sparse (every 8 latent frames); contiguous multi-frame B-blocks cause a snap-in freeze where the scene content hard-locks. "96" (1f) is the default and works for most boundaries; "72,80,88,96" (4f) gives stronger character establishment going into singing / lipsync scenes.
• ingredients (reference-sheet IC-LoRA): conditions the clip on a reference sheet — one composite image with a clean panel per character (face + turnaround), prop, and location (black bg, no text). ltx2_ingredients_to_video(sheet_image, prompt) loops the sheet in-graph into a static reference video at the output res (LoadImage → ImageScale(WxH) → RepeatImageBatch(length) → LTXAddVideoICLoRAGuide, downscale 1) and applies ltx-2.3-22b-ic-lora-ingredients-0.9 @ 1.4. Output is a single-width clip whose characters/props/location match the sheet. Trained bucket 768×448 / 121f / 24fps — stick to it for best results. The sheet must be in the server input/ dir (the CLI --sheet auto-uploads a local path). Prompt = a rich scene description naming the sheet's elements (training format was Reference sheet: …\n\nGenerated video: …; a single cinematic description also works). Bigger panels carry over better; give important elements prominent, clean panels. Author the sheet with make_reference_sheet.py (tile your own panels) or, from a single character ref, the storyboard skill's generate_reference_sheet.py (generates panels via flux2, then tiles). Make the sheet match the render WxH/aspect.

Available LoRAs (19)

• Camera: ltx-2-19b-lora-camera-control-dolly-in/out/left/right/jib-up/jib-down/static
• Quality: ltx-2-19b-lora-detailer, ltx-2-19b-distilled-lora-384
• Style: pixel_art_style_z_image_turbo.safetensors
• Relight: WanAnimate_relight_lora_fp16.safetensors
• I2V: wan2.2_i2v_lightx2v_4steps_lora_v1_{high,low}_noise, lightx2v_I2V_14B_480p...
• Lightning: Qwen-Image-Lightning-{4,8}steps*, Qwen-Image-Edit-2511-Lightning*

Delivery to Discord (you send it, not the script)

comfy_graph.py blocks until the workflow completes. There is no async / webhook / notification path — the sandbox does not have the openclaw CLI, so the script cannot deliver on your behalf. The final stdout always includes a saved: /workspace/outputs/<file>.ext line with the sandbox path.

Your job after the exec returns:

1. Parse the saved: … line from the output.
2. Translate sandbox → host path: /workspace/… → $HOME/.openclaw/workspace-<agent>/… on the host (the message tool resolves paths on the host, so the host's literal absolute path is what media: wants).
3. Send with the message tool's media arg.

python3 /home/sandbox/.openclaw/skills/comfyui/scripts/comfy_graph.py t2i \
  --prompt "…" \
  --output-dir /workspace/outputs/
# stdout: … saved: /workspace/outputs/flux2_t2i_XXXXX_.png …

Then (pseudo):

message({
  action: "send",
  channel: "discord",
  to: "<channel_id>",
  media: "$HOME/.openclaw/workspace-<agent>/outputs/flux2_t2i_XXXXX_.png",
  message: "Here's your image."
})

--notify-target (advisory only)

--notify-target discord:<channel_id> does not send anything. It only makes the script print an extra line [NOTIFY_URL 0] target=discord:<channel> media=<comfy_view_url> caption=<...> so you have a ready-made URL + caption to feed to message. It's convenience, not automation. You still call message yourself.

When to offload to a subagent

Blocking exec is fine for images (< 30s) and short TTS. For video (LTX-2.3 is 2-5 min), spawn a subagent so the main Discord reply doesn't hang:

sessions_spawn({
  task: "Generate t2v of '<prompt>' (seconds=5) via /home/sandbox/.openclaw/skills/comfyui/scripts/comfy_graph.py, then send the resulting video to discord channel <id> with caption '<caption>'. Use exec + message tools. Host path for message is $HOME/.openclaw/workspace-<agent>/outputs/<file>.",
  runtime: "subagent",
  sandbox: "inherit",
  mode: "run"
})

The subagent runs inside the same shared sandbox, does the long gen + delivery, and auto-announces completion to the main session when done. You (the main agent) are free to continue talking.

Debugging an output that looks wrong

Server-side validation (empty outputs, AssertionErrors, missing nodes) is covered by the comfy_query.py history <id> workflow in the comfy_query section above. This section is for the harder case: the workflow ran to completion, the file was saved, but the video looks wrong (glitchy noise, doubled duration, wrong identity, conditioning ignored, etc.).

Step 1 — extract a middle frame and look at it

Before re-rendering or reading code, check what was actually produced. The imageio_ffmpeg Python wheel ships a static ffmpeg binary so this works on any host with uv available:

uv run --with imageio-ffmpeg python3 -c "
import subprocess, imageio_ffmpeg as iio, re
ff = iio.get_ffmpeg_exe()
r = subprocess.run([ff, '-i', 'out.mp4'], capture_output=True, text=True)
m = re.search(r'Duration: (\S+)', r.stderr); d = re.search(r'(\d{2,4})x(\d{2,4})', r.stderr)
print(f'duration={m.group(0)}, dims={d.group(0)}')
subprocess.run([ff, '-ss', '2.5', '-i', 'out.mp4', '-frames:v', '1', '-y', 'mid.png'], capture_output=True)
"

What the frame tells you:

Step 2 — compare your workflow against the official Lightricks examples

The single most useful debugging tool: diff your generated workflow JSON against the official example workflow for the same flow shape. Official examples are at https://github.com/Lightricks/ComfyUI-LTXVideo/tree/master/example_workflows/2.3:

Workflow:

# 1. Dump what your script produces (API format)
python3 comfy_graph.py dump <op> <args> > /tmp/ours.json

# 2. Fetch the official UI-format example
gh api repos/Lightricks/ComfyUI-LTXVideo/contents/example_workflows/2.3/LTX-2.3_ICLoRA_Lipdub_Two_Stage_Distilled.json \
  --jq '.content' | base64 -d > /tmp/official.json

# 3. Diff the topology. The two formats are different (API vs UI), so trace
#    node-by-node which output feeds which input. Focus on the SAMPLER's
#    inputs (latent_image, guider) and the upstream chain feeding them.

The 2026-05-16 "glitchy squares" fix was found exactly this way: our _upsample_between wired LTXVLatentUpsampler.samples from the post-pass-1 separated latent (sep[0]); the official lipdub-2-stage workflow wires it from LTXVCropGuides.latent output (cropped[2]). That single mis-wire caused the pass-2 sampler to refine 2× as many latent frames as the user requested, producing the doubled-duration glitched output.

Step 3 — re-run with the smallest possible reproduction

Don't re-render the full project. Pull one short scene out of the project YAML, render it standalone via comfy_graph.py ia2v directly with all the same flags, and iterate on that. A 4-second scene takes 1-3 minutes vs an hour for the full music video.

Step 4 — bisect against working configurations

If a parameter change broke the output, render with the LAST KNOWN GOOD params alongside the broken ones (same prompt, same image, same audio). When both sit side-by-side in the output directory, the structural difference is usually obvious.

The matrix that works on the LTX-2.3 server today (as of 2026-05-16):

Troubleshooting

HTTP 400 "Node 'LatentAddNoise' not found"

→ flux2.py was updated to remove LatentAddNoise from the graph. If you see this error, the flux2.py workflow builder is stale — re-import it.

HTTP 400 "unet_name: 'None' not in list"

→ This was a bug where opts.get("unet") passed None as an explicit argument, overriding the function default. Fixed by filtering None values with **extra pattern. If you see this, the comfy_graph.py is stale — re-import it. Symptom: any workflow command fails with 400 even though the graph looks correct.

"Input img and txt tensors must have 3 dimensions" error

→ Wrong CLIP model! Use qwen_3_8b_fp8mixed.safetensors (NOT gemma) for Flux2. For LTX-2.3 video, use gemma_3_12B_it_fp8_e4m3fn.safetensors via LTXAVTextEncoderLoader.

"No output" after submit

→ Poll /history/{prompt_id} manually — check node_errors in the response. Use: python3 /home/sandbox/.openclaw/skills/comfyui/scripts/comfy_query.py history <prompt_id>

Slow generation

→ Use distilled models for 4-step generation. Full quality needs 20+ steps.

Memory errors (OOM)

→ Reduce resolution or use smaller models. RTX 3090 has 24GB VRAM.

LTX-2.3 "mat1 and mat2 shapes cannot be multiplied"

→ CLIP model mismatch. Make sure you're using gemma_3_12B_it_fp8_e4m3fn.safetensors with the LTX-2.3 checkpoint, not UMT5 or other CLIP models.

LTX-2.3 "Value not in list" for text_encoder

→ Use a text encoder shown by the server's LTXAVTextEncoderLoader choices. This skill defaults to gemma_3_12B_it_fp8_e4m3fn.safetensors.

For API details → references/api.md For prompt tips → references/prompt-patterns.md For LTX-2.3 prompt writing → references/ltx-prompt-guide.md (camera vocabulary, what to avoid, worked rewrites)