authoring-video-templates
Skill35 starsChanged 15 days agoMITView on GitHub
Use when creating or editing a video template JSON (the template descriptor), adding or changing sections/filters/maps/variables/transitions/looks/motion/audio/layers/registered effects/creative direction, or debugging template validation errors in ffmpeg-video-composer.
In this repository8 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- core-architecture-patternsSkill
- cross-platform-ffmpegSkill
- monorepo-dev-workflowSkill
- ondevice-ffmpeg-engineSkill
- pr-creationSkill
- pr-evidenceSkill
The file
We split this skill into the rules it gives an agent
4 rules. About 0 are common, 1 is distinctive, and 3 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 72
Keep an optional meta.creativeDirection brief (trimmed, 1–4000 characters) when editing.
- Very specific: line 90
Use section kinetic blocks for animated copy.
- Very specific: line 97
Easing: prefer springs ($snappy, $bouncy, $gentle, or spring(k, c)) and omit duration so physics decides.
- Distinctive: line 112
Tune the parameters: never ship all defaults.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Authoring Video Templates
Overview
A template is a JSON descriptor that the engine compiles into a video. Its main keys are global (project-wide defaults) and sections (the ordered list of scenes), with optional meta (display metadata and creativeDirection) and partials (reusable fragments). Validate descriptors against the schema; don't guess fields.
Sources of truth (in order of authority):
- zod source:
packages/ffmpeg-video-composer/src/schemas/(TemplateDescriptorSchemainsection.schemas.ts, re-exported bytemplate.schemas.ts). This defines accepted fields; use the validator and lowering code to check behavior. - Validator and behavior:
packages/ffmpeg-video-composer/src/services/TemplateValidator.ts(+template-validation-rules.ts), native lowering insrc/editor/presets/, and registered contracts inpackages/leclap-mcp/src/effects/. - Machine-readable JSON Schema:
docs/template-descriptor.schema.json— generated from the zod source; feed it to an editor/agent for autocompletion + validation. Regenerate withpnpm --filter ffmpeg-video-composer generate:schema. - Full field reference + examples:
docs/template-configuration.md. - Built-in examples:
packages/leclap-creative-kit/src/templates/*.json.
Two layers: structured sugar vs. raw filters
- Structured sugar (prefer this):
transition,look,grade,motion,audio,layers, animationinputs, and text sugar (caption,titleCard,lowerThird,reveal,global.overlays/look/grade). Editor-friendly camelCase intents that compile to ordinary, on-device-safe FFmpeg filters. - Raw filters (escape hatch):
filters[],inputs[].filters,maps[]— passed to FFmpeg verbatim. A final{ "type": "scale", "value": "output" }conforms a custom frame to the actual project dimensions with aspect-preserving scale/pad and square pixels. Other arguments stay raw. Theirvalueskeys stay FFmpeg-native (x/y/w/h/c/t/fontcolor/fontsize/fontfile/alpha/d/st/color/box/boxcolor/boxborderw), not camelCase, by design.
Structure
{
"global": {
"variables": { "video": "https://…/earth.mp4" },
"orientation": "landscape", // "landscape" (1280x720) | "portrait" (720x1280) | "square" (1080x1080)
"musicEnabled": true,
"music": { "name": "track.mp3" },
"transition": { "type": "fade", "duration": 0.4 }, // xfade name | "cut"; duration in SECONDS
"audio": { "sourceVolume": 1, "musicVolume": 0.5 },
},
"sections": [
{
"name": "intro",
"type": "video", // see Section types
"options": { "videoUrl": "{{ video }}", "duration": 4, "musicVolume": 1 }, // SECONDS
"transition": { "type": "wipeleft", "duration": 0.4 }, // boundary AFTER this section
"look": "cinematic",
"filters": [{ "type": "fadein", "values": { "color": "#000000" } }],
},
{
"name": "outro",
"type": "color_background",
"options": { "backgroundColor": "#101418", "duration": 2 },
},
],
}Section types (discriminated on type)
type | Use for |
video | A clip from options.videoUrl (remote or local). |
project_video | A clip recorded from the device camera; supports framingGuide. |
image_background | Still image (options.pictureUrl); supports kenburns. |
color_background | Solid colour (options.backgroundColor) + composited layers. |
form | User input fields (options.fields); each field name → {{ name }}. |
music | Audio-only / timeline-padding section. |
partial | Reusable or inline fragment expanded before validation/compile. |
effect | Versioned registered effect resolved to a clip before FFmpeg composition. |
Native visual segments live in packages/ffmpeg-video-composer/src/editor/segments/; SegmentFactory maps their type → class. Partials expand first; effect sections require caller-provided resolution.
For effect, author an exact { id, version, props, assets } reference and a required positive options.duration. The core accepts JSON props; the backend validates each registered effect's actual props/assets and runtime. In MCP, discover with get_effect_schema ({ "list": true }), then request the exact id/version contract and use render_preview before compose_video. The trusted Node/Chromium worker is configured by the operator; browser/native callers consume resolved clips. See Registered JSON effects.
Keep an optional meta.creativeDirection brief (trimmed, 1–4000 characters) when editing. It guides authoring, adds no rendering behavior, and must be implemented through explicit native settings or registered props; see creative direction.
Capabilities (brief)
- Transitions —
transition: { type, duration? }onglobaland/or per section (boundary after that section).typeis an xfade name (seeXFADE_TRANSITIONS) or"cut". Effective duration = section ?? global ?? 0.3 s. Any non-cutboundary forces a full-timeline re-encode (costly on WASM/on-device); cuts are a fast stream-copy concat. - Looks & grade —
lookis a preset:eq/curves-based (cinematic/warm/cool/vintage/noir/vivid/dreamy), LUT-backed (teal-orange/warm-film/mono-film/noir-film/vivid-pop→ a singlelut3d+ a bundled.cube, a stronger grade; staged like fonts, runs on every backend, dropped-with-warning wherelut3dis absent), or stylized (duotone/posterize/sketch/glitch/soft-vignette→ small LGPL-safe filter stacks —hue/colorchannelmixer/lutyuvquantize/tint,edgedetect,rgbashift+noise,vignette— nevereq/geq/boxblur).grade(brightness/contrast/saturation/gamma/hue/colorBalance/blur/grain/curvesPreset) →eq/colorbalance/curves/gblur/noise/hue, stacks on top;grain(0..1) lowers tonoise=alls=<0..20>:allf=t+u. - Letterbox — section
letterbox: { aspect, color? }(aspect1..4, e.g.2.39for cinemascope) overlays twodrawboxbars top/bottom to simulate a wider frame; compiles after grade/look so bars sit over the graded image. No-op ([]) whenaspectis narrower than or equal to the frame's own aspect ratio. - Motion — ordered
motion[]:kenburns(image_background,video,project_video, or resolvedeffectclips),rotate,crop,flip,shake(handheld jitter via a wanderingcropwindow,intensity/frequency),pulse(rhythmic zoom viazoompan,intensity/frequency) →zoompan/rotate/crop/hflip/vflip.shake/pulsehave no section-type restriction. - Chroma key — section
chromaKey: { color, similarity?, blend?, background? }keys out a solid screen colour (colorkey) and composites the clip over a flat colour. v1 = solid-colour background only; dropped-with-warning wherecolorkeyis absent. - Audio polish —
global.audio:sourceVolume,musicVolume,normalize(loudnorm/dynaudnorm),ducking(bool or fine-grained). Per-sectionoptions.audioFade(in/out,afade) andoptions.musicVolume.options.audioEffect(echo/telephone/muffled) appends a voice preset (aecho/highpass+lowpass/lowpass) to the-afchain before any fade, skipped entirely whenmuteSectionis true. - Layers —
color_backgroundoptions.layers[]: solid/opacity/gradient boxes composited over the base colour. - Framing guide —
project_videooptions.framingGuide(silhouette): a recording-UI overlay only, never rendered into the video. - Animation & image inputs — (the bundled
animations/*.apngare samples, a last resort; see below) one input per overlay: an.apng/.webp/.gif/.webmanimation or a stillimage(PNG/JPG).options.loop→stream_loop,options.persistent→eof_action=repeat,options.rotation(deg),options.motion(an animated entrance reusing therevealvocabulary →overlayx/y time-expressions),options.animatekeyframe tracks (x/y/scale/scaleX/scaleY/rotate/opacity, pivotoptions.origin; scale/rotate needoptions.scale"W:H" or an html box) for pop-ins, squash, flips, needles and parallax. - Text sugar — prefer these over hand-positioned
drawtext:caption(styled overlay), sectiontitleCardoncolor_background(kicker/headline/subtitle/accent/fade — collapses ~80-line intros), sectionlowerThirdon any visual section (title/subtitle/badge band, composites above animations). On any of them:reveal(none/fade/rise/slide-left/slide-right, bare string or{type,delay,duration,distance,easing}) for an entrance, andeffect: { shadow?, outline? }(TextEffect) for drop-shadow/outline legibility. A title card takes optionalstagger(seconds 0..1, default .15) between non-empty lines; zero starts them together and its accent follows its associated line. A positioneddrawtextfilter also takesexit(same vocabulary + anafterstart time, defaulting to end-at-section-end). Entrance and exiteasingislinear(default),ease-out,ease-in-out, orease-out-back; it curves text alpha and travel. Back easing overshoots travel by about 10% while clamping text alpha to 0..1. Overlay fade motion uses the linear fade filter. Noexitfield exists on caption/titleCard/lowerThird blocks. Sized from the output scale, so they render in any orientation. - Global decorations — authored once in
global, applied to every section (sibling ofglobal.animations):global.overlays[](whole-video text/brand watermark, withpositionanchor + optionalsectionssubset),global.look/global.grade(whole-video colour). Removes per-section{{ brand }}repetition.
Motion system: kinetic typography (for "wow" templates)
Use section kinetic blocks for animated copy. Each block lays its copy out with real font metrics and animates each word or glyph natively, on every backend:
{ "text": { "en": "Make every word land." }, "preset": "cascade", "accent": { "words": "last" }, "exit": "cascade" }- Presets: cascade, rise, drop, slide, pop, impact, tracking-in, typewriter, scramble, wave, highlight, counter, split, fade. Every other field has a preset default: unit, order, stagger, delay, ease, distance, size, align, x, y (
top/center/bottom), maxWidth, accent, effect, exit. - Easing: prefer springs (
$snappy,$bouncy,$gentle, orspring(k, c)) and omitdurationso physics decides; use$expo/$smoothfor controlled moves. - Intensity:
global.motion.energyscales every travel (0 reduced motion, 1.5 hype).global.seedfixes random order and scramble glyphs. - Fonts: word/glyph units need a bundled font (
kinetic_font_unmeasurableotherwise). One dominant block per beat, one accent word per line, and vary presets between beats. - Camera, graphics, transitions: a section
camera(push-in, drift, orbit, handheld;hitson the beats),graphics(flash, bars, underline, frame, corners, wipe, panel) and designed transitions (push-*,swipe-*,zoom-throughwiththrougha letter counter,iriswithoriginandshape,whip-*,drip,bands, withease). Kineticclip("line","block", a box) and layeroptions.clipreveal out of an invisible slot. Use one camera idea and one graphic idea per beat. - Masks:
options.maskon an image / animation / html layer or a video section: ashape(rect + radius, ellipse, SVG pathd) withanimatetracks for irises and wipes, or alayertrack matte (alpha/luma: footage through a word). Prefer a mask to a background-coloured panel that hides part of a layer. - fx primitives:
graphics[]entries{ type: "fx", effect, target, ... }are parametric light primitives (e.g.sheen) clipped to a target. Tune the profile, width, tilt, direction, colour ("$color.accent"), intensity, duration/ease and repeat for the template. - Discovery:
get_motion_catalog(MCP) ormotionCatalog()lists presets with defaults, art-direction rules and a starter.
Compose motion, don't pick stock animations
Generated templates should look authored, not assembled. Follow this workflow:
- Creative direction: audience, brand and energy (
meta.creativeDirection,global.motion.energy). - Motion intent per section: one line about what moves, why, and how it should feel.
- Engine primitives: kinetic,
animatetracks with ease tokens or springs, camera, designed transitions, graphics and fx, roles, and beats and cues for timing. - Tune the parameters: never ship all defaults.
- At most 1–2 signature moves per video.
The creative-kit library animations (packages/leclap-creative-kit/src/library/animations/*.apng: shine sweep, confetti, light leak…) are samples. Use one only as a last resort. motionCatalog().samples maps each one to the primitives that replace it. getMotionWarnings reports fx_untuned, effect_repeated, library_animation_sample, effect_off_theme and decor_overload with hints (services/sameness-lint.ts). See examples/motion-design/kinetic-type.json.
Variables, filters, maps
- Variables — define in
global.variables(string or string[]); reference anywhere with{{ name }}.{{ colorN }}is 1-indexed intocolorsList;{{ form_field }}is a form field's value. - Filters —
{ type, value?, values?, range? }.typeis a raw FFmpeg filter name.valueskeys are FFmpeg-native (see above). Common types:fadein,fadeout,scale,drawtext,drawbox,overlay,vignette. - Maps —
{ inputs, outputs, filters?, options? }to route FFmpeg streams explicitly;@namereferences an input/animation pad; the final output must end withfinal. Only needed for multi-input/overlay sections. text,title,description, fieldlabelareTranslationobjects ({ "en": "…" }) for i18n.
Validating
Validate with TemplateValidator (zod + cross-field rules). zod reports the exact path/field on failure. Cross-field rules reject: a non-cut transition on the last rendering section (dangling_transition); an effective transition duration ≥ the smaller adjacent declared duration (transition_too_long); kenburns outside image_background/video/project_video/effect (motion_unsupported_section); a whole-video animation or watermark with no url (global_animation_missing_url / global_watermark_missing_url); and a caption/global.overlays font string that is neither a bundled id nor a .ttf (unknown_font — catches typos that would otherwise silently fall back to the default).
- Fonts —
fonttakes a bundled id ("bebas"), a.ttffilename, or{ family, weight?, style? }to pull any Google Fonts family ({ "family": "Playfair Display", "weight": 700 }; weight 100..900 in steps of 100, default 400; stylenormal/italic). The object form is not validated against Google's catalogue — a bad family fails at render time naming the family — but it is the only way to get a multi-word family or a specific weight. An unresolvable font fails the render rather than silently drawing the wrong face. Not supported on the browser/WASM backend. After editing any.json, runpnpm fmt.
Common mistakes
- Inventing option keys —
globaland sectionoptionsarestrict; only schema keys are valid. - Using old field names:
audioVolumeLevel→audio.sourceVolume;transitionDuration→transition.duration;musicVolumeLevel→musicVolume;type: "frame"/frames/frequency/overlay→ a singletype: "animation"input. - Forgetting durations are seconds everywhere (
project_videoduration was previously ms —30000becomes30). - Volumes (
sourceVolume/musicVolume) must be 0..1;duration/speed/transition.durationmust be positive. - Using a
{{ var }}that isn't declared inglobal.variables(or isn't acolorN/form field). - Wrong
typestring — it must be one of the literals above (discriminated union).
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

