agentleFS
indranilbanerjeeindranilbanerjee/socialforge/skills/generate-video/SKILL.md

generate-video

Skill43 starsChanged 4 months agoMITView on GitHub

Create a short video from a concept through staged first frame, last frame, motion and render. \"turn this concept into a video\"

7 other projects use a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

8 rules. 0 are common, 4 are distinctive, and 4 are very specific to this project.

  • 0

    Common

    Most of the 5 edited versions we read keep these.

  • 4

    Distinctive

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Payoff per scene: Every scene's payoff field states what the viewer has gained by the time it ends.

    line 70
  2. Very specific:

    Compliance before credits: Run the filled script's narration and overlay text through compliance_check.py BEFORE Stage 2 — a banned phrase …

    line 76
  3. Distinctive:

    After the user picks the final video, post-processing runs before saving:

    line 109
  4. Distinctive:

    Background music: If the video has no audio (sound=False in the clip request), user is asked whether to add background music (optional).

    line 112
  5. Very specific:

    Read from scripts/generate_video.py.

    line 123
  6. Distinctive:

    Every stage requires explicit user approval before advancing

    line 157
  7. Very specific:

    All assets save to {post_folder}/ -- keyframes in keyframes/, video versions in versions/, final video in final/

    line 162
  8. Distinctive:

    If video generation fails or times out, deliver script + storyboard + keyframes as fallback

    line 168

“Common” is measured: kept by at least 6 in 10 of the 5 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

/socialforge:generate-video — Video Production Kit

Script location. If your host does not set ${CLAUDE_PLUGIN_ROOT}, the scripts are in this plugin's scripts/ folder, next to skills/. Run them with Python 3.10 or newer: python, or python3 where python is not found (macOS, many Linux systems).

Generate video production assets through a 5-stage human-in-the-loop pipeline. Each stage requires user approval before advancing.

Context efficiency

Asset-heavy skill. Grep before Read the asset catalog (${CLAUDE_PLUGIN_DATA}/socialforge/brands/<brand>/asset-index.json) — never list the asset directory. Reference generated images / videos by path, not by loading metadata. Brand profile loads once per session.

Prerequisites

  • Credentials must be configured via /socialforge:setup:
    • Vertex AI (registry alias latest-image-google) — used for first-frame and last-frame keyframe generation
    • WaveSpeed API (registry alias latest-video-wavespeed) — used for image-to-video generation
  • Models are never named in this skill: each alias above is resolved to a current model id when the script runs. To see what they resolve to today, run python "${CLAUDE_PLUGIN_ROOT}/scripts/generate_video.py" --list-models (video) or python "${CLAUDE_PLUGIN_ROOT}/scripts/resolve_model.py" --aliases (every alias).
  • Brand profile must be active (/socialforge:switch-brand if needed)
  • Calendar must be parsed (/socialforge:parse-calendar) with video posts identified

Quote, then go (before any paid call)

Generation spends provider credits, so the first paid call waits for a quote and an explicit go. This skill states no price and the model registry stores none: price_book.py is the only source, and it refuses rather than guesses.

  1. List what is about to be generated: for each paid call, the registry alias (resolve_model.py --alias <alias> gives the current model id), the provider, and the units (images, or seconds of video, plus any option that changes the price, such as synchronised audio). The first paid call is Stage 2, so quote the whole chain (the keyframes of Stages 2 and 3 and the clip of Stage 4) once, before Stage 2. Quote only providers that can run here: run generate_video.py without --generate-video first and read its routing block. provider is the one to quote; unavailable lists the providers that cannot run on this setup and why (Veo needs Vertex AI, so a Gemini API key alone cannot make Veo clips); fallbacks_if_quoted lists the others that could run. A fallback you want available is quoted as its own line, and the total you show is the worst case.
  2. Quote it:
python "${CLAUDE_PLUGIN_ROOT}/scripts/price_book.py" --action quote-batch --items '[{"model":"<resolved model id>","provider":"<provider>","units":<n>,"label":"<post id and stage>"}]'

A single item can use --action quote --model <id> --provider <provider> --units <n>. If the quote exits non-zero or lists a blocked item, stop: run /socialforge:price-check to look the missing price up, then quote again. 3. Show the user the total, each line, its source URL, and how old the price is (a price older than 24 hours is stale and is refused; it must be looked up again):

Estimated cost: <total_usd> USD for <n> generations. Prices from <source>, <age_hours> h old (valid for 24 h).
Type "go" to generate. Anything else cancels.
  1. Continue only on an explicit go for exactly this list. Anything else, including silence, cancels and nothing is generated. If the list changes (more posts, another model, an added option), quote again and ask again. A quote is never approval (approved_to_run is always false).
  2. Pass exactly the providers that were quoted: --quoted-providers kling (or kling,veo). Only those can run. If the first provider fails and the next one was not quoted, the run ends with video.status: NEEDS_REQUOTE and nothing was spent on it: show the user the new line and total, wait for a new go, and never widen the list yourself.

The 5-Stage Pipeline

Stage 1: Video Concept + Script (no API call)

Claude generates 2-3 video concept ideas based on the post brief, brand voice, and platform requirements. Each concept includes:

  • Working title and hook
  • Visual narrative arc (opening, middle, close)
  • Suggested duration and pacing
  • Tone and style direction

The user picks one concept (or requests refinements). Then fill the script scaffold (generate_script in generate_video.py) from the chosen concept and the post's actual brief, in the brand's voice — every [FILL] replaced, no placeholder survives into Stage 2. The scaffold enforces the structure; this pass supplies the craft, under four rules the scaffold carries with it:

  1. Hook first, never the logo. The open earns attention with the single most arresting thing the brief supports; the brand mark lives as the corner watermark and in the end card. Three seconds of logo reveal is the classic retention killer this pipeline used to scaffold by default.
  2. Payoff per scene. Every scene's payoff field states what the viewer has gained by the time it ends. A beat that only sets up the next beat is where viewers leave — give it a payoff or fold it.
  3. The pairing rule. The hook's text overlay and the post's caption (written by adapt-copy) do different jobs and never echo. Check against the adapted copy if it already exists for this post.
  4. Compliance before credits. Run the filled script's narration and overlay text through compliance_check.py BEFORE Stage 2 — a banned phrase caught in a script costs nothing; caught in a rendered video it costs the whole generation chain. Claims in narration follow the same rules as claims in copy: sourced or absent.

The user approves the filled script before any generation spend.

Stage 2: First Frame Generation (Vertex AI, latest-image-google)

First: Quote, then go (the section above). Nothing is generated until the user types go.

Generate 2 first-frame options based on the chosen concept. These set the opening visual and establish the look and feel.

  • Images are shown inline in the terminal for immediate review
  • User selects one or requests a regeneration with adjusted direction

Stage 3: Last Frame Generation (Vertex AI, latest-image-google)

Generate 2 last-frame options that complete the visual narrative arc, matching the approved first frame.

  • Images are shown inline in the terminal for immediate review
  • User selects one or requests a regeneration with adjusted direction

Stage 4: Video Generation (WaveSpeed, latest-video-wavespeed)

Using the approved first and last frames, generate 2 video versions through generate_video.py --generate-video --image <first-frame> --last-image <last-frame> --quoted-providers <quoted> (the WaveSpeed image-to-video endpoint, 3-15 seconds; Veo clips are 4, 6 or 8 seconds and a length between is rounded up, which the result's duration_note says). The first frame goes to every provider in the chain; the last frame goes to the kling rung only, as the clip's end image. After each run, read video.last_frame_used before presenting a clip as landing on the approved last frame — see "Keyframe and reference inputs" below. If the result has video.provider_notice (or fallback_from and earlier_attempts), tell the user in one sentence which provider made the clip and why the first one did not (fallback_from lists only providers that were tried and failed; a rung named in skipped_unavailable never started and cost nothing, so do not say it failed); never present a fallback clip as if the first provider made it.

  • A video gallery is opened in the browser for side-by-side comparison
  • User selects the final version or requests a regeneration

Stage 5: Post-Process, Save & Deliver

After the user picks the final video, post-processing runs before saving:

  • Logo watermark is automatically added to the video via video_postprocess.py
  • Subtitles: User is asked whether to burn subtitles into the video (optional). SRT was already generated from the script and is saved separately regardless.
  • Background music: If the video has no audio (sound=False in the clip request), user is asked whether to add background music (optional).
  • Platform resize: Video is automatically resized for each target platform (letterbox/pillarbox with black padding, no stretching)

Save all final assets to {post_folder}/ -- keyframes in keyframes/, video versions in versions/, platform-resized final videos in final/:

  • Video files (.mp4) — post-processed and resized per platform
  • Script — timestamped narration/dialogue
  • Storyboard — shot-by-shot visual breakdown with keyframe references
  • SRT subtitle file (.srt) — for captioned playback

Keyframe and reference inputs — what generate_video.py implements

Read from scripts/generate_video.py. Never describe the video as doing more than this.

InputFlagWhat is true today
First frame (image-to-video)--image <path>Passed to every provider in the chain. The kling rung requires it — without one, and once its credential check has passed, that rung records bad-input and the chain moves on to a text-to-video rung. The veo and higgsfield rungs treat it as optional and run text-to-video when it is absent. Use a PNG or JPEG: file type is set from the extension (Veo) or always sent as PNG (the higgsfield rung).
Last frame--last-image <path> (requires --generate-video)Sent to the kling rung as the clip's end image (the WaveSpeed image-to-video API's end_image, documented there as the end frame for guided transitions), so the clip is guided toward it. Only that rung takes one: the veo and higgsfield rungs have no end-frame input. A clip made by either of them (because --provider named one of them, or because kling had no key or failed and the chain fell through) carries last_frame_used: false and a last_frame_note; it was not steered to the approved last frame. A --last-image path that does not exist is an input error (exit 1), never dropped quietly. Use a PNG or JPEG.
Reference images for videononeNot implemented.

Practical consequence: Stage 3's approved last frame is a real model input on the kling rung only. Under --provider auto, a supplied --last-image makes the chain try kling first (without one, auto sends clips of 8 seconds or less to veo whenever Vertex AI is set up, and veo cannot take an end frame; a Gemini API key alone does not count). Passing --provider kling says the same thing explicitly; an explicit --provider veo or higgsfield still wins and leaves the frame unused. Then read video.last_frame_used in the result. true means the end image was sent. false means it was not, and last_frame_note says which rung made the clip: tell the user, put the intended ending into the motion prompt in words, and do not describe the clip as landing on the approved frame. Even when it was sent, the model is guided toward the frame rather than guaranteed to reach it — look at the clip's final seconds before the user approves it.

Output Per Video Post

AssetFormatAlways Generated
ScriptMarkdownYes
StoryboardMarkdown + keyframe imagesYes
ThumbnailPNG/WebP (via compose-creative)Yes
First framePNGYes (Stage 2)
Last framePNGYes (Stage 3)
AI Video ClipMP4 via WaveSpeed (latest-video-wavespeed, image-to-video, 3-15 seconds)If pipeline completed
SRT subtitles.srtIf video generated

Video Types

TypeDurationAI GenerationProduction Notes
hero_video30-90sPartial — AI generates 3-15s hero clip; full version needs filmingScript + storyboard + AI teaser clip
mini_case_study30-60sYes — AI animation from keyframesFull pipeline supported
short_reel15-30s recommended (Instagram Reels and YouTube Shorts accept up to 3 min)Yes — ideal for AI generationFull pipeline supported
storyup to 60s per frame (Instagram Stories)Yes — image-to-video animationFull pipeline supported
talking_head30-120sNo — needs filmingScript + storyboard only (use --script-only)

Rules

  • Every stage requires explicit user approval before advancing
  • --script-only skips Stages 2-4 and generates script + storyboard only
  • --thumbnail generates a video thumbnail via compose-creative (independent of the pipeline)
  • AI video clips are never auto-saved; user must confirm the final selection
  • Thumbnails use the same creative mode system as static images
  • All assets save to {post_folder}/ -- keyframes in keyframes/, video versions in versions/, final video in final/

Timeout & Fallback

  • AI video generation (Stage 4): 300-second timeout (the video model can take several minutes for high-quality output)
  • Keyframe generation (Stages 2-3): 60-second timeout per image
  • If video generation fails or times out, deliver script + storyboard + keyframes as fallback

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.