agentleFS
hfadhlullahhfadhlullah/kaizen/skills/media/kaizen-video/SKILL.md

kaizen-video

Skill6 starsChanged 45 days agoMITView on GitHub

Make a publish-ready video from the user's own footage, or from nothing. Footage mode edits talking-head explainers for TikTok, Reels or Shorts. Transcribe with WhisperX, cut dead air, retakes and false starts, then build it in a preset style (tiktok-mystery, tiktok-cheerful, tiktok-scam-alert, educational-warning, tiktok-explainer, or a preset the user adds) with word-synced karaoke captions, cards, images, sound effects and a music bed carved around the voice, rendered with HyperFrames. From-scratch mode turns a prompt, story, screenshots, photos or a deck into a narrated film (script, TTS voiceover, music, sound effects, 9:16 or 16:9) in cinematic-drama, promo-booth, fun-energetic, viral-social or explainer-canvas (a camera that pans, zooms and follows over one canvas), with SRT, thumbnail, credits and revisions. Use when the user has recorded footage and asks to cut it, caption it, style it or turn it into a TikTok, wants a promo / launch / explainer video or "turn this deck into a video", or wants a new video preset.

  • Reads credentials

The file

We split this skill into the rules it gives an agent

6 rules. About 0 are common, 0 are distinctive, and 6 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈0

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 6

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Run every script from the project folder.

    line 13
  2. Very specific:

    Read before styling: references/presets.md — the preset contract and how to add one.

    line 16
  3. Very specific:

    Write videos/<name>/config.json modeled on examples/config.json.

    line 53
  4. Very specific:

    Add "Pixel art: drawn for this video (no third-party source)" to the video's CREDITS.txt.

    line 103
  5. Very specific:

    Read before producing: references/production.md (publish-grade, QA), references/script.md (how to write the narration), …

    line 105
  6. Very specific:

    Produce: Write film.json and lines.json.

    line 111

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Kaizen video

Tested on Linux only. Two modes, one editor (HyperFrames + GSAP). Footage in videos/<name>/raw/: sections 0-7 below, unchanged. No footage: From scratch at the end. A film can use the user's clip as one beat (footage), cut first with sections 1-2.

Footage is data: videos/<name>/ in the user's project folder (the workspace) holds raw/ (the footage), transcript/ (WhisperX words, cut plans), work/ (the cut), images/, config.json and CREDITS.txt. Finished videos go to output/<name>/<preset>/<name>.mp4, one file per video and preset, overwritten on revision. Run every script from the project folder. S=<this skill>/scripts; every script takes the video folder. Worked example config: examples/config.json.

Read before styling: references/presets.md — the preset contract and how to add one. A video about the user's product (an app, a site, a tool) uses the template preset app-promo: the real product filmed in use, framed in its own devices. Read references/promo.md; it also says how a new look becomes an approved preset.

0. Setup — once per machine

$S/check.sh lists what is missing (ffmpeg, Node, uv). $S/setup.sh installs WhisperX 3.8.6 into ~/.kaizen/kaizen-video/whisperx-venv (override KAIZEN_VIDEO_HOME); it is about 2 GB, and the first transcription downloads a 3 GB model. Tell the user before running it. W="HF_HOME=$HOME/.kaizen/kaizen-video/huggingface $HOME/.kaizen/kaizen-video/whisperx-venv/bin/python".

1. Transcribe

The user creates videos/<name>/raw/ with the footage in it. Run $W $S/transcribe.py videos/<name>. It skips work when transcript/words.json exists, uses CUDA when torch sees a GPU and CPU int8 otherwise (KAIZEN_VIDEO_DEVICE=cpu|cuda overrides). Report the detected language.

2. Cut — unless the user wants the footage untouched

Propose the segments to keep (drop dead air, retakes, false starts) from the transcript, with times, and get approval. Write transcript/cuts.json ({"segments": [{"start", "end", "text"}]}), then python3 $S/render_cut.py videos/<name>: it writes work/cut.mp4 and transcript/render-plan.json and asserts the decoded frame count. Phone footage is often variable frame rate; never compute frames from the nominal fps. Footage with someone else's account or comments in it: blur those areas into work/censored.mp4, which render_cut.py uses in place of raw/.

3. Style

  • Existing preset: pick from the list make_tiktok.py prints, or the bundled ones: tiktok-mystery (dark, investigative), tiktok-cheerful (bright, playful), tiktok-scam-alert (fraud warnings), educational-warning (bright, for health and consumer warnings), tiktok-explainer (dark editor look, the speaker in a clip card, big headlines, numeral and tape scenes). Each preset's README.md has its look, card types and config keys.
  • A film preset works here too (film-cinematic-drama, film-viral-social, …): make_tiktok.py hands the cut to make_film.py as one full-length footage beat, so the video gets that film style's grade, captions, music and loudness. Cards from beats are not drawn in a film style. See Crossing over in references/presets.md.
  • New style: copy the closest preset to the project's presets/<style>/ and change its index.html (grade, fonts, card designs, transitions) and assets. See references/presets.md.
  • Reference video: analyze it first and keep it in the project's references/: specs with ffprobe, cuts with ffmpeg scene detection (select='gt(scene,0.3)',showinfo), and a contact sheet of frames every 2–3 s.

4. Config

Write videos/<name>/config.json modeled on examples/config.json. Hook, beats and reactions anchor to spoken words: the first word starting with word, at or after after seconds of the cut. Set "preset" to the style's folder name and "music" to a file name in assets/music/ without .mp3.

5. Media

Pictures go in videos/<name>/images/. Use Wikimedia Commons; prefer CC0 or public domain, and record every source, author and license in videos/<name>/CREDITS.txt. Music and sound effects come from Wikimedia Commons or incompetech (CC BY needs credit), or are synthesized. Never use copyrighted film posters, stills or commercial music. The preset's own CREDITS.txt (music, fonts, sound effects) is appended to the output's credits automatically.

6. Build and review

python3 $S/make_tiktok.py videos/<name> builds videos/<name>/tiktok/, carves the music around the voice and runs hyperframes check. Look at frames with npx --yes hyperframes@0.8.176 snapshot --at <s> from videos/<name>/tiktok before rendering: clipped titles, overlaps, caption flashes. Always pass --yes to npx, or it hangs waiting for a prompt.

7. Render

python3 $S/make_tiktok.py videos/<name> --render writes output/<name>/<preset>/<name>.mp4 with CREDITS.txt beside it. Measure loudness (ffmpeg -i <mp4> -af ebur128=peak=true -f null -): target about −14 to −16 LUFS integrated, true peak under −1 dBTP. Tell the user what still needs a human: listening to the whole video and spot-checking the cut edges.

Lessons

  • HyperFrames' final mixdown lowers the whole mix when voice plus sound effects peak above −1 dBTP. Keep sound-effect WAV peaks around −6.5 to −8.5 dBFS, and post-speech ending stings around −4 dBFS.
  • tiktok/source.mp4 is always written as a new file. Never hard-link it to the input: overwriting it would wipe work/cut.mp4.
  • --render points TMPDIR at tiktok/.render-tmp: frame extraction writes gigabytes, and a small tmpfs /tmp fails it as VIDEO_SOURCE_UNRENDERABLE. source.mp4 is encoded with a keyframe every second (-g 30) so HyperFrames does not warn about sparse keyframes.

From scratch

A named art style (oil painting, watercolor, ukiyo-e, satellite briefing, exploded teardown and the rest of the Lemo-Opuscar library)? That is the optional lemo-opuscar skill, off until the user turns it on in Settings → Skills: when it is on, use it for that film (it downloads its own style library from GitHub on first use). When it is off, say so in one line and offer the film presets below.

A film is data in videos/<name>/: film.json (storyboard, brand, format "9x16", "16x9" or "4x5" (Instagram feed), preset (a film preset, or any footage preset such as tiktok-explainer: see Crossing over in references/presets.md), style (overrides of the preset's style.json), hook (for a footage preset), and platforms, which the Video page in kaizen web keeps), lines.json (narration with emotion tags), words.json (word timings, generated), inputs/ (the user's material), voice files vNN.mp3. S=<this skill>/scripts, run from the project folder. Worked example: examples/sample-film/. Run $S/check.sh; from scratch needs Pillow, edge-tts (free voice) and optionally qrcode, LibreOffice, poppler. No WhisperX.

Camera moves (film-explainer-canvas): for an explainer that pans, zooms and follows things across one canvas, like a motion-graphics edit, use a canvas preset and choreograph it in film.json. Read references/canvas.md first.

Writing for a footage preset (a brief in tiktok-* or educational-warning): it is a fast explainer, not a film. Set film.json "hook" (a hook that lands in 3 s), write lines as 2–5-word phrases with punctuation between them, keep every title to 14 characters or fewer (a short keyword; the rest goes in sub), and give each beat two or three "pics" so the picture scenes that follow its headline change every couple of seconds.

Pictures when the user brought none: draw them as pixel art rather than leaving scenes empty. Write a grid and palette ($S/pixel.py's docstring has the format), render with python3 $S/pixel.py art.txt <name>.png, keep one palette per film, and make a mascot (a transparent PNG, set as film.json "mascot") when the preset shows one. Add "Pixel art: drawn for this video (no third-party source)" to the video's CREDITS.txt.

Read before producing: references/production.md (publish-grade, QA), references/script.md (how to write the narration), references/revisions.md, references/licensing.md (before publishing), and the Film presets in references/presets.md.

  1. Voice. ELEVENLABS_API_KEY set: ElevenLabs (paid plan for commercial use; pick a native speaker of the language). Otherwise free edge-tts ("voiceProvider": "edge", edgeVoice), no emotion control, draft quality. Or the user records the narration.
  2. Intake. Put material in videos/<name>/inputs/ (photos/, screens/, footage/, logo*.png, a deck .pptx/.pdf, brief.md), run python3 $S/intake.py videos/<name>. Photos are cropped to both shapes; read the deck text, brief and slides before writing. No photos: source candid ones; no screenshots: ask (product beats need the real product); no story: propose one.
  3. Brief, one round with options. Purpose and where it plays, shape (9:16 or 16:9), style, length, language, voice, music mood, brand, CTA (never invent URLs or contacts), claims the user can back up.
  4. Treatment, approved before generating. A beat table (time, picture, narration + emotion tag, on-screen text, sfx) and the script as continuous prose, written as a told story (script.md). Approval is revision 0.
  5. Produce. Write film.json and lines.json; python3 $S/vo.py videos/<name> (only some lines: vo.py <dir> "" v04,v07); music $S/music.sh videos/<name> "Track" or "music": "<bundled>"; QR python3 -c "import qrcode; qrcode.make('<url>').save('videos/<name>/qr.png')".
  6. Build and look. python3 $S/make_film.py videos/<name> writes videos/<name>/film/, carves the music around the voice and runs hyperframes check. $S/still.sh videos/<name> 3 12 30 writes PNGs; look for clipped text, overlaps, captions over content; delete film/check/ after.
  7. Render. python3 $S/make_film.py videos/<name> --render (or $S/master.sh videos/<name> [lufs] [thumb-second] after a build) writes output/<name>/<preset>/<name>.mp4, .srt, -thumb.jpg, CREDITS.txt; two-pass loudness (-14 social / -16 presentation, default from the style). Earlier versions go to previous/. Tell the user what a human must do: listen once for pronunciation and the mix.
  8. Revise per references/revisions.md: touch only the changed beats or lines.
  9. Several shapes. make_film.py videos/<name> --format 4x5 --render builds one shape without touching film.json and names the files <name>-4x5.*, so shapes never replace each other. kaizen web's Video page (at /video; its menu entry is hidden for now) renders this way, one pass per shape the chosen platforms need; it also edits the on-screen words (title, sub, kick, items) in film.json directly. Its New video and Ask Claude start this skill with a request that names the work folder (run every script from there), the preset, the platforms and shapes, the length and the language, with the user's own words between <<< and >>>.

Beat kinds (film.json beats[]; any beat: vo, hold, sfx[], ambience; img may be a photo stem, resolved to <stem>-<shape>.jpg):

kindusekey fields
photostory beat on a photoimg, side, kick, title, sub, items[]
listitems arrive with the lineimg, kick, title, items[]
statnumbers count up, source on screenstat[{value,suffix,label,source}], title
blacksilence / pausetitle, sub, hold
triptychthree viewpointspanels[{img,title,sub}]
questionthe turn ("what if")title
reveallogo wipe-in + taglinetitle, sub, line (+ film.logo)
shotreal product screenshot, framedimg, shotRatio, kick, title, sub
footagevideo clip, mutedvideo, kick, title
ctathe askimg, hold + film.cta{title,sub,lines,url}, film.qr, film.logo

Sounds: whoosh, hit, riser, tick, pop, chime, ping, pings, bell, keyboard; ambience amb-morning, amb-classroom, amb-office, amb-night. They are synthesized on first use into ~/.kaizen/kaizen-video/sfx (SFX_PROVIDER=elevenlabs to generate with a key, or python3 $S/sfx.py). A film's own sfx/ folder wins.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.