video-pipeline
Skill132 starsChanged 39 days agoMITView on GitHub
End-to-end video production workflow for the AI Edit Video system - from the user request to the MP4 in outputs/, coordinating HyperFrames (scenes) and Remotion (assembly). Read this when starting any video or when building the backend render queue.
- Sends data out
6 other projects use a version of this, 1 of them edited.
Adopted by
Most recently taken up by
- notivn/AIEVv2edited, first seen Nov 8, 2026
- kingbootoshi/video-alchemytheir own version, first seen Oct 25, 2026
- 439436269-ctrl/video-pipeline-skillstheir own version, first seen Oct 15, 2026
- Prawal-Bhuyan/claude-skillstheir own version, first seen Oct 8, 2026
- pjh2443391725-crypto/claude-skilltheir own version, first seen Jul 16, 2026
- yfge/video-skills-suitetheir own version, first seen Feb 28, 2026
Other versions6 projects use a version of this.
Word for word · 0
None found.
Edited · 1
- notivn/AIEVv2★ 13659% same
Wrote their own version · 5
- kingbootoshi/video-alchemy★ 9same name
- 439436269-ctrl/video-pipeline-skills★ 1same name
- yfge/video-skills-suite★ 1same name
- pjh2443391725-crypto/claude-skill★ 0same name
- Prawal-Bhuyan/claude-skills★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository20 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- ai-illustrationsSkill
- auto-cutSkill
- background-musicSkill
- color-gradingSkill
- gsapSkill
- hyperframes-cliSkill
- hyperframes-registrySkill
- hyperframesSkill
- key-layoutSkill
- make-a-videoSkill
- noti-tiktok-full-textSkill
- noti-tiktok-vnSkill
- noti-youtube-editSkill
- offline-voiceSkill
- remotion-assembleSkill
- short-form-videoSkill
- skill-authoringSkill
- website-to-hyperframesSkill
- webui-designSkill
The file
We split this skill into the rules it gives an agent
7 rules. 0 are common, 0 are distinctive, and 7 are very specific to this project.
0
Common
Most of the 3 edited versions we read keep these.
0
Distinctive
This author's own choices. Worth a look.
7
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 20
Create video-projects/<kebab-case-name>/ with index.html, compositions/, assets/, renders/, hyperframes.json, meta.json.
- Very specific: line 83
Update meta.json: status: "done", output: "outputs/...".
- Very specific: line 103
When using one in a video: copy it into video-projects/<name>/assets/sound-effects/ then declare it in meta.json - a project must contain …
- Very specific: line 112
Small edit -> re-render only what changed: if you edited one scene, re-render that scene (render -c <scene>), do not re-render the whole …
- Very specific: line 117
Transcription runs only ONCE: - if transcript.json already exists, reuse it, NEVER transcribe again.
- Very specific: line 130
Rendering a single scene: use the -c flag, not a positional argument: npx hyperframes render -c compositions/s01.html --quality draft …
- Very specific: line 132
The
sub_timeline_readiness_timeoutwarning: when rendering a template file with -c: the render still comes out correct (best-effort) but …
“Common” is measured: kept by at least 6 in 10 of the 3 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Video Pipeline - from request to MP4
Engine roles (never swap them)
| Task | Engine | Why |
| Scene motion graphics: kinetic typography, karaoke captions, count-ups, data tables, callouts, shaders | HyperFrames | HTML + GSAP is its strength, and the Vietnamese skills already ship the fixes |
| Stitching scenes + source footage, transitions between scenes, mixing audio + sound effects, full-video caption overlay, final export | Remotion | Programmatic assembly, <Sequence>/<Audio>/<OffthreadVideo> |
How the two engines talk to each other: intermediate MP4 files in video-projects/<name>/renders/. HyperFrames does not know Remotion exists and vice versa - the only shared thing is the manifest.
Project lifecycle
1. Initialize
- Create
video-projects/<kebab-case-name>/withindex.html,compositions/,assets/,renders/,hyperframes.json,meta.json. meta.jsonis the central manifest:
{
"id": "tiktok-paper-gpt5",
"name": "Mổ xẻ paper GPT-5",
"width": 1080, "height": 1920, "fps": 30,
"status": "draft",
"scenes": [
{ "id": "s01-hook", "src": "compositions/s01-hook.html", "durationInFrames": 90, "render": "renders/s01-hook.mp4" },
{ "id": "s02-talking","srcVideo": "assets/talking-head.mp4", "from": 2.5, "to": 14.0 }
],
"audio": {
"voice": "assets/voice.mp3",
"sfx": [ { "file": "assets/sound-effects/whoosh-01.mp3", "atFrame": 88 } ]
},
"output": null
}scenes[]is the contract between the two engines: a scene withsrcis rendered by HyperFrames; a scene withsrcVideois footage used as-is. Remotion reads this file to assemble - never hardcode the scene list in the Remotion code.
2. Build the scenes (HyperFrames)
- Copy
assets/brand/brand-tokens.cssinto the project, write compositions following thewindow.__timelinesconvention. - Vietnamese videos: apply the verified fixes (gradient text losing diacritics -> render text as real elements + use
background-clipcorrectly; check that every diacritic is present on the first/last frame of each reveal). - Lint clean before rendering:
npx hyperframes lint. - Render a draft of each scene:
npx hyperframes render --quality draft --output renders/<scene>.draft.mp4.
3. Frame verification (MANDATORY, before assembly)
ffmpeg -ss <hero-moment-seconds> -i renders/<scene>.draft.mp4 -frames:v 1 verify/<scene>.pngInspect every image: does the Vietnamese text keep all its diacritics? Is any text overflowing the edges? Any unexpected white/black frames? Is a face cropped? If anything is wrong, fix the scene - do not move on.
4. Assembly (Remotion) - see the remotion-assemble skill for details
- The Remotion composition reads
meta.json, builds<Sequence>entries fromscenes[], and inserts sfx atatFrame. - Render a full draft -> review it in the web UI -> approve.
5. Automated QC (MANDATORY, blocks final)
curl -s -X POST http://localhost:6869/api/projects/<id>/qc -H "content-type: application/json" -d "{}"The server measures the latest draft with ffmpeg and returns { report: { status, checks[] } }:
| check | meaning | what to do on fail |
resolution | dimensions/fps do not match meta.json | re-render with the correct settings |
loudness | far off -14 LUFS | adjust the mix volume, do not fix it by pulling the peak |
truepeak | clipping (> -0.5 dBTP) | lower the source gain, see "SFX loudness and lead silence" in the noti-tiktok-vn skill (voice headroom before layering SFX) |
blackframes | black frames in the MIDDLE of the video (leading/trailing fades are ignored) | a gap at a scene change - fix transitionOverlap/durationInFrames |
freeze | frozen picture >= 2s | the scene is missing animation, or the footage is broken |
tail-silence | silent tail > 1.2s | trim the tail |
av-duration | picture and sound differ by > 0.5s | wrong total durationInFrames |
safe-area | ALWAYS passes, returns frames = images with the obstructed bands outlined in red | you MUST Read every image and judge for yourself; if any text sits in the red zone, pull it inward, see the key-layout skill |
status: "fail"-> the server rejects theassemble-finaljob with 409 QC_REQUIRED / QC_FAILED. Fix the root cause, re-render the draft, run QC again. Do not useforce: trueto dodge it unless the user explicitly asks.status: "warn"-> consider fixing, not blocking.- The final report must state the QC results and how each fail/warn check was handled.
6. Final
- Re-render the HyperFrames scenes at
--quality standard. - Remotion final render ->
outputs/<project>-v<N>.mp4. - Update
meta.json:status: "done",output: "outputs/...". The web UI reads status from here.
7. Thumbnail + publish package
curl -s -X POST http://localhost:6869/api/projects/<id>/thumbnail -H "content-type: application/json" -d "{\"title\":\"...\",\"frameAt\":12}"
curl -s -X POST http://localhost:6869/api/projects/<id>/publish -H "content-type: application/json" -d "{}"publish generates .srt/.vtt from the transcript and has the AI write the title/description/hashtags for TikTok, YouTube and Facebook following the Style Design. Precondition: the transcript must be the FINAL one. If it was cut/remapped, write the final version to assets/transcript.final.json (the transcript loader prefers this file) - otherwise the subtitles will drift out of sync with the cut video.
Render queue rules (backend)
- Every render goes through the queue - including the ones Claude runs by hand. Jobs are written to SQLite:
id, projectId, type (scene-draft|scene-final|assemble-draft|assemble-final|image-gen), status, progress, log, startedAt, finishedAt. - Jobs run in parallel up to
QUEUE_CONCURRENCY(default 2, configurable via env; set 1 on weak machines). Safety constraint: two jobs from the same project never run at the same time (to avoid trampling renders/meta) - parallelism only happens across different projects. - Progress: parse the CLI stdout (both engines print frame progress) -> update the DB -> push SSE to the web UI.
- Failed jobs: keep the complete log in the DB, show it in the UI, and do not auto-retry more than once.
- Draft always comes before final. The backend rejects a final job if the project has no successful draft for the current version of the scenes.
Sound effects
- Shared library:
assets/sound-effects/, every file has an entry inassets/sound-effects/library.json(file,tags,durationMs, and adescriptionwritten in Vietnamese). - When using one in a video: copy it into
video-projects/<name>/assets/sound-effects/then declare it inmeta.json- a project must contain all of its own assets (re-rendering must not depend on a library that may change). - The Sound Effects page in the web UI reads
library.json, plays previews inline, and allows uploading new files (the backend updates the json).
Speeding up renders (this machine has a GTX 1660 - verified 2026-07)
The number one reason the pipeline burns an entire hour: re-rendering a draft of the WHOLE video after every small edit. The rules:
- Verify layout/text with
npx hyperframes snapshotorinspect(a few seconds) instead of rendering a full draft (tens of minutes). Render a full draft only ONCE, when every snapshot is good, and final ONCE. - Small edit -> re-render only what changed: if you edited one scene, re-render that scene (
render -c <scene>), do not re-render the whole composition. The split-scene + Remotion-assemble pipeline is built exactly for this. - HyperFrames speed flags (the backend queue adds them automatically; when running by hand you MUST remember them):
-w 8 --browser-gpufor every render; add--gpu(NVENC) for drafts. Final keeps CPU encoding (libx264) for maximum quality. Measured in practice: ~30% faster on short scenes, more on long ones. - Transcription runs only ONCE - if
transcript.jsonalready exists, reuse it, NEVER transcribe again. - Remotion speed flags (when running
npx remotion render/stillby hand you MUST add):--concurrency 8 --gl angle. WITHOUT--gl angle, Remotion's Chrome renders with the software renderer (SwANGLE) - the CPU carries 100% while the GPU idles (symptom: Task Manager CPU ~95%, GPU ~5%). On Linux use--gl angle-eglinstead ofangle.
Known issues (verified in production 2026-07)
meta.jsonmust respect the data-type contract (the web UI reads it directly - a wrong type crashes the page):outputis a path STRING (e.g."outputs/<id>-v1.mp4"), NOT an object. Extra metadata (duration, quality, renderedAt, ...) goes into a separateoutputInfofield if needed.scenes[].durationInFramesis a number; sfx are placed withatFrame(a frame number) as in the schema at the top of this file.- Rendering a single scene: use the
-cflag, not a positional argument:npx hyperframes render -c compositions/s01.html --quality draft --output renders/s01.draft.mp4. A<template>sub-composition must be referenced by index.html throughdata-composition-srcfor-cto be able to render it. - Words running together on per-word reveals: the HyperFrames pipeline swallows the whitespace between inline-block
<span>s - separate words withmargin: 0 0.14emon.word, do not rely on whitespace in the HTML. - The
sub_timeline_readiness_timeoutwarning when rendering a template file with-c: the render still comes out correct (best-effort) but wastes another 45s waiting - acceptable for drafts; if you want it gone entirely, render through index.html. - A TS comment containing the glob path
*/(e.g.video-projects/*/meta.json) closes the block comment early -> a baffling compile error. Writevideo-projects/<id>/meta.jsoninstead. - Vietnamese JSON sent with curl -d from Git Bash on Windows loses its diacritics (verified 2026-08: the
thumbnail title arrived at the server as "B?n th? kh?ch..." and rendered broken). The console codepage mangles
UTF-8 before curl sends it. For any API body containing Vietnamese, POST from Node instead:
node -e "fetch(url, { method: 'POST', headers: {'content-type': 'application/json'}, body: JSON.stringify({...}) })"(or write the JSON to a file and usecurl --data-binary @file).
Checklist before reporting completion
- If the brief enables autoCut: the cut was done per the
auto-cutskill, a second silencedetect pass verified the cut version, and the report states the seconds/segments removed - Every scene passed frame verification, Vietnamese text keeps all its diacritics
- Audio is in sync at the start/middle/end (check 3 random points)
- Sound effects land on the right frames, and their volume does not bury the voice (sfx ~10dB below voice)
- The output matches the dimensions/fps in
meta.json - Automated QC ran on the draft with no remaining
failchecks (the report spells out any leftover warn checks) meta.jsonhas updatedstatus+output- The thumbnail was generated and Read to verify the diacritics are complete
- The publish package was generated (
.srt/.vtt+ metadata), using the FINAL post-cut transcript - Any new lesson learned was written into the relevant skill
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

