html-video — HTML becomes video, on your laptop

Bring your local coding agent. Describe a video, or paste an article link / GitHub repo, and the agent turns it into a multi-frame, fully animated video — then renders it to a real MP4 right on your machine. One agent loop, pluggable rendering engines, a curated template gallery, optional AI soundtrack.

Prerequisites

Setup

cd html-video
pnpm install
pnpm -r build

Verify the install:

node packages/cli/dist/bin.js doctor

How it works

  prompt / link / repo
        │
        ▼
  ① source fetch        studio pulls the URL or repo server-side, flattens to Markdown
        │
        ▼
  ② agent loop          agent reads the material + picked template style, emits a
        │               content-graph (storyboard) + one HTML block per frame
        ▼
  ③ content-graph       multi-frame IR — nodes + edges, topo-sorted into frame order
        │
        ▼
  ④ per-frame HTML      each node becomes a self-contained animated HTML frame
        │
        ▼
  ⑤ engine render       headless Chromium records each frame → webm per frame
        │
        ▼
  ⑥ ffmpeg              each webm → mp4 (libx264), concat into one video
        │
        ▼
      your.mp4

Usage patterns

1. Start the studio (interactive UI)

node packages/cli/dist/bin.js studio
# Opens at http://127.0.0.1:3071

In the studio: pick a template, chat with your agent, edit per-frame text, add a soundtrack, and export MP4.

2. CLI: search templates

node packages/cli/dist/bin.js search-templates --intent "github stars race" --top 3
node packages/cli/dist/bin.js search-templates --intent "product promo 30s" --top 5

3. CLI: inspect a template

node packages/cli/dist/bin.js inspect-template --id frame-data-chart-nyt

4. CLI: create and render a project

# Create a project
node packages/cli/dist/bin.js project-create --name "my-video" --template frame-glitch-title

# Set variables
node packages/cli/dist/bin.js project-set-vars --project my-video --vars '{"title":"Hello World"}'

# Preview
node packages/cli/dist/bin.js project-preview --project my-video

# Render to MP4
node packages/cli/dist/bin.js project-render --project my-video

Turn a link into a video

This is the primary use case: hand the agent a link, get a video back.

User:   做一个解读视频 https://mp.weixin.qq.com/s/…
Agent:  Reads the article → builds a multi-frame explainer from the real content

Supported sources:

• Web article → fetched and flattened to Markdown (WeChat 公众号 supported)
• GitHub repo → description, structure, and README pulled via public API
• Just a prompt → agent writes content from scratch

Template gallery (21 templates)

Templates are organized by category:

Browse all 21 templates in the studio gallery (/api/templates).

Multi-frame videos (content-graph)

For videos with multiple scenes, the agent produces a content-graph:

{
  "nodes": [
    { "id": "intro", "kind": "text", "label": "Opening title" },
    { "id": "data1", "kind": "data", "label": "Key stat #1" },
    { "id": "outro", "kind": "entity", "label": "Logo end card" }
  ],
  "edges": [
    { "from": "intro", "to": "data1", "kind": "sequence" },
    { "from": "data1", "to": "outro", "kind": "sequence" }
  ]
}

Each node becomes one frame (self-contained animated HTML). The studio lets you reorder frames, edit text inline, and re-render individual frames.

Single-frame videos take a fast path that skips the content-graph entirely.

AI Soundtrack

In Settings → Audio, add a MiniMax API key. Then in the Soundtrack panel:

• Background music — describe a mood (e.g. "calm cinematic ambient, slow build")
• Narration — type a script for TTS

Both are mixed into the exported MP4 via ffmpeg (music ducked under voice).

Supported agents (14 backends)

Auto-detected on PATH; switchable from the studio's top bar:

Open Design (Vela) · Windsurf CLI · Trae CLI · Claude Code · Cursor Agent · Codex CLI · Gemini CLI · Grok Build · Qwen Code · OpenCode · GitHub Copilot CLI · Aider · Hermes · Anthropic Messages API

Architecture

packages/
├── core/                  Project / Asset / ContentGraph types, registries, orchestrator
├── content-graph/         Multi-frame storyboard IR (nodes + edges, topo-sort)
├── runtime/               Agent runtime — detect / spawn / stream (14 backends)
├── adapter-hyperframes/   Hyperframes engine adapter — real render via Chromium + ffmpeg
├── adapter-remotion/      Remotion engine adapter — React-based compositions
├── cli/                   `html-video` command + studio HTTP server + source fetching
└── project-studio/        Browser studio UI (chat, templates, frames, soundtrack, export)
templates/                 21 curated, license-clean video templates
research/                  RFCs (engine adapter / template metadata / agent skill / content-graph)

Rendering engines

The engine adapter interface is pluggable — swap engines without changing the storyboard or the agent workflow.

Key constraints

• Always render to verify — studio iframe previews use cached fonts; only exported MP4 shows the real render output. FOUT (flash of unstyled text) only visible in MP4.
• Multi-frame concat — when mixing frames from different engines, use concat FILTER (not demuxer) to rebuild the timeline correctly.
• Duration control — use durationMode: 'explicit' for multi-frame exports to prevent the engine from auto-extending frame durations.
• License-clean templates only — every template has SPDX license + provenance trail (origin / via_skill / transformation). No template ships without clear permissive license.

Related projects

License

Apache-2.0