Scroll Cinema

Two crafts in one: (A) the frontend pattern for scroll-driven film sections, and (B) the film production pipeline that makes a continuous, seamless film out of stills, real photos, and AI clips. Built and battle-tested on a production storefront; the pitfalls below were all hit for real.

A. The frontend pattern (capability-tiered, non-negotiable)

One section, three renderings, chosen by capability — this is the difference between "trend site" and "janky site":

Mechanics (see reference-component.jsx for a complete React/framer-motion implementation):

• Track & stage: a tall track (height: ~25-35vh per film-second) with a position: sticky; top: 0; height: 100vh stage inside. useScroll({ target, offset: ['start start','end end'] }) → progress 0..1.
• Scrub loop: scroll handler writes progress to a ref (never state); a single rAF loop eases the playhead: smooth += (target*duration - smooth) * 0.14; if (|video.currentTime - smooth| > 0.02) video.currentTime = smooth. The easing converts wheel steps into cinematic inertia.
• Text overlays are opacity/transform-only (GPU-cheap), timed to the film's act boundaries in seconds, fading OUT during camera-travel segments so moves play clean.
• Perf gating (mandatory): video src attaches only when the section is within ~1.5 viewports (IntersectionObserver + rootMargin: '150% 0px'), preload="none" until then, posters meanwhile. A film with preload="auto" at mount WILL download megabytes on initial page load — measured mistake. In chapter mode, only the active chapter's video plays; pause the rest.
• Video element: muted playsInline always; never call .play() in scrub mode (seek only).

B. Producing the film

The goal is ONE continuous take — no dissolves. Dissolves between unrelated clips read as a slideshow; the user will notice and say so.

Scene strategy (learned the hard way)

1. Real photos beat generated scenes. Ground every act in the user's real assets when they exist (product shots, process photos, phone footage frames).
2. AI-generated wide shots of technical apparatus WILL have wrong anatomy (tubes to nowhere, impossible plumbing). Two working fixes: (a) stay close — macro framing where each frame only needs local truth; (b) relight, don't invent — use an image-EDIT model with the real photo as reference: "keep this exact [object] and its geometry perfectly identical; re-photograph it in [lighting/scene]". Geometry stays real; only light is invented. This preserved a printed "50/42" joint marking and a product label perfectly.
3. Never image-to-video a still that carries text/labels — i2v drifts text (a Jupiter glyph became "24"). For label-bearing stills use Ken Burns instead: forward zoompan + reversed copy concatenated = seamless "breathing" palindrome loop, zero drift, zero cost.
4. Grade real daylight photos into the film's palette with ImageMagick/ffmpeg (this is color work, not AI). Pitfall: ImageMagick -vignette defaults to WHITE — always -background black -vignette ....

Seamless transitions: first/last-frame conditioning (the core trick)

Default: MiniMax H3 Max on fal.ai. It takes a start frame (image_url) and an end frame (end_image_url) and makes one camera move that starts EXACTLY on frame A and lands EXACTLY on frame B. It's cheap per clip and fast (a 5s move renders in under a minute). Use film.py in this skill's folder. It defaults to fal + H3 Max and also runs on Venice with the same flags:

python3 film.py quote                                     # price of one move, spends nothing
python3 film.py flf A.png B.png "One continuous slow camera move, no cuts: the camera [travel from A to B]." a-b.mp4
python3 film.py check a-b.mp4 A.png B.png                 # SSIM join check (see below)
python3 film.py still "<full shot description>" B.png --ref A.png --ref product.jpg   # keyframes

Provider: --provider fal|venice or $FILM_PROVIDER. Otherwise the script uses fal when it finds a FAL_KEY (env, .env in the cwd or a parent, or ~/.config/fal/env), then VENICE_API_KEY. Keyframes use Nano Banana Pro (nano-banana-pro, or the /edit variant when you pass --ref) on either provider.

H3 Max specifics (verified against the live schemas):

• No aspect parameter. The canvas follows image_url, so every keyframe must share the film's aspect (generate all at 16:9). Otherwise the end frame gets reframed and the join breaks.
• duration is 5-15 s (5 is the sweet spot per move). resolution is 480P/768P/1080P. 1080P is an upscale of the 768P latent, so 768P is the default and the scrub encode is 1152-1280 wide anyway.
• prompt_expansion_mode is required on fal. film.py sends disabled so your camera-move sentence is used literally (balanced/quality let MiniMax rewrite it).
• H3 always renders an audio track, and it can't be switched off. Drop it at encode (-an).
• Submit all moves at once. Each needs only its two finished keyframes, so the film takes about as long as its slowest move. On fal, poll the status_url/response_url the submit call returns: they drop the endpoint's sub-path.

Prompt shape that works: "One continuous slow camera move, no cuts: " + the physical travel from A to B (dolly, crane, orbit, push-in), or a transformation when the style allows it (the scene morphs, the camera passes through a surface). One sentence, with no text or logos in either frame.

Chain rule: every segment must start on the previous segment's literal last frame. With FLF the shared keyframe IS the chain point: segment N ends on keyframe K and segment N+1 starts from K. When assembling, drop the first frame of every segment after the first (trim=start_frame=1) so the shared keyframe doesn't sit on screen for two frames. For a non-FLF segment, extract its exact last frame: ffmpeg -sseof -0.05 -i seg.mp4 -frames:v 1 -q:v 2 chain.png (-sseof BEFORE -i).

This also lets generated camera travels land on real photographs: the transition's end frame IS the real photo. That's how you fuse generated motion with authentic material.

Join check, every clip, before assembly: film.py check scores the clip's first and last frames against the two keyframes (SSIM at 160x90). A clip that lands scores ~0.8-0.9, slightly lower at 480P or when the model lifts the exposure. An unrelated frame scores ~0.1. Below 0.75 means redo or re-prompt the move. Look at the extracted .first.png/.last.png either way.

Keyframes

• Make keyframe A first. Then make every later keyframe as an EDIT that passes A (or the previous shot) as a reference, so place, light and palette carry over. Add the user's real product or subject photo as a reference whenever it must appear, and open the prompt with "Keep the subject from reference N exactly as it is (shape, colours, details)."
• Write each keyframe prompt as a complete description of ONE frame: subject, framing, camera position, setting, light, palette, texture. Don't put readable text or logos in any shot.
• Show the stills to the user, and only animate the ones they approve.

Assembly & encoding (see ffmpeg-recipes.md for full commands)

• Normalize every segment: same WxH (e.g. 1152x648 or 1280x720), fps=24, setsar=1, crop to fill, never pad (padding bakes black bands into the film). Then use the concat filter, with no xfade anywhere.
• Scrub master must be all-intra: -an -c:v libx264 -crf 28 -x264-params keyint=1:scenecut=0 -movflags +faststart. With every frame a keyframe, every scroll position seeks instantly. The cost is ~0.3MB/s at 1152x648, the price of random access, so budget ~8-12MB for 30s. Grainy or B&W film compresses badly: add hqdn3d=2:2:4:4 to the normalizer and go to crf 30.
• Mobile chapter loops: normal encode (crf 26-27), looped as a palindrome (split[f][r0];[r0]reverse[r];[f][r]concat) or with a tail-crossfade: [0]trim=end=D-1[v0];[0]trim=start=D-1[v1];[v1][v0]xfade=fade:1:0.
• VERIFY EVERY JOIN in the assembled film too: extract frames at boundary±0.1s, montage them side by side, and look. for t in 4.9 5.1 ...; do ffmpeg -ss $t -i film.mp4 -frames:v 1 j$t.png; done; montage .... Joins that aren't pixel-matched read as cuts.

Budget discipline

Run film.py quote before any batch. On fal it reads the live per-second rate from fal's pricing API. That is the base rate, and higher resolutions cost more (check the model page). On Venice it uses the free /video/quote. Typical cost per 5s move: H3 Max Turbo ~$0.08, H3 Max ~$0.16-0.40 depending on resolution and current fal rates, and a Nano Banana Pro keyframe ~$0.15. A four-act film (5 keyframes + 4 moves) comes to about $1.50-2.50. State the total before generating, and draft moves with h3-turbo when a prompt is untested. Shoot stills, show them to the user, then animate only the winners. A blind API probe with junk input still queues a real paid job, so don't send one. Use throwaway, spend-capped keys and delete them when the film is done.

Order of work (whole feature)

1. Read the brand: palette, register, existing assets. Collect the user's REAL photos/footage first.
2. Storyboard acts + transitions; get the user's eyes on keyframes BEFORE animating (stills are cheap, motion isn't — and taste vetoes happen at every step).
3. Produce keyframes (real > relit-real > generated-closeup, all at the film's aspect), then H3 Max FLF moves on fal (film.py flf, all queued at once), film.py check each, then assemble + verify joins.
4. Build the component (copy reference-component.jsx as the starting point), retime FILM_TEXT_SEC/FILM_BOUNDS to the actual cut.
5. Perf pass: deferred src, data-saver tier, measure payloads per device class before declaring done.