motion-tracking-from-video
Skill1 starsChanged 10 days agoMITView on GitHub
Measure what objects visibly do in a video - where each one moves, turns and grows, frame by frame - with picsart_media_track_motion, work out each move's easing (curve, overshoot, bounce) and turn it into scene keyframes; then measure your render the same way and compare them with picsart_media_track_motion_diff. Covers targets, estimators, tolerances, reading results, and placing tracked motion on a layer. Use when a scene must reproduce a reference video's motion, or a render's motion must be checked against one.
1 other project uses a version of this.
Adopted by
Most recently taken up by
- PicsArt/gen-ai-skillsword for word, first seen Nov 1, 2026
Other versions1 project uses a version of this.
Word for word · 1
- PicsArt/gen-ai-skills★ 7100% same
Edited · 0
None found.
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository48 other files it gives its agents.
- animate-designSkill
- app-ui-animationSkill
- asset-auditSkill
- caption-decorationsSkill
- character-riggingSkill
- custom-transitionsSkill
- design-layer-importSkill
- edit-footageSkill
- edit-on-your-computerSkill
- edit-pipelineSkill
- face-hidingSkill
- figma-storyboard-animationSkill
- film-cuttingSkill
- film-finishingSkill
- footage-analysisSkill
- footage-reframingSkill
- get-startedSkill
- large-scene-assemblySkill
- linked-element-motionSkill
- make-cartoonSkill
- make-screen-videosSkill
- motion-design-craftSkill
- motion-intentSkill
- motion-pattern-librarySkill
- motion-reviewSkill
- motion-scene-continuitySkill
- photo-slideshowSkill
- picsart-video-rulesSkill
- quality-reviewSkill
- quick-merge-and-exportSkill
- realistic-color-gradeSkill
- recreate-videoSkill
- reel-structureSkill
- reference-fragment-matchingSkill
- resting-frame-rulesSkill
- retention-hooksSkill
- scene-3d-authoringSkill
- scene-animation-referenceSkill
- scene-building-basicsSkill
- scene-template-remixSkill
- screen-recording-analysisSkill
- screen-recording-layoutsSkill
- screen-recording-zoomSkill
- screen-ui-redesignSkill
- social-postingSkill
- speech-aware-cuttingSkill
- spoken-captionsSkill
- visual-styleSkill
The file
We split this skill into the rules it gives an agent
8 rules. About 0 are common, 3 are distinctive, and 5 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
5
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 39
Leave fps out, so it samples at the video's own rate, and run the window over the whole move: from the frame before it starts to the frame …
- Very specific: line 97
End the window before the object is covered: When it slides behind something, its confidence falls and its pose freezes.
- Distinctive: line 106
A track that is covered, then reacquired: After the confidence collapses, the later samples can be a valid track of the object that …
- Very specific: line 139
An expression: instead of keyframes means the channel fit one within tolerance: orbit (the path is a circle) or sine (a rotation that sways …
- Very specific: line 177
Use the tracker's own fitted keyframes and easings where the scene takes them (a preset or a cubic_bezier).
- Distinctive: line 218
Pixels: When the scene is not the reference's size, multiply each position by the scene's size over the reference's (the same factor on …
- Distinctive: line 223
Easing: keep each keyframe's easing where it is.
- Very specific: line 277
When a seam decision depends on a reference's velocity, path or overlap, the measurement feeds motion-scene-continuity (fetch with …
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Motion extraction
picsart_media_track_motion recovers what objects visibly do in a flattened video: their
position, rotation and scale on every frame, fitted into scene keyframes. It does not recover the
keyframes someone authored. It produces new motion data that reproduces what is on screen.
Looking decides what to track. The tracker measures where, every frame, to a fraction of a pixel. Never write a motion's numbers from a still or from memory: every timing and distance comes from a tracking result. The end points are not enough: a move is matched when it feels the same, so its easing (how it speeds up, slows down, overshoots and settles) is measured too, and built from the numbers.
Cost and receipts
Measure every required window with picsart_media_track_motion and
picsart_media_track_motion_diff, applying picsart-video-rules's SPEND rule to their billing
contracts. Each replies with a job receipt by default: follow it as picsart-video-rules says. Targets that share a tracking window go in one call, as many as
its targets schema allows: one t0/t1 holds for every target, and each must be fully visible at
t0. A target that enters later, or is covered before t1, gets a call with its own window.
The method
- Look at the first frame of the window with
picsart_media_video_stills, and choose what to track. On a flat background,{ "auto": true, "max": 9 }finds the distinct objects att0and tracks each, ranked by how much they move. Objects that overlap merge into one, so give those a box each instead. - Track.
picsart_media_track_motionwith the video'ssource, the windowt0/t1in video seconds, andtargets: each onebbox([x, y, w, h]att0, in video pixels),points, orauto. Settolerancefor the source (below). For a card, photo or title, abboxin"mode": "similarity"(position, scale and rotation);"corners"when it skews or tilts in 3D. Leavefpsout, so it samples at the video's own rate, and run the window over the whole move: from the frame before it starts to the frame after it settles, so the easing and the settle are in the samples. - Read the result before you trust it. Per target it returns
animations(the fitted keyframes with their easing, or an expression),static(the channels that did not move),metawith each channel'smaxError, andnoteson anything that weakened the track.maxErroris the honest reconstruction error: a fit with a large one is an approximation, and you say so. - Work out the easing of each move from the dense samples (below).
- Place it on a layer (below), as keyframes with
timeMode: "layer". - Check it. Export your render of the same window, track it with the same targets, and compare
the two with
picsart_media_track_motion_diffand their speed profiles (below).
What the tracker cannot see
It measures where texture goes, frame to frame. Some changes have no texture to follow:
| Change | What to do instead |
| a reveal: a wipe, a mask growing | cropped stills of the box, one per frame of the reveal, for its edge; track whatever drives the reveal |
| a fade or a twinkle | stills either side and through it; the tracker has no alpha (below) |
| non-rigid motion: a flap, a bend, a wobble | track the centre only, and build the shape change as a loop |
| a small idle loop: bobbing, swaying, breathing | track over several cycles, at least one and a half: a sway in rotation comes back as a sine expression; build any other loop from the cycles you measured |
| motion blur on a fast move | track it again with "estimator": "segmentation", whose centre stays on the smear, and read it with fit_motion.py --min-confidence 0.2; check the smeared positions against cropped stills, one a frame. Each move says how many of its samples were smeared (blurred) |
Illustrated templates
Stickers, doodles and cut-out shapes over a photo or a flat background:
- A sticker with a white border tracks best with
"estimator": "segmentation": the border is a clean silhouette, where its flat fill gives features little to hold.
Tolerance, per channel and per unit
tolerance is position in pixels, toleranceScale a ratio (default 0.01) and
toleranceRotation degrees (default 0.5). Never reuse a pixel tolerance for scale: 1.5
"pixels" once declared a 57% shrink static.
| Source | tolerance |
| a compressed reference video | about 1.5 (it measures to about 0.5–2 px) |
| your own render | 0.5 or less |
Too tight on a compressed source and the keyframes chase codec noise.
Choosing targets
- Start where the object is solid. During a fade-in its features are weak, the track dies on
the first frame and freezes. Start
t0once it is fully opaque. - A tight box inside the object's face. A box over a tilted card in a stack catches its neighbours, and the fit mixes two motions.
- Opaque objects only, for rotation and scale. A half-transparent object blends with whatever moves behind it, and its measured pose drifts toward that. Ghosts, depth-faded cards and translucent chrome give approximate motion at best; say so.
- Video playing inside the box: the features follow the content, not the box. Set
"estimator": "segmentation"on that target, which tracks the silhouette. Its centre leans toward bright content, so a still element of the same group, tracked instead, is more exact. - A rotating object takes a box, never points. A point tracks translation only: a corner
turning in place reads about 4 px wrong at 30°. A box in
similaritymode compensates. - End the window before the object is covered. When it slides behind something, its confidence
falls and its pose freezes. Shorten
t1and track again, rather than keep the frozen tail. - Coming in from off screen: while the object is clipped by the frame edge, a box around it is
biased. Start at its first visible frame with a box on the textured part already in view (a
photo's face, not its edge), with
"estimator": "features", so the fit follows the content, not the clipped outline.fit_motion.pythen marks the movestartsBeforeWindow: build the entrance from off screen so it crosses the edge at the measured speed. - Clip the window, not the keyframes. A fit over frozen samples emits false keyframes at the
tail. Track again with an earlier
t1; never prune the keyframes by hand. - A track that is covered, then reacquired. After the confidence collapses, the later samples can be a valid track of the object that replaced it (a card becoming a chip). Check them against a still before discarding them.
- One object, two stretches that need different estimators (a video inside a card, then siblings touching it): track each stretch on its own, with one overlapping frame. The two estimators' reference points differ by about 2 px: measure that offset on the overlap frame and subtract it when you join them, and multiply the second stretch's scale by the overlap frame's scale.
| Content | mode | estimator |
| textured (a photo, a UI card with text) | similarity | features |
| flat or solid, or video inside the box | similarity | segmentation |
| stretched in perspective | corners | features |
| position alone matters | center | either |
Reading the result
- Position is the object's centre, in the video's own pixels. Under a layer's default anchor
(the centre of its box, for every content kind but text), that is the layer's
transform.position. Text anchors at its origin: place it by its alignment instead. - Rotation and scale are relative to
t0: 0° and 1.0 there. The tracker cannot know an object's tilt att0from one box: add the pose the layer starts at yourself. - Rotation is counter-clockwise positive; a scene's rotation is clockwise. A turn the tracker
reports as +12° is
-12in the scene. Flip the sign of every rotation value you place. - Scale comes as
[s, s], because a scene's scale is 2-D. - Keyframe times count from
t0unless the call setnormalizeTime: "absolute". - Easing is a preset or a fitted curve, chosen per segment. The closest of the four presets
(
linear,ease_in,ease_out,ease_in_out) comes first. When none of them fits withintolerance, the segment moves and it has enough samples, it gets a fitted curve instead:{"kind": "cubic_bezier", "control": [x1, y1, x2, y2]}, the scene's own parametric easing, with a y outside 0–1 for an overshoot. Copy either form into the keyframe as it came. A short segment keeps its closest preset, so a sharp snap or a long settle there reads close, not exact: itsmaxErrorsays how close; track it denser to give it a curve. - An expression instead of keyframes means the channel fit one within tolerance:
orbit(the path is a circle) orsine(a rotation that sways back and forth,meta'skindsays which). On a compressed reference real paths often are not pure circles, and the keyframes it falls back to are the honest answer, not a failure. - The tracker has no alpha. It cannot tell a fade from a move. Judge a fade on stills either side of it, and never invent one to cover a jump in pose.
The feel of each move: easing
The fitted animations give each segment's easing, and the report at resultUrl holds every
sample (t, x, y, rotation, scale, confidence). Save that report (as a file, where you
run commands) and read each move's timing curve from it:
python3 scripts/fit_motion.py moves track.json [--target card1] [--into sheet.json]Per move, on each channel that moves, it gives:
- the target, the channel,
fromandto; delay(from the window's start) andat(source seconds), thenduration, up to the frame it settles, andhold, how long it rests after;curve:linear,ease_in,ease_out,ease_in_out, orspringwhen it overshoots and swings back, named from where its peak speed comes (peakAt, 0 to 1) and how far above the mean speed it is;cubicBezier, the curve fitted to the samples (a y outside 0–1 is an overshoot), andpresetwhen one of the scene's four presets fits about as well;overshoot, how far past the rest it goes as a share of the travel,swings, how many times it crosses its rest before it settles, and for a spring its stiffness and damping (mass 1);peakSpeedandmeanSpeed, andstaggerAfter: how long after the previous target's move it starts;keyframesthe scene can take as they are: one curve, or for a spring each swing as its own segment, since one cubic-bezier holds only one overshoot.
A move out and back, or one that bounces back and forth, comes back as one move per leg; a spring's swings about its rest, each smaller than the last, stay one move however lightly damped. Where it rests is measured from the frames that hold still after it; only when the track ends while it still moves is the rest worked out from its swings. A rotation that crosses ±180° is read as the short turn it is. A slow move
is kept: its start is the last frame within the tolerance of the rest before it. --into replaces
every target and channel it measured, so a channel measured still again loses its old move. Track each window of one object under its own target id (card1_in, card1_out): moves of the
same id and channel from another report are replaced.
Use the tracker's own fitted keyframes and easings where the scene takes them (a preset or a
cubic_bezier); take fit_motion.py's keyframes only where it does not, such as a spring with
more than one swing. Either way, the numbers that describe the move come from fit_motion.py.
A move it marks startsBeforeWindow, or not settled, was tracked over too short a window:
widen it and track again.
Record each move where the use case keeps the spec: target, channel, from → to, duration, delay, curve, overshoot, and the tracking result it came from. Never words alone ("slides in").
Known limits of splitting moves. Samples alone cannot tell these apart; fit_motion.py
reads them as below, and its tests pin it. Look at the stills when one matters.
- A trip whose swings shrink like a spring's, with the track ending before it holds still (0→100→30→80, still moving at the end), reads as one spring. Keep a still second after the motion in the tracked window, and the measured rest splits it.
- One turn back by less than the travel, then still (0→100→60), reads as one move with an overshoot (67 % here), as an ease with a big overshoot would. A turn back past that (0→100→40) is two moves.
- A spring that freezes mid-swing and holds there rests where it froze: it may come back as more than one move, and the last one ends where it froze.
- A window that opens on the frame the spring is released, with the track ending after two turns (still moving), cannot show the spring started from rest: it reads as up to three ease moves. Start the window a few still frames before the release, or track it until it settles, and it is one spring.
Creating similar motion
- Build each move from its numbers: its from and to, its duration, its curve (the preset, the
fitted
cubic_bezier, or the spring's keyframes) and its overshoot. - Keep the order and the stagger. The moves start in the reference's order, each
staggerAfterthe one before it, scaled to the scene's time like every keyframe. - Match a text animation by its mechanism, not only its timing: a wipe, letters one by one, a line rising, or a mask reveal that follows a moving object (a sticker that wipes the title in as it passes). Read the mechanism from cropped stills of the reveal, one per frame, and track the object that leads it; then build the same mechanism, with the measured timing.
Placing tracked motion on a layer
The result is in the reference's frame and time. A layer needs its own:
- Time. A keyframe's layer time is its tracked time, plus
t0when times count fromt0, plus the scene second where the reference's second 0 plays, minus the layer'sstart. - Pixels. When the scene is not the reference's size, multiply each position by the scene's size over the reference's (the same factor on both axes) and add the offset of where the reference's frame sits in the scene.
- Pose. Rotation: the layer's starting rotation minus the tracked value. Scale: the layer's starting scale times the tracked value.
- Easing: keep each keyframe's easing where it is; it shapes the segment that starts at it.
An expression is in the reference's time, and an orbit also in its pixels, so these rules cannot move it: when the scene differs from the reference in size or timing, track that object again as keyframes, or place it at the reference's size and time inside a composition sized like the reference.
Checking a render against the reference
Export your render over the fragment and track it with the same box as the reference: the same
part of the object, the same size against it, at the same moment of the move, with the same mode
and estimator. The tracker reports the centre of the box you drew, so another box is another
point, and a card tracked by its face against one tracked by its corner reads 150 px apart while
both move alike.
Then measure the deviation with the starting offset removed:
python3 scripts/fit_motion.py diff ref-track.json our-track.json --offset <ref second 0 minus ours>Both tracks are read at every sample time of either, over the time both cover. For each channel that changes on either side from the first shared sample (a slow pan too), it gives:
startOffsetandplacement: where ours sits at the first shared sample against the reference;maxDev: the raw deviation;moveDev: the deviation once the starting offset is removed, which is the move itself.
The move passes on moveDev, within the threshold the use case sets. The placement is the layout:
report it, and fix it when the layout should match. picsart_media_track_motion_diff gives the raw
maxDev, meanDev and worstT for the same pair. A number replaces "it looks about right".
Then check the feel. A linear move can pass the position check against one that eases. Compare the two speed profiles:
python3 scripts/fit_motion.py compare ref-track.json our-track.json --offset <ref second 0 minus ours>Per move it checks the start, the duration, where the peak speed comes (not when both moves are
linear: a linear move has no peak) and how big it is, the overshoot, the swings and the curve's
name, and says which differ. speed writes each target's
speed over time as CSV, to show the two side by side. Fix the measured differences and check
again while each correction makes progress. If the evidence or a tool limit prevents a match,
report the remaining difference with its numbers and cause. ASK (with your host's own question tool, such as AskUserQuestion, so the options are buttons the user taps, the recommended one first; only where the host has none, the options numbered in one message; in unattended pace, never ask: write it to the project record's openDecisions with the answer you assumed, as picsart-video-rules says) whether to accept that
approximation or change the approach; never label it a pass.
- Like for like. The same targets, the same boxes, the same window, the same tolerance and the
same
--min-confidenceon both sides. Your render tracked from its own second 0 against the reference from its second 4 is a 4 s offset, not a mismatch. - What the deviation says. A constant offset is geometry: a position or a size. A deviation that grows and shrinks along a move is easing. One that peaks at a beat's start or end is timing.
Measuring for a seam
When a seam decision depends on a reference's velocity, path or overlap, the measurement feeds
motion-scene-continuity (fetch with picsart_scene_recipe). Tracking answers what moved, where
and when; continuity judges what that movement communicates and whether the scene should preserve,
replace, interrupt or reset it. A close numeric fit does not by itself prove a readable handoff.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

