blind-review
Skill0 starsChanged 18 days agoMITView on GitHub
Blind review of AI-generated stills, takes and cuts by a fresh subagent that never made them, answering fixed-option questions in JSON that code grades. Use before any still, take or cut reaches a person, after any change to a reviewed file, or to judge whether a fix is better than the current version.
7 other projects use a version of this.
Adopted by
Most recently taken up by
- gmunaro/claude-skillstheir own version, first seen Nov 16, 2026
- voidning/skillstheir own version, first seen Nov 8, 2026
- stevencarpenter/agentstheir own version, first seen Nov 1, 2026
- wellwelwel/lagunetheir own version, first seen Nov 1, 2026
- SanHsien/agent-astromechtheir own version, first seen Sep 27, 2026
- HartreeWorks/research-skillstheir own version, first seen Jun 1, 2026
Other versions7 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 7
- wellwelwel/lagune★ 216same name
- HartreeWorks/research-skills★ 4same name
- stevencarpenter/agents★ 2same name
- gmunaro/claude-skills★ 0same name
- SanHsien/agent-astromech★ 0same name
- voidning/skills★ 0same name
- wenboxia/blind-review★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository4 other files it gives its agents.
- product-sizeSkill
- recurring-castSkill
- scene-setSkill
- sound-mixSkill
The file
We split this skill into the rules it gives an agent
5 rules. 0 are common, 4 are distinctive, and 1 is very specific to this project.
0
Common
Most of the 5 edited versions we read keep these.
4
Distinctive
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 14
Code first: Run the measuring checks (scene-set's frame_check.py, product-size's size.py, recurring-cast's identity.py).
- Distinctive: line 31
Every question in the bank was born from a defect a client or the owner saw after the automated checks had passed.
- Distinctive: line 32
Add your own from every note a client sends.
- Distinctive: line 35
Use a cheaper model (Sonnet) per piece and the strongest (Opus) only on what goes to a person: the plan checkpoint, a fix, the final.
- Distinctive: line 40
Ask pairwise, not by a score: A against B, then again with the order swapped, and count it only when both runs agree.
“Common” is measured: kept by at least 6 in 10 of the 5 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Blind review
The agent that made a piece never grades it, and never on a metric of the effect it wanted. A fresh subagent sees only the pictures, the people, the script and the questions. Vision models are poor at "does this look right?" and decent at a narrow question with fixed answers ("how many fingers are on that hand?"), so the reviewer answers facts and code decides.
Run it
- Code first. Run the measuring checks (scene-set's
frame_check.py, product-size'ssize.py, recurring-cast'sidentity.py). Their numbers go to the reviewer as leads, not verdicts. - Make the sheets: face crops every 0.25 s, the whole frame every 0.5 s, the stills of one person side by side, the reference photos, and for a cut the story frames at 2 fps with the timed lines. Number the tiles and put the time on each.
- Write the questions for this piece from references/checklist.md into
questions.json: id, question, options (always with "can't tell"), and the answers that fail it. - Brief one subagent per piece, with its own crop folder (it may zoom only there). One line per person (what
they wear, where they should look), the script, what the set says this camera sees, then the questions. The
answer is JSON only:
{"<id>": "<option>", "notes": {"<id>": "<tile or time>: <one phrase>"}}. For a hand that works an object, the reviewer describes the mechanics in its own words before it reads the script's. End the brief with "Be honest in your response." - Grade in code:
uv run scripts/grade.py questions.json answers.json. A FAIL stops the piece: fix it, or ask the owner one fix-or-accept question with the frame. Quote the reviewer's notes in the report as they are. - Then look yourself, at full size: a second mic, an earbud and doubled bracelets once passed the questions and were caught only at full size.
Every question in the bank was born from a defect a client or the owner saw after the automated checks had passed. Add your own from every note a client sends.
Model choice
Use a cheaper model (Sonnet) per piece and the strongest (Opus) only on what goes to a person: the plan checkpoint, a fix, the final. On four saved reviews Sonnet gave the same verdicts as Opus; four is a small sample, so re-check on your own pieces. Repeat a review only on a changed file.
"Is the fix better?"
Ask pairwise, not by a score: A against B, then again with the order swapped, and count it only when both runs agree. This comes from published judge research and isn't measured on our pieces yet; calibrate it on choices the owner already made before you trust it.
Calibrate the reviewers
Once in a while, run the brief on past defects you kept (the golden set of your QA) and count what the reviewer catches. A question it never fails on a known defect needs rewording.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

