swebench-reviewer
Skill1 starsChanged 3 months agoApache-2.0View on GitHub
Independently review and correct one existing SWE-bench candidate patch before official evaluation.
In this repository15 other files it gives its agents.
- AGENTS.mdAGENTS.md
- agent-browserSkill
- background-task-runnerSkill
- officecli-docxSkill
- officecli-pptxSkill
- officecli-xlsxSkill
- rememberSkill
- role-publishSkill
- self-evolutionSkill
- skill-publishSkill
- swebench-correctorSkill
- swebench-recoverySkill
- swebench-solverSkill
- vision-analysisSkill
- webcliSkill
The file
We split this skill into the rules it gives an agent
9 rules. About 0 are common, 9 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈9
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 13
Start with git diff and the public issue.
- Distinctive: line 15
Use the deterministic public-contract coverage facts to account for every named setting, API, class, function, and template tag.
- Distinctive: line 17
Check adjacent behavior explicitly: equivalent entry points, custom names or identifiers, ordering and filtering semantics, and error …
- Distinctive: line 20
Treat reconstruction completeness as an invariant: deconstruction must preserve non-default values needed to recreate the configured …
- Distinctive: line 21
Treat migrations and newly added destructive data operations as high risk.
- Distinctive: line 25
Prefer source changes over benchmark-test changes.
- Distinctive: line 26
Run the narrowest available reproduction or focused test before the final decision.
- Distinctive: line 27
Never invent a validation result.
- Distinctive: line 28
Do not use official Harness reports, hidden tests, gold patches, network services, subagents, or external coding agents.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
SWE-bench Patch Reviewer
Work only inside the provided repository. A candidate patch already exists.
- Start with
git diffand the public issue. Do not repeat broad repository exploration. - Identify the exact public behavior contract and confirm that the edited module owns it. Use the deterministic public-contract coverage facts to account for every named setting, API, class, function, and template tag. A cue may be implemented indirectly only when the changed abstraction demonstrably governs it. Reject a template, presentation, transport, or other downstream-only patch when the issue requires direct behavior from a shared setting, storage, configuration, or API surface.
- Check adjacent behavior explicitly: equivalent entry points, custom names or identifiers, ordering and filtering semantics, and error type/message when relevant. For empty-input, filtering, broadcasting, or shape-sensitive fixes, cover an edge matrix: all inputs empty, each input empty independently, mixed empty/non-empty inputs, singleton inputs, and output shape/dtype preservation. For callable, lazy, deferred, or environment-dependent configuration, verify the lifecycle separately: constructor storage, serialization/deconstruction, and the runtime consumer. Do not assume serialization evaluates a callable unless the code proves it. Treat reconstruction completeness as an invariant: deconstruction must preserve non-default values needed to recreate the configured object, including importable callables. Treat migrations and newly added destructive data operations as high risk. Do not accept a patch that avoids an integrity error by deleting existing rows unless the public contract, forward/reverse semantics, idempotency, and focused validation prove that deletion is intended.
- The runtime assigns a 6, 8, or 12 turn budget from patch complexity. Finish investigation before the final three turns.
- During the two turns before the final turn, only
edit_file,write_file, orexecute_shellis permitted. Use shell only for a focused validation command, never for more searching or file reading. - The final budgeted turn is tool-free. Do not request tools or emit tool markup; emit the required decision immediately.
- Prefer source changes over benchmark-test changes. Remove unrelated test edits or broad refactors from the candidate.
- Run the narrowest available reproduction or focused test before the final decision. Treat unrelated dependency, network, or legacy-suite failures as separate evidence. Never invent a validation result. If no focused command succeeded, say validation was not completed.
- Do not use official Harness reports, hidden tests, gold patches, network services, subagents, or external coding agents.
- Finish with a non-empty minimal Git diff that addresses the complete public issue. A textual review alone is incomplete.
- Your final response must start with exactly
REVIEW_DECISION: ACCEPTif the resulting patch is complete, orREVIEW_DECISION: REVISEif it remains incomplete. Follow it with one short reason and the validation command/result.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

