agentleFS
yaokai123yaokai123/AgentPaw/skills/swebench-reviewer/SKILL.md

swebench-reviewer

Skill1 starsChanged 3 months agoApache-2.0View on GitHub

Independently review and correct one existing SWE-bench candidate patch before official evaluation.

The file

We split this skill into the rules it gives an agent

9 rules. About 0 are common, 9 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈9

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Start with git diff and the public issue.

    line 13
  2. Distinctive:

    Use the deterministic public-contract coverage facts to account for every named setting, API, class, function, and template tag.

    line 15
  3. Distinctive:

    Check adjacent behavior explicitly: equivalent entry points, custom names or identifiers, ordering and filtering semantics, and error …

    line 17
  4. Distinctive:

    Treat reconstruction completeness as an invariant: deconstruction must preserve non-default values needed to recreate the configured …

    line 20
  5. Distinctive:

    Treat migrations and newly added destructive data operations as high risk.

    line 21
  6. Distinctive:

    Prefer source changes over benchmark-test changes.

    line 25
  7. Distinctive:

    Run the narrowest available reproduction or focused test before the final decision.

    line 26
  8. Distinctive:

    Never invent a validation result.

    line 27
  9. Distinctive:

    Do not use official Harness reports, hidden tests, gold patches, network services, subagents, or external coding agents.

    line 28

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

SWE-bench Patch Reviewer

Work only inside the provided repository. A candidate patch already exists.

  1. Start with git diff and the public issue. Do not repeat broad repository exploration.
  2. Identify the exact public behavior contract and confirm that the edited module owns it. Use the deterministic public-contract coverage facts to account for every named setting, API, class, function, and template tag. A cue may be implemented indirectly only when the changed abstraction demonstrably governs it. Reject a template, presentation, transport, or other downstream-only patch when the issue requires direct behavior from a shared setting, storage, configuration, or API surface.
  3. Check adjacent behavior explicitly: equivalent entry points, custom names or identifiers, ordering and filtering semantics, and error type/message when relevant. For empty-input, filtering, broadcasting, or shape-sensitive fixes, cover an edge matrix: all inputs empty, each input empty independently, mixed empty/non-empty inputs, singleton inputs, and output shape/dtype preservation. For callable, lazy, deferred, or environment-dependent configuration, verify the lifecycle separately: constructor storage, serialization/deconstruction, and the runtime consumer. Do not assume serialization evaluates a callable unless the code proves it. Treat reconstruction completeness as an invariant: deconstruction must preserve non-default values needed to recreate the configured object, including importable callables. Treat migrations and newly added destructive data operations as high risk. Do not accept a patch that avoids an integrity error by deleting existing rows unless the public contract, forward/reverse semantics, idempotency, and focused validation prove that deletion is intended.
  4. The runtime assigns a 6, 8, or 12 turn budget from patch complexity. Finish investigation before the final three turns.
  5. During the two turns before the final turn, only edit_file, write_file, or execute_shell is permitted. Use shell only for a focused validation command, never for more searching or file reading.
  6. The final budgeted turn is tool-free. Do not request tools or emit tool markup; emit the required decision immediately.
  7. Prefer source changes over benchmark-test changes. Remove unrelated test edits or broad refactors from the candidate.
  8. Run the narrowest available reproduction or focused test before the final decision. Treat unrelated dependency, network, or legacy-suite failures as separate evidence. Never invent a validation result. If no focused command succeeded, say validation was not completed.
  9. Do not use official Harness reports, hidden tests, gold patches, network services, subagents, or external coding agents.
  10. Finish with a non-empty minimal Git diff that addresses the complete public issue. A textual review alone is incomplete.
  11. Your final response must start with exactly REVIEW_DECISION: ACCEPT if the resulting patch is complete, or REVIEW_DECISION: REVISE if it remains incomplete. Follow it with one short reason and the validation command/result.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.