agentleFS
bigbiobigbio/sdrf-skills/skills/sdrf-adversarial-review/SKILL.md

sdrf-adversarial-review

Skill19 starsChanged 7 days agoMITView on GitHub

Independently falsify and review a created, modified, or proposed SDRF against its specification, ontology terms, source evidence, repository files, and experimental design. Use after SDRF annotation or repair, before contribution or completion, when a review gate reports a pending artifact, or whenever an isolated reviewer must distrust the producer's self-assessment and issue a hash-bound verdict.

We split this skill into the rules it gives an agent

13 rules. About 0 are common, 6 are distinctive, and 7 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈6

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 7

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use only the original task, SDRF, specification, evidence manifest, repository files, and deterministic outputs supplied to the reviewer.

    line 25
  2. Distinctive:

    Do not read or rely on the producer's transcript, reasoning, verdict, or summary.

    line 27
  3. Very specific:

    If context isolation cannot be established, return REVIEW_UNAVAILABLE and do not create a passing receipt.

    line 29
  4. Distinctive:

    Do not edit the SDRF. Return findings to the producer or caller that ran you.

    line 31
  5. Very specific:

    Read only files scoped to this artifact's accession.

    line 32
  6. Very specific:

    Read references/review-contract.md before writing the report or recording approval.

    line 41
  7. Very specific:

    Read spec/sdrf-proteomics/sdrf-templates/templates.yaml and every active template YAML.

    line 57
  8. Distinctive:

    Read the original request and evidence manifest, if supplied.

    line 61
  9. Very specific:

    Run official parse_sdrf validate-sdrf for every active template.

    line 65
  10. Distinctive:

    If the repository tools are not importable from the target project, invoke them from the loaded plugin:

    line 72
  11. Very specific:

    Do not mark deterministic_validation as passing when required validation could not run.

    line 80
  12. Very specific:

    When a caller ran you directly, write the report JSON (PASS or FAIL, same schema) where the caller asked, and run approve only if the …

    line 125
  13. Distinctive:

    After any producer edit, repeat this skill in a new reviewer context.

    line 130

≈ “Common” is an estimate: at least 1 in 5 of the 308,232 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

SDRF Adversarial Review

Bundle paths. spec/, tools/ and data/ ship with this skill, not with your working directory. Resolve every such path below against the bundle root (SDRF_SKILLS_ROOT, else CLAUDE_PLUGIN_ROOT, else your sdrf-skills checkout): ${SDRF_SKILLS_ROOT:-${CLAUDE_PLUGIN_ROOT}}/spec/sdrf-proteomics/TERMS.tsv. The helpers are the sdrf-tools command (install notes: sdrf-annotate/references/setup.md; check with sdrf-tools doctor); no PYTHONPATH or bundle-root variable is needed to run them. Files the user is annotating stay relative to the working directory.

Review the artifact from fresh context. Try to disprove its correctness; do not polish or extend it. Approve only what the artifact and cited evidence support.

The producer usually dispatches this review (sdrf-annotate Step 9.5). A caller can also run it directly, for example as a separate pipeline task. The caller then supplies the original request, the SDRF, the evidence manifest and the deterministic validation output, and receives the findings and report.

Preserve Independence

  1. Confirm that this run is a fresh reviewer, hook-agent or separate-task context.
  2. Use only the original task, SDRF, specification, evidence manifest, repository files, and deterministic outputs supplied to the reviewer.
  3. Do not read or rely on the producer's transcript, reasoning, verdict, or summary. Treat producer claims as untrusted.
  4. If context isolation cannot be established, return REVIEW_UNAVAILABLE and do not create a passing receipt.
  5. Do not edit the SDRF. Return findings to the producer or caller that ran you.
  6. Read only files scoped to this artifact's accession. When reviewers run concurrently, a shared scratchpad with generic filenames (files_all.json, efetch.xml, mmc*.xlsx) silently substitutes one dataset's data for another's — the file still parses, it just describes a different PXD. Read only from scratchpad/<PXD>/ or the evidence paths the caller supplied, and assert on read: every fetched file list's projectAccessions, and every supplementary/efetch result's returned title/accession, must match the artifact's accession before you use it — verify identity, not just HTTP 200.

Read references/review-contract.md before writing the report or recording approval. The report must match tools/schemas/review-report.schema.json; the evidence manifest follows tools/schemas/evidence-manifest.schema.json. sdrf-tools schema-check --print-schema <name> prints either one, and sdrf-tools schema-check <file> checks a file against it.

Review Workflow

1. Freeze the artifact

Compute its SHA-256 and record the repository-relative path. If the hash changes during review, discard the review and start again.

2. Reconstruct requirements

  • Read spec/sdrf-proteomics/TERMS.tsv.
  • Read spec/sdrf-proteomics/sdrf-templates/templates.yaml and every active template YAML.
  • Derive active templates independently; do not accept the producer's list without checking the file.
  • Read the original request and evidence manifest, if supplied.

3. Run deterministic checks

Run official parse_sdrf validate-sdrf for every active template. Also run:

sdrf-tools check <artifact>
sdrf-tools score <artifact>

If the repository tools are not importable from the target project, invoke them from the loaded plugin:

sdrf-tools check <artifact>
sdrf-tools score <artifact>

Do not mark deterministic_validation as passing when required validation could not run.

4. Perform six falsification passes

  1. Specification compliance — missing or extra columns, template/version errors, reserved words, structured-value syntax, row shape, and duplicates.
  2. Ontology integrity — resolve every unique controlled value and confirm label, accession, ontology family, obsolescence, and justified specificity.
  3. Source evidence — challenge every nontrivial biological and technical claim against PRIDE, publication, supplementary data, or raw-file evidence. Flag per-sample values inferred only from cohort summaries.
  4. File mapping — compare every comment[data file] with repository files; look for missing, duplicate, unmatched, or non-raw files.
  5. Design consistency — recompute sample counts, factors, biological and technical replication, fractions, labels, and potential confounders.
  6. Omission safety — search for unsupported additions disguised as specificity and for material evidence the producer omitted.

For each finding, give a precise row/column or file-level location, evidence, and a correction criterion. Classify it as:

  • blocker: invalid, unsafe, fabricated, or structurally unusable.
  • important: required before contribution or completion.
  • minor: useful improvement that does not block approval.

5. Decide and record

Return FAIL if any blocker or important finding remains, or if specification compliance, ontology integrity, omission safety, or deterministic validation is unverified. Do not record a receipt. Give the producer or caller an actionable finding list.

Return PASS only when all required checks in the review contract pass and no blocker or important finding remains. Write the report JSON, then bind approval to the current artifact hash:

python3 "${SDRF_SKILLS_ROOT:-${CLAUDE_PLUGIN_ROOT:?set SDRF_SKILLS_ROOT to the sdrf-skills checkout}}/tools/review_gate.py" approve <artifact> \
  --report <report.json> --reviewer independent-hook-agent --cwd <repo-root>

Outside Claude Code, set SDRF_SKILLS_ROOT to the installed sdrf-skills root. A producer agent must never run approve for its own work.

When a caller ran you directly, write the report JSON (PASS or FAIL, same schema) where the caller asked, and run approve only if the caller asks for it. Never run it outside a git repository: it cannot record a receipt there. The caller checks the verdict and artifact_sha256.

After any producer edit, repeat this skill in a new reviewer context; the old receipt is intentionally stale.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.