red-team
Skill2 starsChanged 5 months agoMITView on GitHub
Adversarially verify a measurable claim or result to catch self-deception BEFORE trusting it. Use whenever the loop is about to RECORD a metric, benchmark, "it works", or a research/ML result as progress. Refutation-first, independent. Mandatory for empirical/ML claims.
29 other projects use a version of this.
Adopted by
Most recently taken up by
- darkweb19/skillstheir own version, first seen Nov 8, 2026
- kazemcodes/skill-treetheir own version, first seen Nov 8, 2026
- drewmattie-code/red-teamtheir own version, first seen Nov 8, 2026
- Leostruka/devin-bundletheir own version, first seen Nov 1, 2026
- desertcache/agent-ostheir own version, first seen Nov 1, 2026
- fabiandistler/agentstheir own version, first seen Nov 1, 2026
Used with
Projects with it also use retro, architecture and 10 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
AWSSkillretro
Conduct a retrospective on a coding session.
Also in 4 projects that use this
AnthropicSkillarchitecture
Create or evaluate an architecture decision record (ADR). Use when choosing between technologies (e.g., Kafka vs SQS), documenting a design decision with trade-offs and consequences, reviewing a system design proposal, or designing a new component from requirements and constraints.
Also in 3 projects that use this
- AWSSkill
wizard
Generate an interactive bash wizard that walks a human through steps only they can perform. Use when provisioning infrastructure, setting up credentials or CI secrets, walking an unfamiliar third-party dashboard, or running a one-off migration or cutover. Don't invoke this for steps the agent can perform itself.
Also in 3 projects that use this
GoogleSkillcodebase-design
Shared vocabulary for designing deep modules. Use when the user wants to design or improve a module's interface, find deepening opportunities, decide where a seam goes, make code more testable or AI-navigable, or when another skill needs the deep-module vocabulary.
Also in 4 projects that use this
- GoogleSkill
domain-modeling
Build and sharpen a project's domain model. Use when the user wants to pin down domain terminology or a ubiquitous language, record an architectural decision, or when another skill needs to maintain the domain model.
Also in 4 projects that use this
MicrosoftSkillgrill-with-docs
Grilling session that challenges your plan against the existing domain model, sharpens terminology, and updates documentation (CONTEXT.md, ADRs) inline as decisions crystallise. Use when user wants to stress-test a plan against their project's language and documented decisions.
Also in 4 projects that use this
- +3 more from google
Other versions29 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 29
- zakirkun/deep-eye★ 2.3ksame name
- suboss87/FDEOps★ 940same name
- griffinhilly/claude-code-synthesis★ 67same name
- geekatron/jerry★ 37same name
- Hassaan146/claude-skills★ 18same name
- PostHog/duckhog★ 14same name
- docxology/codomyrmex★ 10same name
- Zach-etier/skills★ 4same name
- drewmattie-code/red-team★ 3same name
- csiddhant796-blip/claude-skills-collection★ 2same name
19 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository2 other files it gives its agents.
- architecture-guardSkill
- design-reviewSkill
The file
Files
red-team — refute the result before you believe it
The #1 failure of a long autonomous loop is convincing-looking-but-wrong progress. This pass tries to FALSIFY a claim. Run it — ideally as a SEPARATE agent prompted to refute, not the one who produced the result — before recording any headline claim.
1. State the claim and its falsifier
- The claim, precisely: "X reaches 92% success on task T."
- What would make it FALSE: "fails on unseen seeds / it exploits a sim bug / a trivial baseline matches it."
2. Run the domain self-deception checklist
Empirical / ML / research:
- Re-run on ≥3 fresh seeds — holds with error bars, or one lucky seed?
- Train/eval leakage — is eval data (or its distribution) seen in training?
- Metric ≠ goal — does it ACTUALLY do the task (watch a real rollout), or just minimize a proxy / latent loss?
- Trivial baseline — does random / zero / copy-last match it? If so it's noise.
- Reward hacking / sim exploit — is "success" from gaming the sim, not solving the task?
- Look at FAILURES, not just the average — where and how does it break?
Software / feature:
- Works on inputs NOT in the tests (edge / empty / malformed / large)?
- Did the test exercise the real path, or is it a tautology / mocked away?
- Run it for real (use
verify) — don't trust green tests alone.
Sim / robotics:
- Holds under perturbation / domain randomization?
- Is "success" a sim artifact (clipping, infinite force, frame-perfect timing)?
3. Verdict
confirmed (survives all) · refuted (a check failed → NOT done; fix or downgrade the claim) · uncertain (→ treat as not done; get more evidence).
Record one honest line: "verified: 92% over 5 seeds, beats baseline by X" OR "REFUTED: single-seed, collapses on re-run." Never record a headline claim that hasn't survived this. Residual risk isn't zero — explicitly flag anything you're <80% sure of for the human to eyeball.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.