agentleFS
ToddGBensonToddGBenson/keel/.claude/skills/secure-code-review/SKILL.md

secure-code-review

Skill0 starsChanged 3 months agoApache-2.0View on GitHub

Review a pull request or diff for correctness, security, failure behavior, and test quality — including the specific failure patterns of AI-authored code. Use when reviewing a PR, running the G3 gate, checking someone else's implementation, or when asked whether a change is safe to merge. Covers calibrating against automation bias.

17 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use vulnerability-triage, competitive-analysis and 1 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • superagent-aiSkill

    vulnerability-triage

    Triage inbound vulnerability reports - GitHub Advisories (GHSA/CVE), bug bounty submissions, HackerOne/Bugcrowd/Intigriti exports, or a researcher's issue - to decide whether a finding is real, by-design, or noise. Reads the report offline, cross-references the project's documented intent and threat model (SECURITY.md, README, code, closed issues), statically audits any PoC without executing it, and emits a structured markdown triage report with a P0-P3 / By-Design severity and a recommended action. Use whenever the user asks to triage, assess, or score a vulnerability report, advisory, CVE, GHSA, or bug bounty submission, or asks "is this a real finding or by-design?".

    Also in 3 projects that use this

  • AnthropicSkill

    competitive-analysis

    Framework for building competitive landscape decks — market positioning, competitor deep-dives, comparative analysis, strategic synthesis. Use when the user asks for a competitive landscape, competitor analysis, peer comparison, market positioning assessment, strategic review, or investment memo deck. Also triggers on "who are the competitors to X", "benchmark X against peers", "build a market map", or any request to systematically evaluate competitive dynamics across an industry.

    Also in 3 projects that use this

  • AnthropicSkill

    incident-response

    Run an incident response workflow — triage, communicate, and write postmortem. Trigger with "we have an incident", "production is down", an alert that needs severity assessment, a status update mid-incident, or when writing a blameless postmortem after resolution.

    Also in 3 projects that use this

The file

We split this skill into the rules it gives an agent

4 rules. 0 are common, 3 are distinctive, and 1 is very specific to this project.

  • 0

    Common

    Most of the 16 edited versions we read keep these.

  • 3

    Distinctive

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Read the AC list. For each one, find the code that satisfies it and the test that proves it.

    line 29
  2. Distinctive:

    Authorization on the object, on every path, server-side: Not on the route. Not in the UI.

    line 37
  3. Distinctive:

    Two requests race: Read-then-write without a lock.

    line 59
  4. Very specific:

    If a plausible bug leaves them green, they are decorative — and worse than absent, because they inflate coverage and buy false confidence.

    line 68

“Common” is measured: kept by at least 6 in 10 of the 16 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Secure code review

Consumers: developer (as reviewer), /review, process/gates/g3-code-complete.md.

Independence first

Review is performed by an identity that is not the author. Spawn a fresh agent invocation that has not seen the implementation reasoning — there are no fresh eyes inside the same reasoning chain, and that correlation is precisely what the control exists to break (AC-5, AIC-2).

Asking the implementing agent to "review its own work carefully" produces a review anchored on the reasoning that created the bug.

Not style

The linter handles formatting. A review spent on naming and whitespace is a review that did not look for the logic error, and both parties leave feeling productive.

Review in this order — it is ordered by how expensive the miss is:

1. Correctness against the acceptance criteria

Read the AC list. For each one, find the code that satisfies it and the test that proves it. Then ask what the AC does not say, and check what the code does there.

Most correctness bugs are not wrong logic. They are unconsidered cases: empty collection, single element, duplicate, unicode, negative, zero, null, the maximum, the maximum plus one.

2. Security

  • Authorization on the object, on every path, server-side. Not on the route. Not in the UI. Trace the alternates: batch endpoint, export, admin route, retry, webhook, the GraphQL resolver someone forgot.
  • Validate at the boundary, encode at the sink. Both, in different places. Validation is not escaping.
  • Every query parameterised — including in tests and migrations, because that is where the copy-paste pattern gets learned.
  • Fails closed. Find the catch blocks. What does the code do when the authorizer throws, the token fails to parse, the store is unreachable? catch { return allowed } is how breaches begin.
  • No secrets, no personal data in logs.
  • Errors say what to do, not how the system is built.
  • Every control the threat model allocated is present — point at the line. If you cannot find it, it is not implemented.

3. Failure behavior

The interesting bugs live here, and tests rarely reach them.

  • What if the dependency is slow rather than down? That path is usually untested and usually worse than the outage path.
  • What happens on partial failure — half the batch written?
  • Two requests race. Read-then-write without a lock. Check-then-act. Non-transactional quota and uniqueness checks.
  • Retry: is the operation idempotent? Does the retry amplify?
  • Timeout: is there one? Is it shorter than the caller's?

4. Test quality

Mentally delete the implementation. Which tests fail?

If a plausible bug leaves them green, they are decorative — and worse than absent, because they inflate coverage and buy false confidence. Detail in the test-strategy skill.

Confirm specifically that every allocated security control has a negative-case test proving it denies. The positive case proves the feature.

5. Maintainability

Will the next person understand it? Is the abstraction earning its cost? Premature generalisation is the most common form of damage a good developer does.


Reviewing AI-authored code specifically (AIC-9)

Agent output is usually right. That is exactly what makes complacent review dangerous — the base rate trains you to skim, and the failures do not look like failures.

Verify claims rather than reading them. The PR says a control is implemented: go find the line. The PR says the tests cover the error path: open the test.

The plausible-but-wrong catalogue

What machine-generated code gets wrong, ranked by how often it survives review:

PatternWhat it looks like
Inverted conditionif (user.isAdmin || resource.isPublic) where it should be && — reads naturally, fails open
Off-by-one at a boundary<= where < belongs; correct-looking, wrong at exactly the limit
Authorization on the routeA clean-looking middleware check that never validates object ownership
Swallowed exceptionexcept Exception: pass with a plausible comment about resilience
Non-transactional check-then-actReads a count, then writes. Correct single-threaded, wrong under load
Confident hallucinated APIA method that does not exist, or exists with different semantics — compiles in a dynamic language, fails at runtime
Hallucinated dependencyA plausible package name an attacker has registered. See dependency-vetting
Tests mirroring the bugTests written from the implementation encode the same misunderstanding and pass
Over-broad error messageInternal identifiers or stack detail in a user-facing string
Plausible constantA timeout, limit, or retry count that sounds reasonable and matches nothing real

State what you verified personally

"LGTM" on agent-authored code is a control failure. The PR template asks for this; mean it:

Traced AC-1..3 to tests and confirmed each goes red with the implementation stubbed. Confirmed DraftAuthorizer:47 checks object ownership, and that the export path at export.py:23 also routes through it. Did not verify the performance NFR.

Approving code you do not understand is a control failure, not politeness. "I don't understand this — explain it or simplify it" is the correct professional response, and it is usually the most valuable comment in the review.


Writing the comment

Specific, reproducible, and about the code:

drafts.py:88 — the quota check reads then writes outside a transaction. Two concurrent autosaves at quota−1 both pass. Reproduce: parallel POST at 4 of 5. Suggest a row lock or a DB constraint.

Not: "this might have a race condition."

The three outcomes

Approve · Request changes — with the specific defect and how to reproduce it · Ask a question — underused, and frequently the most valuable. A question costs the author two minutes and often surfaces that they were unsure too.

Size

Past ~400 changed lines reviewers degrade to skimming and everyone involved knows it. Ask for a split rather than performing a review you know is shallow.

Controls

SA-11 · SA-11(1) · AC-5 (separation of duties) · CM-5 · AC-3 · SI-10 · AIC-2 · AIC-7 · AIC-9.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.