agentleFS
Sign inStart free
joshuaporthjoshuaporth/appsec-skill/.cursor/skills/benchmark/SKILL.md

benchmark

Skill0 starsChanged 6 months agoMITView on GitHub

Runs the full 30-challenge secure-code-review benchmark end-to-end via scripts: Findings (parallel across 01-30), then Scoring (parallel across 01-30), then scoreboard generation from frozen scoring artifacts. Invoke from Cursor Agent with /benchmark.

95 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use ios-clean, ios-fix and 10 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • Saqlain-811Skill

    ios-clean

    Remove the DebugBridge SPM package and all #if DEBUG wiring from an iOS app. (gstack)

    Also in 5 projects that use this

  • Saqlain-811Skill

    ios-fix

    Autonomous iOS bug fixer. (gstack)

    Also in 5 projects that use this

  • seek888Skill

    pair-agent

    Pair a remote AI agent with your browser. One command generates a setup key and prints instructions the other agent can follow to connect. Works with OpenClaw, Hermes, Codex, Cursor, or any agent that can make HTTP requests. The remote agent gets its own tab with scoped access (read+write by default, admin on request). Use when asked to "pair agent", "connect agent", "share browser", "remote browser", "let another agent use my browser", or "give browser access". (gstack) Voice triggers (speech-to-text aliases): "pair agent", "connect agent", "share my browser", "remote browser access".

    Also in 11 projects that use this

  • ShreyRaviSkill

    landing-report

    Read-only queue dashboard for workspace-aware ship. Shows which VERSION slots are currently claimed by open PRs, which sibling Conductor workspaces have WIP work likely to ship soon, and what slot /ship would pick next. No mutations — just a snapshot. Use when asked to "landing report", "what's in the queue", "show me open PRs", or "which version do I claim next". (gstack)

    Also in 8 projects that use this

  • timurgaleevSkill

    document-generate

    Generate missing docs for a feature, module or project as tutorials, how-tos, reference and explanation.

    Also in 7 projects that use this

  • seek888Skill

    open-gstack-browser

    Launch GStack Browser — AI-controlled Chromium with the sidebar extension baked in. Opens a visible browser window where you can watch every action in real time. The sidebar shows a live activity feed and chat. Anti-bot stealth built in. Use when asked to "open gstack browser", "launch browser", "connect chrome", "open chrome", "real browser", "launch chrome", "side panel", or "control my browser". Voice triggers (speech-to-text aliases): "show me the browser".

    Also in 8 projects that use this

  • +1 more from Saqlain-811
  • Other versions95 projects use a version of this.

    Word for word · 0

    None found.

    Edited · 0

    None found.

    Wrote their own version · 95

    1. garrytan/gstack★ 141ksame name
    2. dragonflydb/dragonfly★ 32ksame name
    3. xtermjs/xterm.js★ 21ksame name
    4. udecode/plate★ 17ksame name
    5. Z3Prover/z3★ 13ksame name
    6. androidx/androidx★ 6.1ksame name
    7. samchon/typia★ 6ksame name
    8. PaddlePaddle/FastDeploy★ 3.7ksame name
    9. zalando/skipper★ 3.3ksame name
    10. samchon/nestia★ 2.2ksame name

    85 more not listed

    How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.

  • In this repository3 other files it gives its agents.

The file

Show its rules and text

We split this skill into the rules it gives an agent

13 rules. 0 are common, 7 are distinctive, and 6 are very specific to this project.

  • 0

    Common

    Most of the 81 edited versions we read keep these.

  • 7

    Distinctive

    This author's own choices. Worth a look.

  • 6

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Run exactly one non-interactive workflow for challenges 01..30:

    line 13
  2. Very specific:

    Ensure benchmark/challenges/challenge-01 ..

    line 25
  3. Very specific:

    Ensure benchmark/findings.sh and benchmark/scoring.sh exist and are executable.

    line 26
  4. Very specific:

    Ensure claude is available on PATH (or set CLAUDE=/path/to/claude).

    line 27
  5. Distinctive:

    If any precondition fails, stop and report an abort block (no partial run output).

    line 29
  6. Distinctive:

    Wait for Findings completion and verify artifacts.

    line 36
  7. Distinctive:

    Wait for Scoring completion and verify artifacts.

    line 38
  8. Very specific:

    Run from repo root. CLI flags only (see ./benchmark/findings.sh --help):

    line 43
  9. Distinctive:

    Do not substitute with ad-hoc per-challenge Task orchestration for this skill.

    line 71
  10. Distinctive:

    If missing or malformed outputs are detected, abort before Scoring.

    line 78
  11. Distinctive:

    If missing or malformed outputs are detected, abort before scoreboard generation.

    line 83
  12. Distinctive:

    After successful scoring completeness:

    line 87
  13. Very specific:

    Use the scoreboard skill (.cursor/skills/scoreboard/SKILL.md) for rendering rules (table, glyphs, Pulse, ordering).

    line 91

“Common” is measured: kept by at least 6 in 10 of the 81 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Benchmark (scripted full pipeline)

Run exactly one non-interactive workflow for challenges 01..30:

  1. Findings pass on all 30 challenges in parallel.
  2. Scoring pass on all 30 challenges in parallel (after Findings finishes).
  3. Scoreboard generation from scoring artifacts only.

Challenges 31 and 32 are out of scope.

Preconditions (fail closed)

Before dispatch:

  • Ensure benchmark/challenges/challenge-01 .. benchmark/challenges/challenge-30 exist.
  • Ensure benchmark/findings.sh and benchmark/scoring.sh exist and are executable.
  • Ensure claude is available on PATH (or set CLAUDE=/path/to/claude).

If any precondition fails, stop and report an abort block (no partial run output).

Required execution order

Never interleave phases:

  1. Findings script for all 01..30 (parallel workers).
  2. Wait for Findings completion and verify artifacts.
  3. Scoring script for all 01..30 (parallel workers).
  4. Wait for Scoring completion and verify artifacts.
  5. Generate scoreboard from benchmark/artifacts/scoring/*.txt only.

Script commands (canonical)

Run from repo root. CLI flags only (see ./benchmark/findings.sh --help):

FlagDefault
--start1
--end30
--parallel30
--modelClaude default if omitted
--traceoff (debug: writes NN.trace.log under artifacts)

Full run (01..30, 30 workers):

./benchmark/findings.sh --model sonnet
./benchmark/scoring.sh --model sonnet

Subset or retry (example: challenge 15 only, serial):

./benchmark/findings.sh --start 15 --end 15 --parallel 1 --model sonnet
./benchmark/scoring.sh --start 15 --end 15 --parallel 1 --model sonnet

Notes:

  • Findings must fully finish before Scoring starts.
  • Optional env: CLAUDE=/path/to/claude if the binary is not on PATH.
  • Do not substitute with ad-hoc per-challenge Task orchestration for this skill.

Artifact completeness gates

After Findings:

  • Require exactly 30 findings files: benchmark/artifacts/findings/01.txt .. 30.txt.
  • If missing or malformed outputs are detected, abort before Scoring.

After Scoring:

  • Require exactly 30 scoring files: benchmark/artifacts/scoring/01.txt .. 30.txt.
  • If missing or malformed outputs are detected, abort before scoreboard generation.

Scoreboard step

After successful scoring completeness:

  • Generate the canonical benchmark scoreboard from benchmark/artifacts/scoring/*.txt only.
  • Use the scoreboard skill (.cursor/skills/scoreboard/SKILL.md) for rendering rules (table, glyphs, Pulse, ordering).

Non-interactive run requirement

/benchmark should run continuously through all phases without midpoint prompts, unless blocked by an external dependency (permissions/runtime/tool failure).

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free