agentleFS
DonaldMurilloDonaldMurillo/gofastr/.claude/skills/adversarial-tests/SKILL.md

adversarial-tests

Skill24 starsChanged yesterdayMITView on GitHub

How to run (or commission) an adversarial security-test pass against this codebase without producing 3000 lines of repetitive matrix tests. Auto-loads when the user mentions "red tests", "adversarial tests", "security test pass", "find security gaps", or is about to spawn a sub-agent to author *_security_test.go files. Encodes the "property × surface, not case × file" rule, the naming + triage policy, and a ready-to-paste prompt template for the next pass.

The file

We split this skill into the rules it gives an agent

8 rules. About 0 are common, 6 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈6

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Ask "what surfaces does this property apply to?" before "what attack strings can I generate?".

    line 44
  2. Distinctive:

    When you commission a pass, ask the agent to scan for *each* of these properties across *every* call site, not for "more cases":

    line 49
  3. Distinctive:

    When you commission the next pass, the prompt MUST mention this table and tell the agent to *find new surfaces*, not *new attack strings*.

    line 69
  4. Distinctive:

    Where else does this property apply?: If only one place, it's either a real one-off or wrong-layer.

    line 78
  5. Distinctive:

    Every test the pass *deletes* or *weakens* (rather than fixing production for) records its *why* in exactly two places: the commit message …

    line 116
  6. Distinctive:

    When you want to run another adversarial pass, paste this as the agent's brief (edit the scope line):

    line 176
  7. Very specific:

    After the pass: run ./scripts/test-all.sh and confirm exit 0.

    line 254
  8. Very specific:

    CLEARED: only when the deep pass, the breadth pass, go vet, and govulncheck are ALL dry/clean on it.

    line 304

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Adversarial test pass: how to commission one without drowning

The previous adversarial pass authored 32 _red_test.go files / ~3000 lines, of which ~10 were genuinely unique bugs and ~17 were distinct production fixes; the rest was 60-case matrices testing the same property at different surfaces, plus a handful of wrong- layer tests that contradicted documented contracts.

This skill exists so the next pass produces signal, not volume.

The one rule

Adversarial tests are organized along properties × surfaces, not cases × files.

A "property" is the underlying invariant: no control bytes in outbound headers, every URL field allow-lists schemes, every in-process CRUD method fails closed without tenant context. A "surface" is each place the property must hold (each header writer, each URL field, each method).

The wrong shape (what we got last time):

60 cases × 1 surface per test file, repeated across N files,
where 50 of the cases are noise that test the same byte range.

The right shape:

For each property:
  - 3-5 case shapes (the distinct attack classes, not 60 variants)
  - Asserted at every surface where the property must hold (loop over surfaces, not over cases)
  - One test file per property family, NOT one per surface

A test file with 60 cases is almost always asking the wrong question. Ask "what surfaces does this property apply to?" before "what attack strings can I generate?".

Properties × surfaces this codebase keeps re-encountering

When you commission a pass, ask the agent to scan for each of these properties across every call site, not for "more cases":

PropertyCommon surfacesReference fix
URL scheme allow-list (surface-specific: anchors may allow http(s) / relative / fragment / mailto / tel; head/media/header use narrower policies)any field that flows to <a href>, <img src>, <form action>, <link href>, Link: response header, OpenAPI servers[].url, SEO meta tags, Image/File entity fieldscore-ui/urlsafe/urlsafe.go::Clean / CleanAnchor (with framework/ui/safety.go::safeImageURL), framework/uihost/uihost.go::isSafeHeadURL, framework/crud/crud_upload.go::isSafeMediaURL, framework/experimental/apiversions/version.go::safeReplacementURL
C0 / DEL sanitation on outbound strings (protocol-specific, NOT uniform: SSE scrubSSEDataLines strips CR+NUL only to preserve LF framing; safeLogMethod/safeLogPath fast-path returns clean input unchanged)response header values (Content-Type, Access-Control-*, Link:), log attribute values (method, path), SSE field bodies, DSL opaque literals, route-group prefixcore/textsafe/controlbytes.go::SanitizeControlBytes (called from core/handler/respond.go, core/middleware/cors.go::sanitizeCORSTokens, framework/dsl/dsl.go and framework/routegroup/group.go), core/textsafe/controlbytes.go::ScrubControlBytes (log sinks), core/middleware/logging.go::safeLogMethod (test the fast-path too), core/stream/sse.go::scrubSSEDataLines
Strict JSON top-level parsing (reject duplicate / case-folded / unknown keys); no-op for non-struct-pointer destinations and non-object bodiesevery Bind consumer decoding a top-level JSON object into a struct pointercore/handler/bind.go::validateBodyKeys
Fail-closed scoping (multi-tenant + owner-required) on in-process pathsevery CRUD method that touches DB state (not just the HTTP path; middleware doesn't protect in-process callers)framework/crud/owner.go::requireOwnerContext + framework/crud/owner.go::requireTenantContext + every method in framework/crud/crud_api.go
Forensic completeness on rollback / soft-deletebatch envelope, upsert ON CONFLICT, audit hookframework/crud/crud_batch.go::scrubRolledBackData, framework/crud/crud_upsert.go::errSoftDeletedResurrection
Panic isolation at extension pointsevery place a third-party callback runs (hooks, plugins, custom handlers)framework/hook/hook.go::runHookSafely
Panic-payload scrub at log sinks (a recover() value is attacker-influenced text: control bytes, bidi, secrets)every recover() whose value, or an fmt.Sprint/%v of it, reaches slog/log/fmtcore/textsafe.Recovered
Per-principal seat caps on long-lived streamsevery SSE / WebSocket / Hijack park loop (a2a, rtc, harness rest/ws/mcpserver, kiln events, crud event stream, webmcp)core/stream/seats.go, framework/experimental/harness/control/seats.go
Cache-Control: no-store on per-user 2xx responsesevery handler that resolves a principal and writes a bodybattery/auth writeCredentialHeaders, framework/crud responses, the nostore analyzer
Owner-only file mode on overwrite (a 0600 literal in os.WriteFile applies only at create)every state/secret file written more than onceinternal/fileperm.WriteOwnerOnly / SeedOwnerOnly
Strict decode at every inbound seam (duplicate/case-folded keys at any depth, trailing data)every json.Unmarshal/Decoder.Decode of a client-borne body, frame, cursor, or envelopecore/handler.UnmarshalStrict / DecodeStrict / CheckObjectKeys
Fenced terminal settles (no status regression, no unfenced claim-state write)queue reclaim, webhook inbound status transitions, outbox leasesbattery/queue releaseClaim, battery/webhook terminal fence, GOFASTR1409
Deadlines on probes and childrenevery http.Client construction, every exec.CommandContext with captured outputhygiene clienttimeout, nowaitdelay
Credentials never in argvevery CLI flag that accepts a secretGOFASTR_AUDIT_PASSWORD / GOFASTR_HARNESS_SECRET + the - stdin marker

When you commission the next pass, the prompt MUST mention this table and tell the agent to find new surfaces, not new attack strings.

Triage rubric (apply per test, before writing or accepting)

For every adversarial test the agent proposes:

  1. What property does this assert? State it in one sentence. If you can't, the test is wrong-shaped.
  2. Where else does this property apply? If only one place, it's either a real one-off or wrong-layer. If many, the agent should be testing every surface, not 60 cases on one.
  3. Is this contradicting a documented contract? Grep for the nearest sibling _security_test.go and any doc comment on the exercised function. If yes → flag for human (this is where the "flip the contract" vs "delete the test" decision lives).
  4. Is the test asserting on developer input or attacker input? Developer-supplied configuration (OpenAPI path keys, server URLs passed by the host app, etc.) has a different threat model than request-borne input. Tests that treat developer input as attacker input are wrong-layer → delete.
  5. Cap case count at ~5 per property. 1 happy path + 3-4 distinct attack shapes (e.g. CR, LF, NUL, DEL, not 60 random control bytes). If you need 60 to feel covered, you're confusing coverage with count.

Naming + location policy

The policy:

  • An open finding is a *_red_test.go probe tagged //go:build red that asserts the secure behaviour and fails until the fix lands (make red-tests runs the probes in packages your change affects, GOFASTR_TEST_ALL=1 make red-tests runs every probe; go test ./... never sees them). The fix turns the probe into a permanent _security_test.go test.
  • Permanent file suffix: _security_test.go, with no _red_ infix
  • Drop the redundant <pkg>_ prefix when the file lives in a package of that name
  • Function names ≤ 40 chars stating the behaviour (NOT TestFoo_RedRejectsBarBlahMatrix essays)
  • One file per property family, not one per case shape
  • Merge into the closest existing _security_test.go sibling unless the topic is genuinely new

Record decisions where they're enforced, not in a ledger

Every test the pass deletes or weakens (rather than fixing production for) records its why in exactly two places: the commit message that deletes/weakens it, and, when a surviving sibling test carries the contract, a short comment on that sibling naming what it supersedes. No separate ledger file: the pinning *_security_test.go IS the record, and git log -p is the audit trail. A permanently t.Skipped test is never acceptable. Delete it and leave a comment where the contract is actually tested.

Where to start a pass

The list below is the 2026-07-25 pass's own never-looked list: no pass had run all four signals over these surfaces at that date. Later passes have covered some of them (core/upload and battery/auth OIDC now carry _security_test.go files), so check each entry for an existing _security_test.go before treating it as never-looked, and delete the entries a pass clears. A package that carries _security_test.go siblings has been looked at for the properties those tests name, not for every property in the table above.

  • Auth: battery/auth/oidc.go + oidc_jwks.go (1000+ lines: JWKS fetch, iss/aud, nonce, email_verified; oidc allows http for a localhost issuer, so check a host that merely resolves to loopback), entity_oauth_links.go, oauth_token_store.go, battery/admin entity-screen handler bodies (gating verified, internals not), framework/access/store.go fanout/refresh race, the decide chain.
  • Data: core/upload/, framework/{file,image,openapi,sdk,sdkdocs, tx.go,hook,typed_hooks}, framework/{db,pagination,datexport}, core/schema beyond the Pattern note, crud_batch/stream/events. The sqlite/** in-house engine was REMOVED from the tree (2026-09 round-5 recon confirmed: only the stdlib driver alias + DSN compat shim remain, sibling-tested); the untrusted-.db robustness surface now lives in modernc.org/sqlite — out of repo scope.
  • Agent surface: framework/experimental, kiln/{expr,effect,render} CSP, cmd/gofastr/internal/{packcmd,agentscmd,docscmd}, core/moduleproto/{peer,handshake,methods}.go, framework/agentsinv.
  • Browser: most core-ui/runtime/src/*.js (lightbox, sortablelist, panehost, carousel, combobox, tree, toasts, …), the former core-ui/patterns/** (deleted; its replacements live in framework/ui + framework/headless), core-ui/{seo,app,registry,di,compute,interactive}, framework/ui (100+ files, only the URL-guard sites were read), core/fuzzy.
  • Infra: framework/migrate lock/checksum/dirty + tenant_id injection internals, framework/{cron,event,outbox,fanout,lifecycle} leasing SQL / relay state machine, core/{router,config,featureflag, i18n} + framework/i18n.go (locale-tag → file lookup traversal), core/handler/{bind,respond,context} deep pass, battery/queue scheduler.

Surfaces that pass CLEARED the same day, so do not re-derive them: kiln journal replay, core/markdown XSS, the service worker's fetch(originalRequest) (the shape the 2026 Angular-SW CVE class needs you to get wrong), sandbox _other.go failing closed, QuoteIdent/SafeIdent, webhook delivery + harness/webfetch.go dial-time Control re-check, core/yaml, core/moduleproto framing, CORS, the XFF rate-limit key, static path traversal, the Timeout middleware race, log injection, cache cross-user replay, kiln request→exec, and the codegen file-write sink.

Ready-to-paste prompt for the next pass

When you want to run another adversarial pass, paste this as the agent's brief (edit the scope line):

You are running an adversarial security test pass against the
GoFastr codebase. Read
`.claude/skills/adversarial-tests/SKILL.md` first; it
encodes the policy you must follow. Prior keep/flip/delete decisions
live in git history and in comments beside the surviving tests.

Scope: <e.g. framework/crud + framework/uihost; or "any package
touched on this branch">

Rules:
- Organize by **property × surface**, NOT case × file. For each
  security property you find, scan EVERY surface where it applies
  and assert at each one, not 60 cases at one surface.
- Cap case count at ~5 per property: one happy path + three or four
  distinct attack shapes covering the threat class. No 60-row
  matrices.
- Naming: an open finding is a `_red_test.go` probe behind the `red`
  build tag; its fix converts it to a `_security_test.go` test.
  Function names ≤40 chars stating the behaviour.
- Before authoring a test, grep for the closest existing
  `_security_test.go` sibling. If one exists, merge into it
  unless the topic is genuinely new.
- Before authoring a test that contradicts a documented escape
  hatch or behaviour comment, FLAG it for the user instead of
  silently asserting the opposite. The doc comment is the
  contract until the user explicitly flips it.
- Tests that treat developer-supplied input (OpenAPI path keys,
  server URLs from host app config, etc.) as attacker input are
  wrong-layer; do not author them.
- For every test that the pass deletes or weakens, put the *why* in
  the commit message and a short comment beside the surviving sibling
  test. Never leave a permanently-skipped test.

For each finding, produce:
- The underlying property (one sentence)
- Every surface where the property applies
- The production fix (or `weaken` / `delete` decision)
- The new short test name

Pre-existing properties to extend coverage on (don't re-derive; pin
to known surfaces only). See the table in
`.claude/skills/adversarial-tests/SKILL.md` for symbol references:
- URL scheme allow-list
- C0 / DEL strip on outbound strings
- Strict JSON top-level parsing
- Fail-closed multi-tenant / owner scoping on in-process paths
- Forensic completeness on rollback / soft-delete
- Panic isolation at extension points
- Panic-payload scrub at log sinks (textsafe.Recovered)
- Per-principal seat caps on long-lived streams
- no-store on per-user 2xx responses
- Owner-only file mode on overwrite (fileperm.WriteOwnerOnly)
- Strict decode at every inbound seam (any-depth key walk)
- Fenced terminal settles (queue/webhook/outbox)
- Deadlines on every HTTP client and exec child
- Credentials never in argv (env/stdin channels)

Hard NO:
- No `_red_test.go` file left behind once its finding is fixed
- No 60-case matrices
- No Module_Feature_Precondition_Assertion essay test names
- No silent contradictions of documented contracts

How to invoke

Two ways:

  1. Paste the prompt block above into a sub-agent task (this is the common case; adversarial passes are heavy enough to warrant their own agent context).
  2. Drive the pass yourself by following the rules in this file. The prompt block IS the policy, just internalized.

After the pass: run ./scripts/test-all.sh and confirm exit 0; audit find . -maxdepth 3 -type f -size +500k … for stray binaries per AGENTS.md; review the pass's commit messages for the keep/flip/delete rationale.

Two-pass protocol (MANDATORY)

A security pass is run by two passes with different jobs, never one. This is not optional: a single-pass audit does not get to mark anything clean.

Both profiles run on the same model: each pins model: opus in its frontmatter. A breadth pass on a cheaper tier is not acceptable, because a breadth pass whose silence means nothing cannot be half of a clean-gate.

RoleProfileModelJob
breadthsec-reconopuswalk every file against the property×surface checklist, emit all candidates, report coverage honestly
depthsec-auditoropusthreat-intel anchor → deep discovery (authz/TOCTOU/state-machine) → refute + fix + rule on every candidate

Why two passes: exhaustive breadth and deep reasoning are different jobs; one agent doing both drops coverage the moment a thread gets interesting. The split is job, not capability.

Honest limit. With both halves on the same model, the two passes share a lineage and a tier: their blind spots overlap almost completely. Agreement between them is weak evidence, and one confirming the other is not independent verification. Say so out loud rather than banking it. The diversity that does not come from a model is therefore load-bearing; it is the only diversity in the pass:

  • Web search (sec-auditor job 1) injects vuln classes from the live CVE/advisory corpus that no Claude tier would spontaneously recall. Anchor every pass to current CWE Top 25 / OWASP ASVS / recent Go + primitive-specific advisories, and turn the result into a checklist delta both profiles sweep.
  • Deterministic tools: go vet and govulncheck (both first-party Go-team; no third-party analyzers) are the one second opinion that shares no blind spot with any LLM. govulncheck is a separate install (golang.org/x/vuln/cmd/govulncheck); run it via make vulncheck, which fails closed with the install command if the binary is missing.
  • Proof: a failing test, a curl, a real browser. Evidence is not a second opinion, it is the end of the argument. Prefer it to every model judgment, including your own.

The clean-gate (the non-optional invariant): a surface is CLEARED only when the deep pass, the breadth pass, go vet, and govulncheck are ALL dry/clean on it. One signal's silence never clears a surface. A category nobody ran all four against is "never-looked", not "clean". Track the difference, and publish the not-looked list alongside the findings.

Cross-verify both directions: recon finds → auditor refutes (kills false positives); auditor finds → prove it, then run the deterministic tools where analyzable (catches over-reads). A same-model re-derivation is a sanity check, never a promotion.

Spawn the two profiles with the Agent tool (subagent_type: sec-recon / sec-auditor; the model: is pinned in each profile's frontmatter) or drive them as stages of a Workflow. When running several scopes at once, give each its own scope boundary so the passes don't collide.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.