agentleFS
mindfortaimindfortai/security-skills/skills/retest/SKILL.md

retest

Skill16 starsChanged 5 months agoMITView on GitHub

Retests known findings against the current code using code review and targeted local smoke tests when possible. Use when validating whether reported issues are fixed or still reproducible.

1 other project uses a version of this.

Adopted by

Most recently taken up by

The file

Show its rules and text

We split this skill into the rules it gives an agent

17 rules. About 0 are common, 14 are distinctive, and 3 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈14

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use this skill when the user wants to validate known findings against the current code.

    line 9
  2. Distinctive:

    Start with findings already present in the conversation.

    line 11
  3. Distinctive:

    If key details are missing, ask for the smallest missing set needed to retest.

    line 23
  4. Distinctive:

    Run the narrowest local smoke test that can safely validate the security property.

    line 30
  5. Distinctive:

    Prefer existing tests, local dev servers, test fixtures, and safe read-only probes.

    line 31
  6. Distinctive:

    Do not run destructive, high-volume, production, or third-party tests without explicit approval.

    line 32
  7. Distinctive:

    Keep tests targeted to the finding.

    line 37
  8. Distinctive:

    Use local or test environments only unless the user explicitly authorizes another target.

    line 38
  9. Distinctive:

    Verify both the negative case and, when useful, a positive control.

    line 39
  10. Distinctive:

    Do not treat a passing unit test as proof if it skips the real security control.

    line 40
  11. Distinctive:

    Do not mark a finding fixed from code review alone when a practical local test exists.

    line 41
  12. Distinctive:

    Use temporary stubs only for unavailable third-party services, credentials, nondeterministic providers, or slow dependencies.

    line 42
  13. Very specific:

    When subagents are available, ask an independent subagent to challenge any fixed, still_vulnerable, or high-impact partially_fixed result …

    line 44
  14. Distinctive:

    If local execution is blocked, explain the blocker and provide the exact command or setup needed.

    line 45
  15. Very specific:

    Use status values such as fixed, still_vulnerable, partially_fixed, not_reproducible, blocked, or needs_verification.

    line 55
  16. Very specific:

    Put the overall fixed/still-vulnerable/blocked counts and commands run after the CSV unless the user asks for CSV only.

    line 66
  17. Distinctive:

    Do not overstate what was proven.

    line 72

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Retest

Use this skill when the user wants to validate known findings against the current code.

Start with findings already present in the conversation. If the conversation does not include enough detail, ask the user for the finding list before testing. Do not invent findings.

Inputs

For each finding, identify:

  • Title or claim.
  • Affected feature, endpoint, file, or workflow.
  • Expected vulnerable behavior.
  • Expected fixed behavior.
  • Any provided proof, reproduction steps, test account roles, or payload shape.

If key details are missing, ask for the smallest missing set needed to retest.

Workflow

  1. Build a short retest queue from the current conversation or the user's provided list.
  2. Locate the changed code, relevant tests, routes, handlers, policies, and fixtures.
  3. Review whether the current code appears to remove the vulnerable behavior.
  4. Run the narrowest local smoke test that can safely validate the security property.
  5. Prefer existing tests, local dev servers, test fixtures, and safe read-only probes.
  6. Do not run destructive, high-volume, production, or third-party tests without explicit approval.
  7. Record what was verified, what failed, and what could not be tested locally.

Smoke Test Rules

  • Keep tests targeted to the finding.
  • Use local or test environments only unless the user explicitly authorizes another target.
  • Verify both the negative case and, when useful, a positive control.
  • Do not treat a passing unit test as proof if it skips the real security control.
  • Do not mark a finding fixed from code review alone when a practical local test exists.
  • Use temporary stubs only for unavailable third-party services, credentials, nondeterministic providers, or slow dependencies. Do not stub the security control or vulnerable path being retested.
  • Clean up scratch tests, temporary fixtures, local data, mock servers, config changes, and throwaway scripts before reporting unless the user asked to keep them as regression tests.
  • When subagents are available, ask an independent subagent to challenge any fixed, still_vulnerable, or high-impact partially_fixed result before finalizing it.
  • If local execution is blocked, explain the blocker and provide the exact command or setup needed.

Output Format

Output retest results as CSV with this exact header:

id,severity,confidence,status,category,title,location,asset_or_flow,actor,issue,impact,evidence,fix,test,scope_notes

Use status values such as fixed, still_vulnerable, partially_fixed, not_reproducible, blocked, or needs_verification. Keep the original severity when known; otherwise use the retester's best evidence-based severity.

For retest rows, use:

  • issue: what vulnerable behavior was retested.
  • impact: impact if still vulnerable.
  • evidence: code review facts plus smoke-test result or blocker.
  • fix: remaining remediation if not fixed, or the fix observed if fixed.
  • test: command, local action, or missing setup needed to reproduce.
  • scope_notes: residual risk, confidence limits, and next step.

Put the overall fixed/still-vulnerable/blocked counts and commands run after the CSV unless the user asks for CSV only.

Quality Bar

  • Be evidence-led and concise.
  • Separate code-review confidence from runtime-test confidence.
  • Do not overstate what was proven.
  • Preserve enough command output or file references for another engineer to reproduce the retest.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.