agentleFS
ArcanEdge-AIArcanEdge-AI/coding-agent-playbook-codex/skills/production-testing-qa/SKILL.md

production-testing-qa

Skill3 starsChanged 3 months agoMITView on GitHub

Audit and improve production QA readiness through complete end-to-end flows, existing required repository checks, critical user journeys, realistic failure and boundary coverage, flaky or skipped tests, manual QA, and honest validation summaries. Use for production-readiness assessments and authorized test improvements.

1 other project uses a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

9 rules. About 0 are common, 9 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈9

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Avoid writing brittle tests that only assert implementation details.

    line 48
  2. Distinctive:

    Run the narrowest useful test command after changes.

    line 49
  3. Distinctive:

    Run broader validation before final summary when practical.

    line 50
  4. Distinctive:

    Do not delete failing tests unless they are clearly obsolete and documented.

    line 87
  5. Distinctive:

    Do not weaken assertions to force passing tests.

    line 88
  6. Distinctive:

    Do not mock away the behavior being tested.

    line 89
  7. Distinctive:

    Do not claim coverage exists without evidence.

    line 90
  8. Distinctive:

    Prefer meaningful coverage over high coverage percentages.

    line 91
  9. Distinctive:

    Document unavailable test tooling honestly.

    line 92

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Production Testing QA

Core purpose

Identify behavioral coverage gaps, add practical E2E tests for high-risk areas when authorized, and document validation status. Keep assessment-only work read-only.

Use this skill when

  • The production pass needs testing and QA review.
  • The user wants confidence that the codebase is safe to release.
  • Critical user flows need automated tests or manual QA steps.

Do not use this skill when

  • The task is only code cleanup.
  • The task is only deployment configuration.
  • The user does not want tests added or run.

Check for

  • E2E coverage of important business rules through supported UI, API, or CLI entry points
  • Complete API, service, and database flows with relevant dependencies and observable outcomes
  • End-to-end tests for critical user journeys
  • Broken tests
  • Skipped tests
  • Flaky tests
  • Missing test setup documentation
  • Untested authentication flows
  • Untested authorization rules
  • Untested error states
  • Untested edge cases
  • Missing regression coverage for core behavior

Default workflow

Resolve the task mode first: assessment-only work reports findings without source, test, or repository-document edits; authorized remediation applies only in-scope fixes. Release, production, and destructive operations retain their separate authority requirements.

  1. Identify test framework and commands.
  2. Inspect existing test structure.
  3. Identify critical user flows from routes, UI, API, docs, or product structure.
  4. Classify test gaps as Blocker, High, Medium, or Low.
  5. For authorized test changes or remediation, add practical E2E tests for Blocker and High-risk areas. In assessment mode, run only authorized existing checks and report missing coverage without adding files.
  6. Avoid writing brittle tests that only assert implementation details.
  7. Run the narrowest useful test command after changes.
  8. Run broader validation before final summary when practical.
  9. Report validation results in the requested deliverable; update TESTING_SUMMARY.md only when that artifact is in scope.

Critical flow heuristic

Prioritize test coverage for flows involving:

  • Signup
  • Login
  • Logout
  • Password reset
  • Account settings
  • Billing or payment
  • Data creation
  • Data editing
  • Data deletion
  • Permissions
  • Admin actions
  • File uploads
  • External integrations
  • Webhooks
  • Deployment-critical configuration

Severity guidance

Classify as Blocker when tests reveal failing critical flows, failing builds, or release-blocking validation failures.

Classify as High when critical flows lack coverage and would be expensive, risky, or embarrassing to break in production.

Classify as Medium when useful coverage is missing for non-critical flows.

Classify as Low for nice-to-have coverage or test cleanup.

Guardrails

  • Retain E2E behavioral coverage only. Every created unit test is temporary, regardless of purpose; remove it and exclusively used support code after its still-required assertions, meaningful failures, and boundaries are covered by passing E2E tests.
  • Preserve existing mandatory repository suites until migration is authorized and replacement coverage passes. Do not bulk-delete pre-existing unit or isolated integration tests. Build, lint, types, and static validation remain useful checks.
  • Do not delete failing tests unless they are clearly obsolete and documented.
  • Do not weaken assertions to force passing tests.
  • Do not mock away the behavior being tested.
  • Do not claim coverage exists without evidence.
  • Prefer meaningful coverage over high coverage percentages.
  • Document unavailable test tooling honestly.

Deliverables

Include the following in the requested assessment response or report. Update repository readiness files only when those artifacts are requested or part of authorized remediation:

  • Test coverage findings
  • Tests added or updated
  • Commands run and results
  • Manual QA checklist
  • Remaining test gaps
  • Files changed and reasons

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.