agentleFS
Sign inStart free
EmeaAppGbbEmeaAppGbb/spec2cloud/.github/skills/test-runner/SKILL.md

test-runner

Skill100 starsChanged 7 months agoMITView on GitHub

Execute the appropriate test suite (unit, Gherkin, e2e, smoke) and return structured results. Use during Phase 3 (e2e test verification), Phase 4 (red baseline verification), Phase 5 (contract type compilation), Phase 6 (API/Web/integration slices), Phase 7 (smoke tests against deployment), and on resume (re-validate test state). Trigger when running tests, checking test status, or verifying test baselines.

We split this skill into the rules it gives an agent

7 rules. 0 are common, 4 are distinctive, and 3 are very specific to this project.

  • 0

    Common

    Most of the 17 edited versions we read keep these.

  • 4

    Distinctive

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Use aspire describe --format Json (or the Aspire MCP list_resources tool) to check resource status

    line 23
  2. Very specific:

    If resources are not healthy, run aspire start + aspire wait api --status healthy + aspire wait web --status healthy

    line 24
  3. Very specific:

    If resources show errors, use aspire logs <resource> (or list_console_logs) to diagnose

    line 25
  4. Distinctive:

    When tests fail against the Aspire environment:

    line 52
  5. Distinctive:

    Test runner itself fails (not assertions) → report as infrastructure failure

    line 64
  6. Distinctive:

    Always capture both stdout and stderr

    line 67
  7. Distinctive:

    BLOCKING: If the test runner itself fails (infrastructure failure, timeout, partial execution), this must be reported as "run incomplete" — …

    line 79

“Common” is measured: kept by at least 6 in 10 of the 17 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Test Runner

Execute tests and return structured results for the orchestrator.

Test Commands

TypeCommand
Unit (TypeScript)cd src/api && npm test
Gherkinnpx cucumber-js
E2Enpx playwright test --config=e2e/playwright.config.ts
Smokenpx playwright test --grep @smoke
Allnpm run test:all

Steps

  1. Verify Aspire environment (for Gherkin/e2e/smoke/all) — Before running integration tests, ensure the Aspire environment is running:
    • Use aspire describe --format Json (or the Aspire MCP list_resources tool) to check resource status
    • If resources are not healthy, run aspire start + aspire wait api --status healthy + aspire wait web --status healthy
    • If resources show errors, use aspire logs <resource> (or list_console_logs) to diagnose
  2. Determine test type — Select the test suite based on current phase and task
  3. Run tests — Execute the command, capture stdout and stderr
  4. Parse results — Extract pass/fail counts, failure details, and test names
  5. Detect flaky tests — If a test failed, re-run it once; if it passes on retry, flag as flaky
  6. Diagnose failures with Aspire MCP — On test failure, use Aspire observability:
    • list_console_logs for resource stdout/stderr around failure time
    • list_structured_logs for OpenTelemetry log entries
    • list_traces to find the failing request's distributed trace
    • list_trace_structured_logs with the trace ID for full request lifecycle
  7. Structure output — Format results for the orchestrator

Output Format

Type: unit | gherkin | e2e | smoke | all
Pass: <count>
Fail: <count>
Flaky: <count>
Verdict: GREEN | RED | FLAKY

Failed tests:
- <test name>: <error message>

Aspire MCP Debugging Workflow

When tests fail against the Aspire environment:

1. list_resources              → Are all resources Running + Healthy?
2. list_console_logs(resource) → Any errors in stdout/stderr?
3. list_traces(resource)       → Find the trace for the failing request
4. list_trace_structured_logs  → Full trace lifecycle with all spans
5. execute_resource_command    → Restart a resource if stuck

Edge Cases

  • Test runner itself fails (not assertions) → report as infrastructure failure
  • Aspire resources unhealthy → restart via aspire start (auto-stops previous)
  • Tests exceed 5 minutes → check for hung processes
  • Always capture both stdout and stderr

Mandatory Completion Checklist

The orchestrator MUST verify ALL of the following before marking test-runner as complete:

  • All test suites were executed (unit, integration, e2e, Cucumber) — none skipped
  • Results include: total tests, passed, failed, skipped counts per suite
  • Any failures include file path, test name, and error message
  • Aspire environment was healthy during test execution (resources Running + Healthy)
  • If any tests failed, the failure is reported as a blocking issue — not silently ignored

BLOCKING: If the test runner itself fails (infrastructure failure, timeout, partial execution), this must be reported as "run incomplete" — not as "all tests passed".

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

Reviews can't be read right now.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free