integration

Manually runs the project's integration tests, or, when there is no automation, collects scenarios from a human.

Interaction. Wherever this skill asks the user something, use the host's blocking question tool already in your tool list, matched by capability rather than by a host-specific name; if it is listed but not loaded, load it first with the host's tool-discovery primitive; only when no such tool is listed, or a question call errors, offer numbered options in chat and end your turn to wait for the reply — never answer the question yourself or skip it.

1. Consume Host Documentation First

# Check for integration documentation generated by harness-researcher (Phase 4; consume first if present)
ls docs/verification/integration.md 2>/dev/null && cat docs/verification/integration.md

• If docs/verification/integration.md exists: follow the file's per-stack integration strategy, tools, and run commands.
• If docs/verification/integration.md is absent: fall back to references/web-playwright.md and references/non-web.md.

2. Web-Frontend / Electron / Non-web Detection (heuristic)

For the full list of detection signals and its limitations, see → references/web-playwright.md.

Check Electron first — it is a distinct verdict, not folded into Non-web:

grep '"electron"' package.json 2>/dev/null

If not Electron, check package.json's dependencies/devDependencies against a web-framework allowlist:

# Check for web-framework dependencies
grep -E '"(react|vue|next|nuxt|svelte|@angular/core|solid-js|astro)"' package.json 2>/dev/null

Also check the supporting signals:

# Supporting signals: vite.config, index.html, public/
ls vite.config.* index.html public/ 2>/dev/null

Check the other non-web signals (if present, classify as non-web):

# CLI: bin field, RN: react-native/metro.config.js, Flutter: pubspec.yaml, Go: go.mod/main.go
grep '"bin"' package.json 2>/dev/null
grep '"react-native"' package.json 2>/dev/null
ls metro.config.js pubspec.yaml go.mod main.go 2>/dev/null

The supporting signals stand on their own so that a framework-less web app — plain index.html plus a bundler, no entry in the allowlist — is not filed as non-web and routed to a human. A non-web signal still outranks them: an Electron or React Native project also ships an index.html.

For the full Electron procedure (renderer automation + main-process human-in-the-loop), see references/electron.md — the single source of truth for this exception.

3. If Web — Run Existing Playwright Cases

For details on Playwright config, the testDir/testMatch defaults, and reporter options, see → references/web-playwright.md.

3.1 Parse playwright.config.*

# Detect the config file
ls playwright.config.ts playwright.config.js playwright.config.mjs playwright.config.cjs 2>/dev/null | head -1

Read testDir and testMatch from the config file. The defaults are:

• testDir: ./tests
• testMatch: **/*.@(spec|test).?(c|m)[jt]s?(x)

3.2 Discover Existing Cases

Cases live wherever the config points, so derive testDir in the same command rather than assuming ./tests — that is what makes a zero-case verdict trustworthy enough to scaffold on.

# testDir from playwright.config (§3.1), falling back to Playwright's ./tests default.
# Regex matches the testMatch default exactly: **/*.@(spec|test).?(c|m)[jt]s?(x)
# POSIX find + grep -E: -regextype is GNU-only (BSD/macOS find rejects it, exits 1, and an
# `&& … || echo MISSING` chain then reports every healthy project as MISSING — conflating
# any find failure with an absent directory). `|| true`: grep exits 1 on zero matches, and
# an empty suite is an answer, not an error. MISSING comes only from the [ -d ] test.
TESTDIR=$(grep -hoE "testDir:[[:space:]]*['\"][^'\"]+" playwright.config.* 2>/dev/null | head -1 | sed -E "s/.*['\"]//")
TESTDIR="${TESTDIR:-./tests}"
if [ -d "$TESTDIR" ]; then find "$TESTDIR" -type f 2>/dev/null | grep -E '\.(spec|test)\.(c|m)?[jt]sx?$' || true; else echo "MISSING: $TESTDIR"; fi

Four outcomes, and only two of them authorise scaffolding:

An empty result says "no Playwright cases here", never "this directory is empty" — the two diverge in any project that keeps its unit tests under the same roof.

If there are zero cases, invoke the playwright-scaffold skill to generate a main-screen smoke and run it immediately (not an arbitrary scenario, but a deterministic "does the app come up?" check):

1. Invoke playwright-scaffold via the Skill tool → confirm baseURL (codebase scan + user confirmation) → idempotently generate <testDir>/main.smoke.spec.*. (If Playwright is not installed, the scaffold guides installation.)
2. Run the generated smoke with --reporter=json per §3.3 and report PASS/FAIL per §3.4.
3. In the report, state "this is a starter smoke, so extend it with real cases" (npx playwright codegen <baseURL>).

The runner is npx playwright test (deterministic, SSOT). Use the session's playwright MCP only as an auxiliary path for exploration when cases are absent (it is not the regression SSOT). MCP reference: the "playwright MCP" section of references/web-playwright.md.

3.3 Deterministic Run

If cases exist, run deterministically with --reporter=json (+junit). --reporter=json emits to stdout by default, so to capture it to a file, set PLAYWRIGHT_JSON_OUTPUT_NAME (otherwise the file parsing in §3.4 breaks with ENOENT):

PLAYWRIGHT_JSON_OUTPUT_NAME=results.json \
PLAYWRIGHT_JUNIT_OUTPUT_NAME=results.xml \
  npx playwright test --reporter=json,junit

3.4 Parse Results and Report PASS/FAIL

Parse the results.json specified in §3.3, defensively (a crashed or incomplete Playwright run must still produce a clear FAIL report, not an uncaught exception):

node -e "
  const fs = require('fs');
  let r;
  try { r = JSON.parse(fs.readFileSync('results.json', 'utf8')); }
  catch (e) { console.log('FAIL: results.json missing or invalid (' + e.message + ')'); process.exit(1); }
  const s = r && r.stats;
  if (!s || typeof s.unexpected !== 'number') {
    console.log('FAIL: results.json has no stats — the Playwright run likely crashed before completing');
    process.exit(1);
  }
  console.log('PASS:', s.expected, '/ FAIL:', s.unexpected, '/ SKIP:', s.skipped);
  process.exit(s.unexpected > 0 ? 1 : 0);
"

Report format:

## Integration Test Results — <date>

### Playwright Run Results
| Suite | Passed | Failed | Skipped | Verdict |
|---|---|---|---|---|
| tests/auth.spec.ts | 5 | 0 | 0 | PASS |
| tests/checkout.spec.ts | 3 | 1 | 0 | FAIL |

**Overall verdict**: FAIL (1 failure)
Failed case: tests/checkout.spec.ts › navigate to the confirmation page after checkout completes

4. If Electron — hybrid (renderer automation + main-process human-in-the-loop)

For the full procedure and SSOT, see → references/electron.md.

Electron is a distinct third verdict, not folded into Non-web — it gets partial automation.

1. Run the renderer-process suite the same way as §3 "If Web" (same playwright.config.* parsing, same --reporter=json,junit run, same defensive result parsing from §3.4) — see references/electron.md §1 for the Electron-specific Playwright launch (_electron.launch(), not a plain chromium channel).
2. For main-process scenarios (IPC handlers, filesystem access, native menus), use the same human-in-the-loop procedure as ## 5. If Non-web below — ask the user (structured choice) to collect scenarios, and produce a manual checklist.
3. Report both results together: a Playwright PASS/FAIL table for the renderer + a manual checklist for the main process (see references/electron.md §3 for the combined report template).

5. If Non-Web — human-in-the-loop

For per-type non-web signals and reference OSS details, see → references/non-web.md.

Do not enforce automated integration testing. Determine the specific non-web type from the §2 signal that matched (CLI / React Native / Flutter / Go / generic — do not fall back to a generic list if a specific signal matched), then ask the user (structured choice) for that type's scenarios and pass criteria:

This project was detected as non-web (<type>).
No integration-test automation tool is enforced.

Please provide the following:
1. The core scenarios to verify (e.g., "look up data after user login")
2. The pass criteria for each scenario (e.g., "response code 200, data included")
3. Any test tool you are currently using, if any.

Based on the collected scenarios, write a manual verification checklist and point to applicable OSS (Newman/Maestro/Appium — all Apache-2.0, not automatically enforced). For details, see references/non-web.md.

References

• references/web-playwright.md — web detection signals, testDir/testMatch, reporters, best practices, SSOT URLs (§6)
• references/electron.md — Electron detection + hybrid renderer/main-process procedure, combined report template
• references/non-web.md — non-web type signals, human-in-the-loop procedure, reference OSS
• playwright-scaffold — main-screen smoke generator for web + zero cases (invoked by this skill)