Feature Verify — Runtime-First API Verification

Verify deployed behavior with read-only runtime evidence:

Primary-model analysis → optional independent reviewer → integrated verdict

Use this for post-deploy checks, smoke tests, and production diagnosis. For local tests use the repository verification route; for changes use flow-drive; for code review use change-verdict.

When NOT to Use

• The requested action changes application data, configuration, or deployment state.
• The task is to implement or modify a feature; use flow-drive.
• The task is a code, security, or test review rather than runtime behavior verification.
• There is no read-only endpoint, observation path, or approved test environment that can produce evidence for the requested claim.

Required References

Read these at the indicated phase:

• P0/P3: safety-rules.md and environments.md
• P1/P2/P4/P5: blackbox-testing.md
• P0/P2/P5: verification-playbook.md
• Report: output-template.md

Bash is permitted only for read-only curl and observation queries reviewed against the safety rules. Missing endpoint allowlist means P3 is skipped.

P0 — Scope and Safety

1. Select --env, defaulting to test, and load its configuration.
2. Run scripts/health-probe.sh <health-url> (3 attempts, 2-second timeout).
3. Compare local HEAD with the deployed version and warn on mismatch.
4. Confirm every planned operation is read-only.
5. Determine L4/L3/L2-API/L2-OBS/L1 using the playbook matrix.

API unreachable with logs gives L2-OBS; without logs gives L1. At L1 skip P3/P4. --level L2 remains an alias for L2-API.

P1 — Diff-Lite Scope

Follow blackbox-testing.md § P1. Map git diff main...HEAD --name-only to affected endpoints, dependency chains, active triggers, and passive targets. This phase scopes behavior; it does not judge code quality. If no diff exists, build scope from the user's feature description.

P2 — Test Charter and Approval

Follow blackbox-testing.md § P2. Generate only cases supported by the detected level: L1 regression, L2 active trigger, L3 passive observation, and M1 metrics. Present the charter and wait for user approval before any P3 request.

P3 — API Execution

Run only for L2-API/L3/L4 after approval. For each allowlisted case:

1. Confirm method and endpoint exactly match the allowlist.
2. Run one request at a time with fixed, non-PII test parameters: scripts/api-exec.sh <GET|POST> <allowlisted-url> [json-payload]
3. Record its request ID, HTTP status, response fields, and latency.

The script rejects methods other than GET and POST; the operator must separately confirm that POST is allowlisted and read-only. L2-OBS always skips P3.

P4 — Observation Correlation

Follow blackbox-testing.md § P4.

• L3/L4: correlate each request ID, then fall back to alternate fields and endpoint/time window.
• L2-OBS/L3/L4: scan the observation window for related errors and warnings.
• Observe background services after their expected delay; query affected metrics at L4.
• Record blind spots. A missing log signal is not itself proof of failure.

For L2-OBS use deploy time → now, a user-provided window, or the last 30 minutes, in that order.

P5 — Independent Verdict

1. The primary model forms a conclusion from P3/P4 evidence.
2. Only when the caller explicitly requests a second opinion, dispatch a fresh, isolated read-only reviewer with P1/P3/P4 evidence. The reviewer must assess scope, commands, allowlist compliance, evidence, blind spots, and confidence without receiving the primary conclusion. An explicit --second-opinion=codex-exec may supply an additional one-shot CLI review; it is additive and opt-in, with no hidden fallback.
3. If no second opinion is requested or available, label the result primary model only and state plainly: "Only the primary model's verdict is present; no independent review ran." Do not describe the primary result as independently verified.
4. Integrate available conclusions using the verdict/confidence rules in the playbook, keeping a skipped or unavailable reviewer visible.
5. Render output-template.md.

The report may recommend another skill, but must not auto-invoke it.

Output

Produce the report from references/output-template.md. It must identify the deployed target and local/deployed alignment, record the detected degradation level, list every executed and skipped case with the reason, preserve request/log/metric evidence, state blind spots, and end with the integrated verdict and confidence. A skipped API phase is valid only when the level or safety gate requires it; it is not a passing execution.

Verification

• P0 established environment, deployment alignment, read-only scope, and the degradation level.
• P1 mapped the diff or explicitly documented the feature-description fallback.
• P2 produced a level-appropriate charter and received approval before any P3 request.
• Every P3 request matched the endpoint allowlist and recorded status, fields, and latency, or was marked skipped with a reason.
• P4 correlated logs or metrics when available and recorded the observation window and blind spots.
• P5 contains the primary analysis, any requested independent review (or the explicit primary-only state), and the integrated verdict from the playbook.
• The final report matches output-template.md and does not claim evidence for a phase that was skipped or unavailable.