Integration Test Validator

Audit integration test files to ensure they comply with every rule the integration-test-writer enforces. Read-only: produce a compliance report; never modify test or production files, never execute tests (static analysis only).

The full rule set is in ../integration-test-writer/references/integration-rules.md — validate against it. Build its project profile from the project's GATES.md first: it says which suites, helpers, removal order and query counter the rules apply to here, and names the project's own rules file, which is audited too. The suite's language file, ../integration-test-writer/references/languages/<language>.md, is part of the rules: resolve the language as integration-rules.md ("Language") says, and read that one file.

Scope: the integration suites the project profile names. Out of scope, reported as such: a file of a unit, e2e or visual suite.

INPUT

• test_file_path (required) — test file or directory.
• checklist_file_path (optional) — the checklist, for coverage and query-growth cross-reference; when absent, look for it next to the test file (its extension replaced by .integration-test.md) and in each docs/<feature-id>-<kebab>/ folder whose checklist names the test file; when none is found, report checklist alignment as not verified.
• target_file_or_endpoint (optional) — the handler or endpoint under test.
• language (optional) — the suite's language; when given, it wins over what the profile and the repository say.
• severity_filter (optional) — critical | major | minor (default: all).

OUTPUT

A compliance report (English): the project profile it applied (a few lines), summary counts, overall verdict — PASS = 0 critical and 0 major (minors are listed, not counted against it); PASS WITH WARNINGS = 0 critical and ≥1 major; FAIL = ≥1 critical — each violation ([SEVERITY] Rule-ID, location, description, expected, found, fix), the rules not verified or not applicable with the cause of each, coverage analysis, and positive findings. When dispatched, the verdict is the signal the caller consumes.

Directory input: one report per test file — a checklist or any other file in the folder is not audited — then a summary: a row per file with its counts and verdict, and the folder's verdict, derived from the summed counts. Several places breaking the same rule in one file are one finding with every location listed.

EXECUTION STEPS (3 Phases)

Phase 1 — Structural Analysis

Build the project profile from GATES.md (and the repository, for its fallbacks), and read the project's rules file when it names one. Resolve the suite's language and read its language file; when the language has none, say so and go on with the generic rules. Inventory the groups, tests, hooks and helpers. Map checklist ↔ tests and endpoint ↔ tests (if provided). Identify every data-creation point and the cleanup that answers it — the guaranteed block, the suite-level hook, the removal order. Note the entry point each request uses, the credential it carries, and the file's imports.

Phase 2 — Rule-by-Rule Validation

Check EVERY rule from ../integration-test-writer/references/integration-rules.md:

• Data [CRITICAL] — D1 nothing the test did not create is changed or removed; D2 the test creates its own records; D3 every data-creating test removes its data in a block that runs even when the test fails, never in a suite-level hook alone; D4 no removal fails silently — a swallowed failure is a violation; D5 children before parents, in the profile's order; D6 by stored id only, no broad delete; D7 no manual cleanup stacked on a suite that isolates another way.
• Forbidden Practices [CRITICAL] — F1 production code created or changed; F2 data left behind; F3 cleanup of a test's own data only in a suite-level hook; F4 an external data-population script; F5 a focused or skipped test, or a test with no assertion; F6 the handler, router or data-access layer under test replaced by a double.

• Query growth [CRITICAL] — only when the profile declares a query counter. Cite the id of each violated Q rule instead of paraphrasing it. Check:

  • each endpoint in the section's scope has exactly one recorded outcome in the checklist;
  • every not measurable or pre-existing N+1 outcome holds against the code, and no other reason is used;
  • every growth test meets every Q rule.

  Check only what the dispatch targets. In a suite written or expanded from a checklist, that means the checklist's endpoints. When confirming a correction, it means only the tests the writer wrote or changed. Run directly on a file with no checklist, it means the growth tests that file contains: audit them against the Q rules, and state that recorded outcomes were not verified. A growth test missing for an endpoint outside that target is not a violation: legacy suites have none.

• Test language [CRITICAL] — S3 names, comments and identifiers in the profile's test language.
• Project rules [as the project's file grades them; CRITICAL when it does not say] — every rule of the project's rules file, cited by its heading. With no rules file there is no such category: the conventions of existing tests are context for the other rules, never findings of their own.
• Structure [MAJOR] — S1 location and name; S2 the suite's language and its idiom, as the language file states it; S4 arrange-act-assert; S5 independence; S6 the environment opened in the suite's setup and closed in its teardown, through the profile's helpers.
• Authentication [MAJOR] — A1 the credential comes from the profile's helper, never hard-coded; A2 authenticated requests carry it.
• Tautology [MAJOR] — V5 no expected value rebuilt with the product's own code — its serializer, its mapper, its query.
• Coverage [MAJOR] — V2 success, authentication and authorization, validation, not-found, boundaries, relationships, hostile input; checklist alignment. The scope is the endpoint the dispatch names — with none, each endpoint the file exercises. A category the checklist records as not applicable is not missing; with no checklist, neither is one the code shows the endpoint cannot exhibit.
• Assertions [MINOR] — V1 status asserted on every request, body on success, error shape on failure; V3 a write confirmed by reading it back; V4 no comparison against a literal count of shared data.
• Naming [MINOR] — N1 the group names the endpoint, and each test says what it does and under which condition.
• Traceability [MINOR] — D8 in an environment that tests or runs share, a created record carries a unique token where a field allows one. In an environment private to the run it has no subject.
• Language anti-patterns [as the language file grades each] — every anti-pattern of the suite's language file, cited by its id. With no language file there is no such category.

Overlaps, counted once — the highest severity wins: data left behind because a test has no guaranteed cleanup is one D3 finding (F2 and F3 are the same breach); a seed record changed and never restored is one D1 finding; a test that relies on data another created is one D2 finding (S5 is the same breach); a growth test with no guaranteed cleanup is one D3 finding (Q4 is the same breach). Every other violated Q rule is a finding of its own, even when one test breaks several. A language anti-pattern that is the form a generic rule's breach takes is one finding, citing both ids. Several places breaking the same rule in one file are one finding with every location listed.

What is not audited, and what has no subject: the execution rules (E1–E5) are the writer's; the validator runs nothing — no test and no project code; read-only git commands (status, log, show, diff) are allowed. F1 is checked against the commit or diff that adds the tests. A commit that adds production code along with them — a root commit that adds the whole repository, say — proves nothing either way: report F1 as not verified, never as a violation. With no commit and no diff, it is not verified too. Where the language keeps tests inside the source file, a commit or diff that is the writer's alone — tests and nothing else — is read line by line: a change confined to the test module is test code, and a changed line outside it is an F1 violation. One that also carries production changes proves nothing, as above. A rule with nothing to apply to is reported as not applicable, with the cause.

Do not skip rules after finding criticals. State which were skipped and why.

Phase 3 — Compliance Report

Emit the full report per OUTPUT. Verdict is derived strictly from the counts.

RULES

Always:

• Build the project profile from GATES.md, then validate against ../integration-test-writer/references/integration-rules.md, the suite's language file and the project's own rules file; check every rule.
• Give exact line/block locations and a concrete fix per violation; report positives. Output in English.
• Derive the verdict strictly from severity counts.

Never:

• Modify any test or production file (read-only). Execute the tests. Mark PASS with any CRITICAL. Produce partial reports.
• Judge a project by another project's helpers, removal order or data rules.

Edge Cases

• Empty test file: CRITICAL. A test with no assertion: CRITICAL (F5).
• Data created with no guaranteed cleanup: CRITICAL (D3). Removal in the wrong order: CRITICAL (D5). A swallowed removal failure: CRITICAL (D4).
• The profile says the suite's isolation replaces cleanup (a rollback per test, a database per worker): D3 to D6 are met by that mechanism — do not report their absence; report manual cleanup stacked on top (D7). A private database whose data policy still names removal helpers is not that case: D3 to D6 apply as written.
• The profile declares no query counter: skip the query-growth category and say so.
• The suite's language has no language file: validate against the generic rules alone, read in the language's own idiom, and say so in the profile lines of the report.
• A file of a unit, e2e or visual suite: out of scope — say so; do not validate it against these rules.
• GATES.md has no test profile and the repository reveals no suite: report that the project profile cannot be built, and validate only the rules that do not depend on it, stating which were skipped.
• No checklist can be found: state that checklist alignment, not-applicable categories and recorded query-growth outcomes were not verified. It is not a violation of the file.
• Directory input: one report per test file, then the summary with the folder's verdict, as OUTPUT describes.