E2E Test Validator

Audit end-to-end test files to ensure they comply with every rule the e2e-test-writer enforces. Read-only: produce a compliance report; never modify test, harness or production files, never execute tests (static analysis only).

The full rule set is in ../e2e-test-writer/references/e2e-rules.md — validate against it. Build its project profile from the project's GATES.md first: it says which profiles, folders, test language, fixtures and data policy the rules apply to here.

Scope: the project's e2e test directory. Out of scope, reported as such: the visual suite, and the harness's own files — the runner config, the global setup, the seed tests, the fixtures module and any module imported only by them (a sessions helper, say), as the e2e section of GATES.md names them. A harness-like file shipped with a feature's tests (its own config, global setup, or a support module shared between spec files) is not out of scope: it is a CRITICAL H2 violation.

INPUT

• test_file_path (required) — test file or directory.
• checklist_file_path (optional) — the e2e-test.md for coverage cross-reference; when absent, look for docs/<feature-id>-*/e2e-test.md from the files' feature tag.
• contract_path (optional) — the feature's contract.md, to check that tags name real ids; when absent, look for docs/<feature-id>-*/contract.md from the feature tag. Ids repeat across features: never check a tag against another feature's contract.
• severity_filter (optional) — critical | major | minor (default: all).

OUTPUT

A compliance report (English): the project profile it applied (a few lines), summary counts, overall verdict — PASS = 0 critical and 0 major (minors are listed, not counted against it); PASS WITH WARNINGS = 0 critical and ≥1 major; FAIL = ≥1 critical — each violation ([SEVERITY] Rule-ID, location, description, expected, found, fix), traceability analysis, and positive findings. When dispatched, the verdict is the signal the caller consumes.

Directory input: one report per file, then a summary. A finding about the feature rather than a file (a missing checklist) is reported once, in the summary, and counted once. Several places breaking the same rule in one file are one finding with every location listed — one per severity when the rule has two (V2, V3).

EXECUTION STEPS (3 Phases)

Phase 1 — Structural Analysis

Build the project profile from GATES.md (and the harness files for its fallbacks). Inventory describe/test/hooks/fixtures (and their scope and dependencies)/helpers and each test's tags. Map checklist ↔ tests and contract ids ↔ tags. Identify every navigation, action, assertion, request capture, page.route mock, API call (its URL base and whether its status is checked), data-creation point and cleanup point. Note the file's path (which profile folder), its language and its imports.

Phase 2 — Rule-by-Rule Validation

Check EVERY rule from ../e2e-test-writer/references/e2e-rules.md:

• Forbidden Practices [CRITICAL] — waitForTimeout, fixed sleeps or networkidle (V2); retries or describe.configure({ retries }) (E4); a mocked happy path (V7); a filter validated only by an empty result (V5); a test with no action, load-only (V1); computed-style or box measurement (Boundary); creating or editing production code, the harness or the visual suite (H2); direct database access — SQL, ORM, database client (D2); test.describe.serial, state carried between tests, or a data-creating fixture that is not test-scoped (S5); acting on an item whose identity the test does not know — whatever renders first (nth-child, .nth(n), getByRole('option').first()) — or XPath (V3). Choosing an item from data (a record the test created, an entry of a captured response) and acting on it by name or id is correct.
• Traceability & placement [CRITICAL] — the evaluator finds coverage only by tag and the session only by folder, so a miss here re-opens the gap on every evaluation: S1 the file sits in the folder of the profile whose session the flow needs (per the project profile), named per S1; S3 every test carries the anchored feature tag (the route's slug tag for work without a feature) and the id of every surface it touches, and a criterion tag only where it proves that criterion completely. With a contract, every surface and criterion tag must name a real id.
• Session [CRITICAL] — A1 no login in a test, hook or fixture (no filling the sign-in form, no POST to the login route); A2 no forged token/session, no anti-bot bypass; A3 no storage-state file read directly — another profile's session only through its harness API fixture. The flow under test runs in the profile folder's session.
• Data [CRITICAL] — against the project's data policy: D1 no mutation of a protected or pre-existing record, no child created under one, and no submit pressed on a form bound to one — referencing configuration is allowed, editing it is not; D2 the test creates its own records, parents before children, with a run token, through the UI or HTTP API; D3 one test-scoped fixture per created record, each depending on its parent's (or a single record in try/finally) — a fixture that creates a parent and then a child, so the parent leaks when the child's creation fails, is CRITICAL; every removal attempted, each status checked, a throw naming what was left; a catch that swallows a failed removal is CRITICAL; D4 removal through a delete route and an API fixture allowed to use it; D5 no creation without a removal path; D7 a precondition the test cannot create is read from declared data and asserted; A3 API calls to the API origin, never relative to the page, with setup statuses checked.
• Missing data [CRITICAL] — V4 no test.skip, early return or count-guarded branch that lets an absent precondition pass; the precondition is asserted.
• Language [CRITICAL] — S4 English in test/describe names, comments, variables and helper names. Product copy is exempt: labels, button names and messages inside locators and assertions are matched in the product's language, which is correct, not a violation.
• Naming [MINOR] — S4 names state the behavior they check: an English name that states something else (a predicate named for another request) is MINOR.
• Checklist [MAJOR] — when validating a feature's tests, its docs/<feature-id>-<kebab>/e2e-test.md exists, its coverage table has one row per e2e row of the contract's Test-suite hint (and none for other suites), lists every test title and records the D4 residue, and every row carries an outcome of the closed list in ../e2e-test-writer/references/e2e-rules.md ("Coverage outcomes"), worded as that list words it — tests (unproven until proven), disputed entries naming one test each (alone or next to the row's tests), a not e2e-testable reason, out of e2e scope — <suite>, or not in this request. The evaluator reads coverage from it and rejects any other wording.
• Structure [MAJOR] — S2 the project's test language and import form; test/expect from the fixtures module when one exists; no module system mixed with the harness's.
• Assertions [MAJOR] — V3 role/label/text locators first; structural markers only where the accessible name is unstable or absent; an item matched exactly (an unanchored #12 also matches #123). V5 a filter or scoped list checked with data in which it can fail. V2 a snapshot read (count(), allTextContents()) only after a web-first assertion proved the state it reads. Requests captured before the action that fires them.
• Requests [MINOR] — V6 the requests the action should fire are captured and asserted; exactly-once asserted for every request whose duplicate would be a defect (a submit that creates, a search).
• Execution [MAJOR] — nothing that forces headed mode (E1).

V8 (never weaken an assertion) is a behavior of the writer, not a static property — flag it only when a comment or diff context shows an expectation changed to match an observed defect.

Overlaps, counted once: a test that writes on a record it did not create is one D1 finding; a proof too weak for its data is one V5 finding (MAJOR), plus an S3 finding (CRITICAL) only when the contract's criterion needs the missing proof; a wrong profile folder is one S1 finding. H1 is the writer's precondition: note missing harness pieces in the summary, uncounted.

Do not skip rules after finding criticals. State which were skipped and why.

Phase 3 — Compliance Report

Emit the full report per OUTPUT. Verdict is derived strictly from the counts.

RULES

Always:

• Build the project profile from GATES.md, then validate against ../e2e-test-writer/references/e2e-rules.md; check every rule.
• Give exact line/block locations and a concrete fix per violation; report positives. Output in English.
• Derive the verdict strictly from severity counts.

Never:

• Modify any test, harness or production file (read-only). Execute the tests. Mark PASS with any CRITICAL. Produce partial reports.
• Judge a project by another project's profiles, folders, language or data rules.

Edge Cases

• Empty test file: CRITICAL. Load-only test: CRITICAL (V1).
• Data created with no cleanup, a cleanup that swallows a failed removal, or a parent that leaks when its child's setup fails: CRITICAL (D3). Delete of an id the test did not create: CRITICAL (D1).
• Login in a beforeAll: CRITICAL (A1) — one login per file still multiplies across files and spends the rate limit.
• Product copy inside getByRole/getByLabel/toHaveText in the product's language: not a Language violation.
• File of the visual suite, or a harness file of the repository: out of scope — say so; do not validate it against these rules.
• GATES.md has no e2e section and the harness reveals no profiles: report that the project profile cannot be built, and validate only the rules that do not depend on it, stating which were skipped.
• No contract can be found for the feature tag: state that S3 ids and criterion completeness were not verified.