Web SDK instrumentation — E2E matrix from DESIGN

Use for pulse-web-otel/ instrumentations registered via InstrumentationRegistry (OTLP logs/traces/metrics, ecommerce-demo Playwright).

Relationship

Plan folder must exist on disk (DESIGN.md + active PLAN at minimum). If the branch only described docs in chat but never committed them, write or restore the plan folder first—this skill cannot invent PulseFeature names or flush rules without those files.

Self-heal: If review finds a recurring E2E gap (missing gate-off, wrong flush assert, spec not on e2e:web-sdk-gates script), add one line to web-sdk-instrument/reference.md section F per lifecycle Principle 8 in web-sdk-instrument/SKILL.md so the next matrix pass includes it by default.

Inputs (read in order)

1. DESIGN.md in the instrumentation’s plan folder — entrypoint and links.
2. PLAN-B-*.md (or equivalent) — canonical signal shape, flush rules, gate names, minimum unit/E2E matrix if present.
3. ADR-*.md — decisions that imply tests (e.g. logs vs metrics, deferred behaviors).
4. 04-contract-parity.md — web vs Android; divergence → explicit test or documented skip.
5. src/semconv.ts (PulseWebSemconv) — exact pulse.type, bodies, attribute keys (no invention).
6. src/remote-config.ts — PulseFeature string for gate-off seeds (must match exactly).
7. examples/ecommerce-demo/e2e/fixture.ts — findAllLogs / waitForLog for logs; findAllSpans / waitForSpan (or equivalent) for traces—do not use log helpers alone when the signal is span-only. JSON OTLP requires .env.test VITE_PULSE_FORMAT=json. If pulse.type varies per observation (e.g. network.200 vs network.404), add a prefix match or predicate helper in fixture.ts and list it in the checklist (Input 7 touchpoint).
8. Existing specs — examples/ecommerce-demo/e2e/*.spec.ts that mention the instrumentation or pulse.type.

Output A — comprehensive case list (before writing code)

Produce a numbered checklist grouped by category. Each item is one atomic Playwright scenario (or clearly marked “unit-only / Vitest”).

Categories (force coverage)

Grill pass (mandatory)

Answer yes/no with where the test lives or explicit deferral in ADR:

1. Could getAttr pass with string where we need number? → use typeof === "number" + Number.isFinite.
2. Gate-off polluted by earlier captures? → otlp.reset() after proof-of-life log.
3. New spec file listed in examples/ecommerce-demo/package.json → e2e:web-sdk-gates script (lifecycle reference.md D3).
4. Demo UI actually reaches the code path? (D0a — no vacuous pass.)
5. E2E asserts attrs from PerformanceResourceTiming (or metrics derived from it)? → confirm real timing entries exist for that traffic (see D0e / Anti-patterns).

Demo readiness — D0e (timing / Resource Timing)

Before writing E2E that assert PerformanceResourceTiming-sourced or downstream metric fields (e.g. protocol version, transfer sizes): page.route fulfillment often does not produce real Resource Timing entries in Playwright. Run a quick probe or document explicit deferral in ADR/PLAN-B—otherwise assertions vacuously pass or encode wrong assumptions.

Revalidate (second pass)

• Collapse duplicates; mark P0 if missing D2b or positive-path session.id / screen.name where logs are asserted.
• Cross-check PulseFeature ↔ featureName in seeded config (e.g. click not clicks).

Output B — review existing E2E

For each relevant e2e/*.spec.ts:

1. Map tests → cases from Output A — mark covered / partial / missing.
2. Upgrade partial tests: add missing attrs, numeric finiteness, reset pattern, getAttr keys from semconv.
3. Note chromium-only skips where PerformanceEventTiming / platform APIs require it.

Commands (after matrix agreed)

cd pulse-web-otel && yarn test:run
cd pulse-web-otel && yarn workspace ecommerce-demo e2e:web-sdk-gates

Append gate results to CI / PR description (optional pulse-web-otel/progress.txt) after green gates.

Anti-patterns

• Asserting only toBeDefined() for numeric contract attrs on positive paths.
• Gate-off without otlp.reset() after session.start.
• Gate-off seeded with sessionSampleRate: 0 on the wrong features[].featureName (e.g. session instead of network_instrumentation)—instrumentation still enabled; test lies.
• Adding e2e/foo.spec.ts without appending to e2e:web-sdk-gates in package.json.
• Asserting PerformanceResourceTiming-derived attrs for traffic served only via page.route—entries usually absent; defer in ADR or assert only attrs not tied to Resource Timing for stubbed requests.
• Span signals asserted only with waitForLog / findAllLogs—use waitForSpan / findAllSpans when PLAN-B says trace.

Optional deep dive

For a printable matrix template (copy-paste table), see reference.md.