Writing Robust Tests - ISTQB design + evidence

If the current repo has a dedicated test skill, it wins - it knows the golden dataset and the project commands.

This skill vs. others

• This skill: the code exists, the tests are missing or fragile
• superpowers:test-driven-development: the code does not exist yet (test first)
• detection-sweep: a whole-project audit, not a targeted file

Principle

A green test you have never seen red proves nothing. The output of this skill is not "tests written" but "tests whose failure has been demonstrated".

1. Adapt to the project (2 min, before writing)

• Framework + commands: read package.json scripts / Cargo.toml / Makefile / CI config
• Read an existing neighboring test and copy its conventions (page objects, helpers, fixtures) before inventing
• Data: locate the project's reference seed/fixtures; never invented magic data

2. Design (ISTQB, apply don't copy)

• Level first: unit (~70%) / integration-contract (~20%) / E2E (~10%). Choose deliberately - see the mock trap §4
• Black-box: Equivalence Partitioning · Boundary Value Analysis (threshold 100 → 99/100/101) · Decision Table · State Transition (an entity's lifecycle) · Use Case
• Experience-based: error guessing (null/empty/negative/Unicode/SQLi/XSS), regression checklist
• FIRST: Fast (unit without DB/network) · Isolated · Repeatable (no clock or random) · Self-validating · Timely

3. Absolute E2E rules (any Playwright/Cypress project)

1. data-testid for every locator - never text=, placeholder, CSS class, XPath. If the component has none: adding the data-testids to the component is part of the task, not an excuse for a fragile locator
2. Page Object: locators / actions / assertions separated, scoped locators (parent.getByTestId('child'))
3. ZERO hard waits (waitForTimeout, sleep) - wait for a concrete state: visible, URL, network response
4. Zero assertion coupled to the implementation's timing (a toast that disappears after 3 s tested with a 5 s timeout = fragile coupling) - drive or neutralize the clock, or test the state not the duration
5. AAA, a single act, one concept per test ("and" in the name → split); no conditional test.skip()

4. The total-mock trap

Mocking the whole API in an "E2E" test makes it a UI component test: it will never detect a backend contract break. Choose explicitly:

• True E2E: real backend + seeded data - covers the contract
• Component test: mocked API - fast, but declare it as such and complete it with a contract test

Mock at the boundaries only (external network, clock), never in the middle of the system under test.

5. Mandatory evidence: the test can fail

For each test (or homogeneous group):

1. Run → green
2. Temporarily break the code under test (invert the condition, return a wrong value) → the test MUST go red with a readable message
3. Restore → green

A test that stays green with broken code is a Liar - rewrite it.

Traps & rationalizations

Exit condition

• Project conventions reused (neighboring test read)
• data-testid added to the components touched, page object created/updated
• ZERO hard waits, zero fragile locator, zero timing coupling
• Each test seen red (mutation) then green
• Suite run, output pasted in the summary - never "it should pass"

Tooling

• Advanced techniques (automated mutation testing, contract harness, fuzzing, quality gates): see comprehensive-test-strategy/reference/testing-advanced.md - shared with the strategy skill.