agentleFS
Sn4kyGitSn4kyGit/qa-agent-blueprint/.claude/skills/scaffold-suite/SKILL.md

scaffold-suite

Skill2 starsChanged 7 days agoApache-2.0View on GitHub

Create the Playwright project a project/product is missing — package.json, a playwright.config.ts encoding the evidence policy, and a first runnable smoke spec with a correct TC-ID. Use for "/scaffold-suite", or when PROJECT.md → "Test assets" says "none yet" and a ticket needs a repeatable check.

  • Reads credentials
  • Installs packages

The file

We split this skill into the rules it gives an agent

3 rules. About 0 are common, 1 is distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Read "Test assets" from the effective project file: if a suite already exists,

    line 22
  2. Very specific:

    Keep the layout: smoke.spec.ts and tests/fixtures/observation.ts land under tests/ (the fixture wires the observation channel — …

    line 33
  3. Very specific:

    Write a credential VALUE into a spec, a fixture, a config or the chat — CLI specs get credentials via qa-secrets.mjs run <KEY…> injection, …

    line 74

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

/scaffold-suite — give the evidence promise a producer

The blueprint demands traces/trace.zip, logs/, video on failure and a screenshot at the assertion point, and the evidence gate blocks a session without them. Those are all Playwright configuration. Until this skill runs, nothing in a fresh setup produces them: PROJECT.md → "Test assets" says "none yet", and the first /test-ticket is asked to satisfy a schema whose producer does not exist.

This is the one place where the evidence policy becomes executable. Read .claude/rules/core/evidence-policy.md → "Capture matrix" alongside the config template; if they ever disagree, the rule is right and the config is a bug.

Steps

  1. Resolve scope and location. Which project, and which product if it has any. Read "Test assets" from the effective project file: if a suite already exists, stop — extend it via test-automation instead. Scaffolding over a working suite is how two competing configs appear in one repo.
  2. Ask where it belongs. A suite usually lives in its own checkout next to the app (../<key>-web-tests/), not inside this blueprint repo — the blueprint holds knowledge, the app repo holds tests. Never guess this; it decides who reviews the specs.
  3. Copy the templates from templates/ into that location and substitute:
    • <KEY>, <PRODUCT> — for the header comments and the package name,
    • <CAT> — a TC category from the effective project file ("Test assets" → TC categories); the product's namespace when a product was routed. Keep the layout: smoke.spec.ts and tests/fixtures/observation.ts land under tests/ (the fixture wires the observation channel — docs/pipeline/04-execution.md — into every spec that imports test/expect from it instead of @playwright/test). Also copy templates/probes/ — the recon probe is the producer for pipeline phase 2 (docs/pipeline/02-recon.md), and the hybrid/cli-first modes have no non-blind spec source without it. (Variant selection — Playwright probe vs. recon.wdio.spec.js — belongs to /recon; scaffolding from scratch here is Playwright-only. WDIO suites get probes via /recon step 4.)
  4. Wire the base URL. baseURL comes from STAGE_BASE_URL, never a literal. Point the user at their own way of setting it (shell, CI variable, direnv) — this blueprint does not load dotenv files, and a stage URL in a committed config is how someone eventually runs a suite against production. Credentials are different from the base URL: specs read them from process.env by key name, and the runner injects them — preferred: node .claude/scripts/qa-secrets.mjs run <KEY…> -- npx playwright test (loads only the named keys from the QA home's .env, redacts values in all output). No wholesale dotenv.config() in the suite config: it would pull every secret in the file into every worker for nothing.
  5. Install and prove it runs: npm install && npx playwright install chromium, then npx playwright test. A scaffold that has never executed is a promise, not a suite — the smoke spec exists precisely so the first run is a real run.
  6. Record it in PROJECT.md (or the PRODUCT.md, if the suite is product- specific) → "Test assets": location, config file, TC categories. Otherwise the next session scaffolds it a second time.
  7. Anything surprising during setup → retro-learn, and declare it in the run's learnings as usual.

What the config deliberately does

SettingWhy it is not a preference
screenshot: only-on-failureCapture matrix. The PASS screenshot is taken by the test at the assertion point, which is the moment worth keeping
video: retain-on-failureFull videos on pass waste storage and review time
trace: on-first-retry in CI, retain-on-failure locallyThe CI retry exists to capture the flake's trace, not to turn red green
retries: 0 locally, 2 in CIA flake must reproduce locally to be fixable
workers: 1 in CIA shared stage tolerates one writer; parallel workers corrupt each other's test data
outputDir / reporter under QA_EVIDENCE_DIRArtifacts land in the run's evidence directory instead of a scratch folder nobody attaches

What this skill never does

Write a credential VALUE into a spec, a fixture, a config or the chat — CLI specs get credentials via qa-secrets.mjs run <KEY…> injection, browser-MCP sessions via the --secrets masking (.claude/rules/core/security.md). Point baseURL at production. Scaffold on top of an existing suite. Or claim the suite works before npx playwright test has actually passed once.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.