scaffold-suite
Skill2 starsChanged 7 days agoApache-2.0View on GitHub
Create the Playwright project a project/product is missing — package.json, a playwright.config.ts encoding the evidence policy, and a first runnable smoke spec with a correct TC-ID. Use for "/scaffold-suite", or when PROJECT.md → "Test assets" says "none yet" and a ticket needs a repeatable check.
- Reads credentials
- Installs packages
In this repository34 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- accessibility-testingSkill
- agentswitchSkill
- api-testingSkill
- bug-reportSkill
- bug-reproductionSkill
- demoSkill
- deploy-to-stageSkill
- exploratory-testingSkill
- failure-analysisSkill
- model-setupSkill
- onboard-productSkill
- onboard-projectSkill
- perf-analysisSkill
- qa-helpSkill
- qa-metricsSkill
- qa-reportSkill
- re-boardSkill
- reconSkill
- release-readinessSkill
- report-to-jiraSkill
- retest-bugSkill
- retro-learnSkill
- risk-based-testingSkill
- security-testingSkill
- skill-reviewSkill
- test-automationSkill
- test-case-managementSkill
- test-reliabilitySkill
- test-ticketSkill
- visual-testingSkill
- webdriverio-testingSkill
- whats-newSkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 1 is distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 22
Read "Test assets" from the effective project file: if a suite already exists,
- Very specific: line 33
Keep the layout: smoke.spec.ts and tests/fixtures/observation.ts land under tests/ (the fixture wires the observation channel — …
- Very specific: line 74
Write a credential VALUE into a spec, a fixture, a config or the chat — CLI specs get credentials via qa-secrets.mjs run <KEY…> injection, …
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
/scaffold-suite — give the evidence promise a producer
The blueprint demands traces/trace.zip, logs/, video on failure and a screenshot
at the assertion point, and the evidence gate blocks a session without them. Those are all
Playwright configuration. Until this skill runs, nothing in a fresh setup produces
them: PROJECT.md → "Test assets" says "none yet", and the first /test-ticket is
asked to satisfy a schema whose producer does not exist.
This is the one place where the evidence policy becomes executable. Read
.claude/rules/core/evidence-policy.md → "Capture matrix" alongside the config
template; if they ever disagree, the rule is right and the config is a bug.
Steps
- Resolve scope and location. Which project, and which product if it has any.
Read "Test assets" from the effective project file: if a suite already exists,
stop — extend it via
test-automationinstead. Scaffolding over a working suite is how two competing configs appear in one repo. - Ask where it belongs. A suite usually lives in its own checkout next to the
app (
../<key>-web-tests/), not inside this blueprint repo — the blueprint holds knowledge, the app repo holds tests. Never guess this; it decides who reviews the specs. - Copy the templates from
templates/into that location and substitute:<KEY>,<PRODUCT>— for the header comments and the package name,<CAT>— a TC category from the effective project file ("Test assets" → TC categories); the product's namespace when a product was routed. Keep the layout:smoke.spec.tsandtests/fixtures/observation.tsland undertests/(the fixture wires the observation channel —docs/pipeline/04-execution.md— into every spec that importstest/expectfrom it instead of@playwright/test). Also copytemplates/probes/— the recon probe is the producer for pipeline phase 2 (docs/pipeline/02-recon.md), and the hybrid/cli-first modes have no non-blind spec source without it. (Variant selection — Playwright probe vs.recon.wdio.spec.js— belongs to/recon; scaffolding from scratch here is Playwright-only. WDIO suites get probes via/reconstep 4.)
- Wire the base URL.
baseURLcomes fromSTAGE_BASE_URL, never a literal. Point the user at their own way of setting it (shell, CI variable, direnv) — this blueprint does not load dotenv files, and a stage URL in a committed config is how someone eventually runs a suite against production. Credentials are different from the base URL: specs read them fromprocess.envby key name, and the runner injects them — preferred:node .claude/scripts/qa-secrets.mjs run <KEY…> -- npx playwright test(loads only the named keys from the QA home's.env, redacts values in all output). No wholesaledotenv.config()in the suite config: it would pull every secret in the file into every worker for nothing. - Install and prove it runs:
npm install && npx playwright install chromium, thennpx playwright test. A scaffold that has never executed is a promise, not a suite — the smoke spec exists precisely so the first run is a real run. - Record it in
PROJECT.md(or thePRODUCT.md, if the suite is product- specific) → "Test assets": location, config file, TC categories. Otherwise the next session scaffolds it a second time. - Anything surprising during setup →
retro-learn, and declare it in the run'slearningsas usual.
What the config deliberately does
| Setting | Why it is not a preference |
screenshot: only-on-failure | Capture matrix. The PASS screenshot is taken by the test at the assertion point, which is the moment worth keeping |
video: retain-on-failure | Full videos on pass waste storage and review time |
trace: on-first-retry in CI, retain-on-failure locally | The CI retry exists to capture the flake's trace, not to turn red green |
retries: 0 locally, 2 in CI | A flake must reproduce locally to be fixable |
workers: 1 in CI | A shared stage tolerates one writer; parallel workers corrupt each other's test data |
outputDir / reporter under QA_EVIDENCE_DIR | Artifacts land in the run's evidence directory instead of a scratch folder nobody attaches |
What this skill never does
Write a credential VALUE into a spec, a fixture, a config or the chat — CLI specs
get credentials via qa-secrets.mjs run <KEY…> injection, browser-MCP sessions
via the --secrets masking (.claude/rules/core/security.md).
Point baseURL at production. Scaffold on top of an existing suite. Or claim the
suite works before npx playwright test has actually passed once.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

