Website QA

The gate every page passes before launch, and the check re-run after any edit. Two parts: the automated basic tests, then the performance check.

1. Basic tests (fix to green)

npm test runs the Playwright suite from the new-website skill's templates/astro/ overlay against a production build (astro build && astro preview — the static output Cloudflare serves). Test the contract, not the cosmetics; red → green → commit.

• a11y.spec.ts — axe WCAG 2 A/AA + best-practice on every page, light AND dark; zero violations.
• seo.spec.ts — title ≤60 / description 120–160 (hard floor AND cap); og/twitter mirror title/description; canonical == og:url == absolute URL; one <h1>; JSON-LD parses; the sitemap matches PAGES exactly (drift alarm: a route missing from PAGES fails here — the website-seo-geo contract).
• navigation.spec.ts — every PAGES route returns 200; internal links resolve with no redirect hop (absolute URLs on the production host count as internal).
• anchors.spec.ts — the other half of internal-link integrity: every in-site #fragment (same-page #x, cross-page /page#x, relative about#x, or /?q#x) points at a real element id on its target page — links and shipped-page target ids are read from the live DOM. Catches the dead-anchor bug navigation.spec can't see — a link that 200s but scrolls nowhere because the id doesn't exist. Offline.
• orphans.spec.ts — the inverse of navigation.spec: every PAGES route must be reachable from the home page by following internal links. A page can be in the sitemap and pass every other spec yet be linked from nowhere (orphaned — crawlers find it late, humans never do). Crawls the in-site link graph from / and fails listing any unreachable page. Deliberately-unlinked pages opt out via ORPHAN_EXEMPT. Offline. The judgment-side report (inbound count, thin pages, where to add links) is the internal-link-audit skill.
• middleware.spec.ts — functions/_middleware.ts, which astro preview never runs, called directly: previews noindexed, live-domain responses unchanged, and the project alias <project>.pages.dev 301ing to the live domain only once CANONICAL_URL is set. Offline.
• analytics-server.spec.ts — analytics server mode, the middleware and functions/go/email.ts called directly with a stand-in for Plausible: one page view per page a visitor opens and none for assets, prefetches, scripts or previews; an email tap counted and answered with the mailto: link; crawlers and links from other sites given the "[at]" hint, never the address. Offline.
• images.spec.ts — every <img> has alt + width/height + an explicit loading (lazy, or eager for the LCP image); raster sources are WebP/AVIF in src, srcset and <picture> (the website-design-system rules).
• phone-images.spec.ts — loads each page as Lighthouse's phone (412px, 1.75x), scrolled to the end, and judges every raster the page itself downloads (not those inside an iframe), CSS backgrounds and masks included (Lighthouse does not size those). Fails a file wider than a 3x phone could use (1.1 × the width it is drawn at × 3) when a copy that wide saves an estimated 4 KiB or more (estimated by pixel area, which can run high); a downloaded file nothing on the page shows (often a preload whose media drifted from the CSS breakpoint); one of 4 KiB or more that is downloaded but hidden at phone width (by its size or visibility, by opacity: 0 on it or an ancestor, or clipped away by an ancestor's overflow: hidden, such as a carousel's waiting slides); and one whose drawn width it cannot work out. A file used only as a border image, cursor or shape-outside is not judged by width, and the log names it. Prints how many files it judged by width and names the hidden ones under 4 KiB. The fixes are in section 2.
• tone.spec.ts — no em dashes / contractions / buzzwords in copy or metadata. Language-aware: the contraction/buzzword rules apply only to <html lang> en* pages; the em-dash ban is universal (the website-content-guide rules).
• positioning.spec.ts — each page in the POSITIONING map carries its positioning term (case-insensitive) in <title>, <meta description> and its <h1> or intro paragraph; declared body phrases (market category + core positioning phrase) appear in the body. Hermetic string containment, not a density check (the website-positioning spine). Empty map = green from commit 1. Also warns (without failing) when a PAGES route has no entry and isn't in POSITIONING_EXEMPT — an un-positioned content page is a lost opportunity, surfaced rather than silent. This is a mechanical string check, not a judgment read — for "does the offer actually read clearly", see website-positioning-check.
• placeholders.spec.ts — leftovers a visitor could read. Reads each page's rendered copy, <head> metadata, alt texts, aria-labels and JSON-LD, plus the web manifest, for an unfilled [SLOT] (the "[MISSING: …]" content token and the starter's [BRACKET] slots alike), filler text, an author note (TODO), an unrendered template expression or a dummy name. The CI grep for [MISSING: is the fast source-side half; this is the half that reads what a page serves. The starter's privacy page, imprint and manifest sit in UNFILLED_UNTIL_LAUNCH and are reported, not failed, until filled and, for the two legal pages, until their draft notice is cleared as AGENTS.md §5 says; a filled target fails until its entry is deleted, so the list empties itself and must be empty at launch (new-website §4). A "[MISSING: …]" on a page only warns on your own machine, so a draft branch can be pushed, and fails in CI. Not read: the 404 page, llms.txt, and the starter's example values (the name Example, hello@example.com); those stay a launch-checklist item.
• privacy-host.spec.ts — the host the privacy page tells visitors to block (the text marked data-analytics-host) is the host in ANALYTICS.scriptHost. A [BRACKET] slot there passes (placeholders.spec.ts reports it), and so does a page without the mark.
• email.spec.ts — no plaintext (harvestable) email address in the served HTML (addresses go through <EmailLink>, which obfuscates; the website-design-system rule).
• tables.spec.ts — every table is in a .table-wrap of one kind (stack or figures) with its region, label and tabindex; stacked tables carry their roles and a label equal to each column header; no page or table scrolls sideways at 360/320px; a stacked table is a table again from 641px; no cell sets hyphens: auto. Tests the CSS on a fixture too, so a site with no table still proves it. See website-design-system, "Tables".
• links.spec.ts — offline outgoing-link guard: no STALE_DOMAINS (domains you've retired) reappear in an anchor href. Hermetic — makes no network calls. The liveness of the links you keep is the network-dependent outgoing-link-audit skill (pre-launch / monthly, not in CI).
• served-build.spec.ts — the server answering on the test port serves this checkout's own build (a random id per test run, from playwright.config.ts), not another checkout's preview that took the port during the build, nor another build in the same folder.

First fill tests/_helpers.ts PAGES with the real routes. A failure is a real defect — fix the page, do not loosen the test (Rule 12: never silently skip). For diagnosing a specific failure, test-failure-triage; for intermittent ones, flaky-test-debugger.

1b. TDD — what NEW tests to write for a feature

The specs in §1 are the always-on baseline. When you add a feature, write its test first (red → green → commit): state what "correct" means before building it, then implement until green. Test the contract/behaviour, not the cosmetics (assert "the form shows a success state", not "the heading is 32px") — a test that can't fail when the logic breaks is worthless (Rule 9).

When a new route is added, the only change needed for the baseline to cover it is adding its path to tests/_helpers.ts PAGES — a11y, SEO, navigation and image checks then apply automatically.

Feature → the high-value test to add:

Keep it thin: a marketing site needs a high-value harness, not 1000 unit tests. Don't test a cosmetic you'll change next week; do pin every contract a third party could silently break.

1c. Enforce the gate on push (the pre-push hook)

The baseline only protects the site if it actually runs before a deploy. The scaffold ships a pre-push git hook (scripts/hooks/pre-push) that runs scripts/verify.mjs (the same as npm run verify: npm ci only on its first run or when package.json or package-lock.json changed, npm run check, then npm test, which builds once; short output) and check_seo.py (if present), and refuses the push if anything is red — so a broken build never reaches the deploy branch. A site without scripts/verify.mjs gets the older npm run build + npm test. It's wired automatically: the prepare script in package.json (node scripts/wire-hooks.mjs, a line with no shell syntax) points core.hooksPath at scripts/hooks on npm install — when the site is the root of its git repo. In a subfolder of a bigger repo it leaves that repo's hooks alone, so the gate is not wired there; say so rather than assume it runs.

This is the local complement to CI: Cloudflare Pages builds on push independently of CI, so without this hook a red test would still deploy. The hook is what makes "fails → does not ship" literally true on a direct-push-to-main workflow, without forcing a slower PR flow.

Offer it as a choice — never silently impose or remove it (Rule 1/Rule 12). When setting up or handing off a site, surface both directions:

• Relax for one push: git push --no-verify (skips the hook for that push only).
• Disable entirely: git config --unset core.hooksPath (re-enable with npm install, or git config core.hooksPath scripts/hooks).
• Tune what it runs: edit scripts/hooks/pre-push (e.g. leave out a slow step; CI still runs everything).

A team that wants the speed of direct pushes and the safety of the gate keeps it on; a solo builder mid-experiment may want it off. Make the call explicit with the user; don't decide for them. (Requires the per-machine npx playwright install chromium from SETUP.md, or the test step errors — a fail-loud, fixable by setup, not a reason to drop the hook.)

2. Performance — Lighthouse / PageSpeed (FCP & LCP)

The agent runs Lighthouse itself; the user is asked only when it cannot.

1. Install it in a scratch folder outside the site, so the site's package.json and its pre-push hook stay untouched: npm init -y && npm install lighthouse@13 there (without the npm init, npm walks up to the nearest package.json and installs into it). Lighthouse needs Chrome; use Playwright's, whose path this prints when run in the site folder: node -e "import('@playwright/test').then(m=>console.log(m.chromium.executablePath()))".

2. Run it in the scratch folder (there npx finds that install, not the latest version), five times, then print the median run's score, FCP and LCP:

   for i in 1 2 3 4 5; do CHROME_PATH="<the path above>" npx lighthouse <url> --only-categories=performance --output=json --output-path=run$i.json --chrome-flags="--headless=new" --quiet; done
   node -e "const r=[1,2,3,4,5].map(i=>require('./run'+i+'.json')).sort((a,b)=>a.categories.performance.score-b.categories.performance.score)[2]; console.log(Math.round(r.categories.performance.score*100), r.audits['first-contentful-paint'].displayValue, r.audits['largest-contentful-paint'].displayValue)"

   Without --headless=new it opens a window, and fails where there is no display. Lighthouse runs as a phone by default; add one run with --preset=desktop for the desktop numbers.

3. Measure a deployed URL, never a local server. python -m http.server sends files uncompressed, which once inflated sizes and misled a decision; astro preview compresses, but is still not Cloudflare. Before launch, measure the main preview (PUBLISHING.md, step 3). It runs no analytics (they are on for production only), so measure production again after launch and report both. A single-stage site has no preview: its first measurement is production, right after the first upload.
4. Report the median run (the one the command prints) with its FCP, LCP and the insights it raises.
5. When Lighthouse cannot run, open https://pagespeed.web.dev/ in the built-in browser and read the result there. Do not lean on the keyless PageSpeed API: its quota runs out.
6. Judge image sizes by phone-images.spec.ts, not by Lighthouse's image insight alone: that insight judges one device and skips CSS images.
7. If neither 1–4 nor 5 works, ask the user to run PageSpeed Insights on the page and paste the mobile and desktop FCP and LCP.

Targets (Core Web Vitals "good"): FCP ≤ 1.8s, LCP ≤ 2.5s, judged on mobile (field data preferred). Then act on what the report flags. Common fixes already baked into the starters / website-design-system:

• LCP image: eager-load + fetchpriority="high", correct format/size, no lazy.
• Render-blocking CSS: Astro inlines it (inlineStylesheets: 'always').
• Fonts: self-hosted woff2, font-display: swap, preload only above-the-fold.
• Layout shift (CLS): explicit width/height on media.
• Defer non-critical JS (analytics is async plus a short inline stub, production-only; none in analytics server mode).

Fixes that worked on a live site built with this toolkit (mobile 98–99 to 100, LCP about 2.0s to 1.5s):

• Analytics, when Lighthouse flags its script: defer both the init and the library, and place the library after the site's own deferred script. The starter keeps its async tag; this is for a site that needs it. A test that reads the built page and checks the two scripts' order keeps it from drifting back.
• A CSS background hero: a phone-sized copy (1200px wide worked), with <link rel="preload" as="image" media> pairs whose queries match the CSS breakpoint exactly (for a 720px breakpoint, (max-width: 719px) and (min-width: 720px)), and a test on each side of the breakpoint that only that side's file downloads. phone-images.spec.ts runs at 412px only, so it cannot see drift near the breakpoint.
• srcset and sizes: sizes mirrors the CSS, as min() of the real column widths. Test it with a width sweep: a fixed list of width/ratio pairs, a fresh browser context for each, and the file expected at each. On the site above, at ten such pairs, Chromium picked the smallest file at least shown width × ratio wide. A site's own sweep lists its own pairs.

For deeper technical SEO (crawl, indexability, Core Web Vitals strategy), run the existing seo-audit skill — do not duplicate it here.

Done means

All basic tests green; FCP/LCP within target on the agent's median run of the main preview (production for a single-stage site), or on the user's PageSpeed run when the agent could not run one (or the remaining gaps explicitly listed, not hidden), with a production run after launch; launch checklist in new-website §4 satisfied.