cross-browser-checker
Skill1 starsChanged yesterdayMITView on GitHub
Run the same pages, states and journeys in Playwright's Chromium, Firefox and WebKit, three times each, and report where they behave differently - script errors, failing steps, content that isn't rendered, missing elements, input fallbacks, layout and network differences - keeping only differences that repeat in every run and listing the rest as flaky. With a repository, also scan CSS, JavaScript and HTML for features the project's browserslist targets don't all support (web-features and caniuse data, pinned and hash-checked), and link them to the runtime differences they explain. Never says "compatible", and never treats WebKit as Safari.
- Installs packages
In this repository33 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- api-contract-testerSkill
- bug-reproducerSkill
- ci-failure-triagerSkill
- ci-suite-runnerSkill
- code-security-auditorSkill
- db-migration-reviewerSkill
- dead-test-finderSkill
- exploratory-charter-writerSkill
- flaky-test-hunterSkill
- hypercare-anomaly-watcherSkill
- i18n-checkerSkill
- journey-mapperSkill
- journey-to-testsSkill
- mobile-app-auditorSkill
- mutation-testerSkill
- performance-budget-auditorSkill
- post-deploy-validatorSkill
- privacy-flow-auditorSkill
- qa-orchestratorSkill
- release-readiness-gateSkill
- requirements-traceability-auditorSkill
- resilience-proberSkill
- risk-based-test-plannerSkill
- test-case-designerSkill
- test-data-auditorSkill
- test-impact-selectorSkill
- test-suite-evaluatorSkill
- ticket-readiness-reviewerSkill
- ticket-to-tddSkill
- ui-crash-hunterSkill
- visual-regression-reviewerSkill
- wcag-auditorSkill
The file
We split this skill into the rules it gives an agent
4 rules. About 0 are common, 0 are distinctive, and 4 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
4
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 115
Before the screenshot it scrolls through the page a viewport at a time, waits up to 5 s for lazy images and returns to the top, and sets …
- Very specific: line 288
Only captured runs are compared.
- Very specific: line 352
Every candidate is promoted, needs_review (for example media that may play in branded browsers with proprietary codecs, and every candidate …
- Very specific: line 365
When a site serves different scripts per engine (compare scripts across engines), say so for every difference on that page.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Cross-browser checker
Mission
Compare one version of a site across engines and report the differences that matter, each with evidence from both sides. Separate real engine differences from flaky behaviour and from rendering noise (font metrics, native controls). Explain differences with the features the project's targets don't support, and say clearly what wasn't tested.
The UI crash hunter finds crashes in one browser; the visual regression reviewer compares a page with its earlier self; this skill compares engines with each other. A behaviour every engine shares is not a cross-browser finding, even if it's a bug: note it for the crash hunter.
Non-negotiable rules
- Never "compatible". The verdict is
differences_found,no_differences_detected(scoped to these pages, states and engines) orinconclusive. Real Safari, older versions and real devices areuntestedunless the user ran them. - WebKit is not Safari. Playwright's WebKit lacks Apple's media, font and platform layers (on
Windows and Linux it is a different build again). Never call a WebKit result a Safari result;
out_of_scopealways lists Safari and real devices. Some differences come from the Playwright build itself:data/engine-limitations.jsonlists the known ones (WebKit on Windows decodes no AVIF; Playwright's Chromium has no H.264 or AAC). A candidate that matches one is markedbuild_limitations, has low confidence and goes toneeds_review, never straight to a difference: confirm it in real Safari (macOS or iOS) or the branded browser first, and say so. - A difference must repeat. Each check runs 3 times per engine. A difference counts only when it
holds in every run of the affected engine and in no run of the reference (Chromium). Anything in
between is
flaky, listed with its run counts. - Evidence from both sides: screenshots and messages from the reference and every affected engine.
- Static feature findings are
possibleuntil an engine shows the difference; then they areconfirmed_at_runtime, linked to it. - Safe targets. Loopback needs nothing; other hosts need
authorised_by. Non-GET requests are blocked unlessallow_form_submission. - Record every engine and data file as
ran,not_runorerrored, with versions and hashes. - Third parties are blocked unless the targets allow them. By default the harness aborts every
request to a host that isn't the site, loopback or in
allowed_hosts(analytics, ads, embeds, fonts from another host) and records each one inblocked_requests; console errors and failed requests caused by the blocking are left out of the runs. Their behaviour is then not compared: the report'sscope.third_partylists the blocked hosts and a limitation says so."third_party": "allow"compares them too. - Pages are modified for capture, and the page's CSP is bypassed to do it. Capture injects a
style that freezes animations and transitions, and one that sets
content-visibility: visiblefor the screenshot. A Content-Security-Policy with a hash or nonce instyle-src(usual on Next.js) refuses both, so every context is created with Playwright'sbypass_csp("bypass_csp": true, the default; it changes nothing for real users). Each style is read back after injection; one that didn't apply makes the runerrored. With"bypass_csp": falseon such a site, every run errors and the verdict isinconclusive. - Be gentle with remote sites. Page loads are spaced by
rate_limit_seconds(0.5 s off loopback); a 429 or 503 is retried afterRetry-After(the shared harness policy: a wait overmax_retry_after_secondsstops instead), and recorded; every run counts the requests it sent,runs.jsontotals them, andrequest_budgetstops capturing once reached (later runs areskipped). Each check costs engines × reruns page loads, and a framework page can make 50 or more requests: size pages × viewports to the agreed budget first. A Next.js App Router page prefetches the routes its links point to, so one load of a page with many links can send 80 or more requests: count a page's requests in a first run before sizing the plan. - A failed capture is never a pass. A run that couldn't load, inject its styles or capture is
errored(orskippedwhen the request budget ran out), with the error text; an engine none of whose runs captured iserrored. Only captured runs are compared, and when any planned engine-versus-reference comparison couldn't be made and no difference was found, the verdict isinconclusive: the summary says so and a limitation names what wasn't compared. Zero comparisons made is never "no differences", and confidence is then low. A run that loaded but can't be compared is screened and recordederroredtoo (see "Environment and blank runs"). - Motion isn't compared by default. Runs ask for reduced motion and freeze animations, for
stable screenshots, so
out_of_scopesays engine differences in motion weren't compared."reduced_motion": "no-preference"and"freeze_animations": falseturn that off.
Inputs
| Input | Default | Notes |
targets | required, except static only | targets.json: site, pages, states (actions, critical_path), viewports. A page or state may list viewports, a subset of the top-level list, to run only there (a mobile menu state at the mobile width); a state inherits its page's. Actions are the shared harness's steps: {"click": sel}, {"fill": sel, "value": v} (or {"fill": [sel, v]}), {"select": sel, "value": v}, hover, press, check, uncheck, wait (ms), wait_for, goto; every step is checked before a browser starts |
third_party | block | block aborts and records requests to other hosts; allow lets them load |
allowed_hosts | [] | Other hosts that are part of the site (its own CDN) |
service_workers | allow | block blocks them in every run |
scroll | true | Scroll through each page before the full-page screenshot (see Capture) |
bypass_csp | true | Create contexts with Playwright's bypass_csp so the capture styles apply under a strict CSP |
reduced_motion, freeze_animations | reduce, true | no-preference / false to compare motion too |
rate_limit_seconds | 0.1 on loopback, else 0.5 | Seconds between page loads |
max_retries, max_retry_after_seconds, stop_on_429 | 2, 60, false | 429/503 retries after Retry-After (the shared harness policy); a longer Retry-After, or any 429 with stop_on_429, errors the run |
request_budget | none | Stop capturing once the pages have sent this many requests; later runs are skipped |
engines | all three | Missing ones are not_run |
browserslist | from the repo, else Next.js's defaults for a Next.js project, else a Vite project's build.target (or Vite's default), else defaults | package.json or .browserslistrc |
repo | none | Turns on the static feature scan |
static_only | false | Feature scan without browsers; every finding stays possible, run_status is not_run and the verdict inconclusive (see "Static only") |
storage_state | none | A signed-in test session for an app behind sign-in (see "Apps behind sign-in") |
reruns | 3 | Per engine and check |
authorised_by, auth, allow_form_submission | as in browser_harness.py |
Workflow
capture.py needs Playwright (pip install -r requirements.txt, python -m playwright install
chromium firefox webkit; ask first). The rest is stdlib.
1. Data
python scripts/fetch_data.py # caniuse-db and web-features, checked against npm integrity and SHA-2562. Capture
python scripts/capture.py targets.json --out <run>/capturePer page, state, viewport and engine, 3 fresh runs: a full-page screenshot, console and page errors,
failed requests, visible elements with boxes and input types, images with their natural size,
horizontal overflow, the scripts loaded (to spot user-agent sniffing), and each step's outcome.
Before the screenshot it scrolls through the page a viewport at a time, waits up to 5 s for lazy
images and returns to the top, and sets content-visibility: visible while the screenshot is taken:
Firefox and WebKit draw a full-page screenshot without scrolling, so loading=lazy images and
content-visibility: auto sections below the fold would otherwise come out blank and look like
content that isn't rendered ("scroll": false turns this off). runs.json records the platform,
the capture settings, the planned checks (page or state × viewport) and what each scroll did, and
per run run_status (ran, errored, skipped), the error, requests_sent and any 429/503
backoff; requests totals them. Each text element and form control records its computed
font family (and faces records, per family, the width of a test string drawn with it, so the same
family string resolving to another face is seen), an SVG shape records its stroke width, an element
another element covers at its centre (a modal overlay, a sticky panel) is
marked occluded with covered_by, and panels that scroll their own content are listed in
scrollers with their scroll offset. A full-page screenshot shows only the first box of such a panel
(an app shell of height 100vh): content further down is compared by DOM box only, so say so in the
limitations and scroll the panel by hand when a finding depends on it. Check the engines' status and the warning capture prints before
going on: an errored run is never a pass. Run one engine at a time (--engines webkit) when memory
is short.
Environment and blank runs. Some runs load but show the test environment rather than the engine:
- A CSP
upgrade-insecure-requestson a plain-http target (a localnext start, a loopback server): engines that honour it on that host (Playwright WebKit, and Safari) fetch every subresource overhttps://from a server that only speaks http, so every stylesheet and script fails and the page renders unstyled. Playwright'sbypass_cspdoes not cover it, and on https hosting it does nothing. Capture recordscsp_upgrade_insecure_requests(from the response header or a meta tag) and warns; a run whose own requests to the site were upgraded to https and failed iserroredwith the reason, andcandidates.jsoncarries oneenvironmententry (causehttps_upgrade) instead of dozens of request, element and font candidates. Serve the site over https, or for the capture through a loopback proxy that drops the directive, and capture again; the report must name the cause in a limitation (the validator checks). - A blank or incomplete rerun: capture records each screenshot's
uniformity(the shared harness'simage_uniformity). A rerun with under half the median element count of its sibling reruns (and 20 or more fewer), or a one-colour screenshot when theirs aren't, is captured once more (retriedrecords why); if the retry is no better, the run iserrored. compare_engines.py screens the same way for older captures (screened_runs), so one bad load is one comparison not made, never hundreds of flaky entries.
A dark page can be one colour for 98% of its pixels and still be a good capture, so uniformity alone never screens a run: only against its own reruns.
Each run starts in a fresh browser context, so a service worker the site registers is never in
control on the first load: "service_workers": "allow" lets it install, but its caching is not
compared. Say so when the site has one.
Apps behind sign-in (MSAL, OIDC or another identity provider). Without a session a single-page app
renders only its sign-in page, and following the sign-in redirect leaves the site. The least invasive
route is a captured session: a person signs in to a test account by hand with
python scripts/browser_harness.py capture-session --url <app> --out <file outside the repo>, and the
shared harness loads the file as storage_state into every engine's context. It carries cookies,
localStorage, IndexedDB and sessionStorage (put back on its origin before the app's scripts run), so an
MSAL cache in sessionStorage works too; the run stays read-only and blocks third parties, so a token
refresh fails: capture shortly before the run. The file holds live tokens: never commit it, never copy
its contents into runs.json, the report or a screenshot name, and delete it after the run; runs.json
records only counts. Without a session file, the app's own test sign-in or mock-auth mode is the
alternative. Never use a real user's account. Without a session, capture what renders
signed out (the sign-in page), set run_status to partial with the reason, and say in the
limitations which pages weren't compared because they need a session.
3. Static feature scan (with a repo)
python scripts/browserslist_lite.py --repo <repo> # see what the targets resolve to
python scripts/feature_scan.py --repo <repo> --out <run>/features.jsonUnresolvable browserslist terms (Firefox ESR, and, regional usage) are listed, and the scan can't
assess them (<browser> all, as in not op_mini all, is resolved). A Next.js project with no browserslist config compiles for Next's own documented
targets (Next 16: Chrome, Edge and Firefox 111, Safari 16.4), so those are used, and source says so.
A Vite project with no browserslist config compiles for its build.target: the scan reads it from
vite.config.* when it is a literal (['chrome100', 'safari15'], 'modules',
'baseline-widely-available'), else uses the Vite major's documented default (Vite 7:
baseline-widely-available, Chrome and Edge 107, Firefox 104, Safari 16; Vite 3-6: modules,
Chrome 87, Edge 88, Firefox 78, Safari 14), with iOS Safari following Safari; source says which.
esNNNN/esnext name no browser and are reported as not assessed.
The walk prunes node_modules, .next, coverage and caches, and skips what doesn't
ship to browsers: test and story files (*.test.*, *.spec.*, *.e2e.*, *.cy.*, *.stories.*),
test directories (e2e, test, tests, spec, specs, unless the directory holds a package
manifest such as package.json, which makes it a product package of that name) and Node-only
scripts (a node shebang, or an import of a node: module, fs, child_process and other Node
built-ins: CLIs, build scripts, dev servers). Server-only files are left out too and listed in
skipped.server_only with the reason: tool configuration (next.config.ts, eslint.config.mjs,
postcss.config.mjs, *.config.*, .*rc.js), type declarations (*.d.ts), modules that import
server-only or start with 'use server', and, from the shared Next.js reader (next_app.py), App
Router route handlers (route.ts), generated metadata routes (robots.ts, sitemap.ts,
opengraph-image.tsx, icon.tsx, manifest.ts), Pages Router API routes, middleware and the
instrumentation hook. Pages, layouts and server components are scanned: their markup and class names
reach the browser. A script written for both Node and the browser (an
environment check such as ENVIRONMENT_IS_NODE or typeof window, plus use of document, window,
navigator or WebAssembly: Emscripten glue, a UMD bundle) is browser code and is scanned. Build
output (dist/, build/, out/) is not pruned by name here: the shared census decides whether such a
folder is generated or a committed front end served as written, and every folder left out is listed in
skipped.pruned with its reason. --include DIR (repeatable) scans a folder a rule would leave out,
such as a hand-written dist/ that a FastAPI StaticFiles(directory="dist") mount, express.static
or send_from_directory serves; excluded folders that hold code are listed in
skipped.excluded_with_code with the file that serves them (served_by), and the warning names them
with the --include to use. A Python, Go or other server whose served folder was scanned gets
language_coverage.status: static_served (the server language is not browser code). A Node server that renders pages from template
literals still has its inline <script> and <style> blocks scanned, and so does an HTML page held
in a string constant of server-side code (a Go raw string for html/template, a Python or Java/Kotlin
triple-quoted string) or a template file (.tmpl, .gohtml, .jinja, .erb). Vendored and
third-party trees are left out (the shared lang_census.py: vendor/, third_party/, bundles, a
package with its own manifest and LICENSE under drivers/ or an SDK directory). When another language
dominates (a Go tool), features.json carries language_coverage and a limitation (also a warning on
stderr) saying what was and wasn't read: with no page in a string constant, 0 uses is not evidence,
so scan a rendered page instead (copy it outside the repository and pass that folder as --repo).
features.json lists what was left out under skipped; look at it when a finding you expected is
missing.
Server-rendered sites (a Flask or Django app rendering Jinja or Django templates named .html, Rails,
Laravel) are supported: their templates and stylesheets are the browser code, and they are scanned like
any page. When the repository is mostly Python, Ruby or PHP and the scanned markup includes templates
(HTML under templates/, views/, layouts/ or partials/, or with {% %}, {{ }}, <% %> or
@extends), language_coverage.status is server_rendered: it names the templates scanned, lists HTML
outside the template and static folders (markup_outside_templates, a stale rendered page or an old
build's output, which may not be served: check before reporting a use in one), and writes no language
limitation, because the server-side code is not browser code. scanned_files lists every file read
(count, and up to 300 paths).
Data directories are pruned before they are walked: a directory holding at least 200 files of which at
least 90% are data files (.json, .geojson, .csv, .tsv, .xml, .ndjson, .jsonl, .txt,
.yaml, .parquet, .avro) and none code or markup is listed in skipped.data_dirs with its file
count. --exclude DIR (repeatable, repository-relative) leaves out any other directory, such as an
archive of old builds (skipped.excluded).
JavaScript APIs, not syntax: a build transpiles syntax for its targets but doesn't polyfill APIs, so
the scan looks for APIs that newer targets have and older ones don't (structuredClone, .at(-1),
findLast, toSorted, Object.hasOwn, Object.groupBy, the new Set methods,
Promise.withResolvers, Array.fromAsync, iterator helpers, AbortSignal.any, the async Clipboard
API, which also needs a secure context, replaceAll, Intl.Segmenter, WeakRef, error cause), and
the APIs an app or game wrapper depends on (WebAssembly, SharedArrayBuffer, the Fullscreen API, which
iPhone Safari lacks on elements, Pointer Lock and its Chromium-only unadjustedMovement option, Keyboard
Lock, Gamepad, the Cache API, IndexedDB, Web Audio, touch events), with support from web-features. A
pattern found in the code that every target supports is listed in supported_features (no finding,
but checked), so an API absent from both lists is outside the pattern set: say so in the report rather
than reading the scan as complete. A polyfill the project loads (core-js) is not detected: say so when a
finding depends on it.
CSS-in-JS (Emotion, MUI's sx and styled, styled-components) generates its CSS at runtime, so
neither the source scan nor built CSS sees it. When the repo depends on such a library, the scan reads
the styles written literally in the scripts that use it: object styles (a camelCase property with a
string value such as textWrap: 'balance', a selector key such as '&:has(.x)', an @container
key) and tagged template literals (styled.div`...`, css`...`). Those uses are marked
via: "css-in-js", css_in_js records the libraries, and a limitation says that styles computed at
runtime were not scanned; the report must carry it.
Utility-class CSS (Tailwind) lives in class strings, not stylesheets. When the repo depends on
Tailwind, class strings are matched against the known non-baseline utilities (text-balance,
backdrop-blur-*, bg-clip-text, has-[...], @container, field-sizing-*, starting:), and
utility_css says only those were scanned. The CSS the framework generates (colour functions,
@property, layers) is scanned only from built CSS (--built-css <dir>, or .next/static/css
when it exists, in the repo or beside any next.config.* up to three levels down, for a Next.js
project kept in a subfolder and built with next build <dir>); otherwise features.json carries a limitation, which the report must repeat.
Minified built CSS has every rule on line 1, so its uses carry a column and repeats of one feature on
one line collapse into one entry with occurrences, kept apart by guard: the unguarded uses come
first (with guarded counting the uses of that line inside an @supports block), so the snippet is
the use that has no fallback; in built CSS only the @supports around a use counts as its guard. A
gradient's colour space (in oklab, as Tailwind 4's --tw-gradient-position writes) is
gradient-interpolation, not an oklab() colour. The built JavaScript (.next/static/chunks) is not
scanned: APIs are checked in the source, so polyfills the build adds are not seen; features.json
says so in a limitation and lists .next as build output whose CSS was read. A finding whose pattern
degrades harmlessly (::-webkit- pseudo-elements, as in Tailwind's preflight reset) carries
degrades; say it is a progressive enhancement. Tailwind 4 registers its variables with
@property and emits its own fallback for engines without it (@layer properties { @supports (...)
{ *, ::before, ::after { --tw-...: initial } } }); when built CSS has that block, its @property
uses count as having a fallback. Tailwind 4 itself requires Chrome and Edge 111, Firefox 128 and
Safari 16.4: when a declared target is below that floor, utility_css.targets_below_floor lists it
and a limitation says the project's real support floor is ambiguous.
4. Compare engines
python scripts/compare_engines.py <run>/capture/runs.json --features <run>/features.json --out <run>/candidates.json [--jobs 4]Only captured runs are compared; comparisons lists every planned engine-versus-reference pair and
why any wasn't made (an errored or skipped run on either side). Differences: page and console errors, failed steps, failed requests and undecoded images, missing or
extra elements, input type fallbacks, box differences, overflow, and content not rendered (an element
with under 40% of the reference's ink). Font metrics, native form controls (their size change is
judged apart from a move: a control that also moved is judged as a move), line wraps (a row of items
under one parent, such as nav links, that keep their order but wrap onto another line in one engine,
with no item resized beyond font metrics: the whole row and what moved with it is one dismissal),
consequences of other differences (a box that grew because something inside it changed, an element
below it that moved), cumulative drift (an element that moved with the content just above it, by text
metrics adding up down a long page) and ink measured on boxes of different shape (an inline element
on another number of lines, or a box whose area changed more than twofold), boxes of an element
another element covers (occluded: an overlay or sticky panel lies over it) and boxes inside a panel
the automation scrolled by a different amount in each engine (moved by exactly that offset), inline
elements that moved inside a parent block that kept its size (inline reflow: a link in a paragraph whose
text breaks at another word), and every layout difference of a run laid out at another effective
viewport (capture records inner and fullscreen after the steps: a "click to play" that enters
fullscreen resizes some engines to the screen) are dismissed automatically and counted, with reasons in
candidates.json. Request failures that are an engine's normal behaviour (a media request cancelled
with net::ERR_ABORTED once it has the metadata, an image request the page cancelled when it swapped a
srcset candidate or a /_next/image preload, Firefox's NS_ERROR_PARSED_DATA_CACHED) are not
compared; benign_requests lists them. One failed request on several pages is one candidate. When the
reference shows almost no ink on an element (under 2%: a video on a near-uniform frame), the other
engines are compared with each other, so a video blank in one engine only is still caught (its
reference is then the engine that drew it). A step that fails, or a page error thrown, in every run of
every engine is no difference, but the state was not exercised: same_everywhere lists it (and a
warning), and the report must name that page or state in a limitation (the validator checks) and hand
the bug to a crash hunt. A layout difference of one element (by
its signature, such as a shared header or footer) on several pages is one candidate: pages lists
them and reach counts them. Repeated siblings are one component: :nth-of-type(N) and :nth-child(N)
are taken out of the grouping key, so 40 cards with the same undecoded image are one candidate, whose
elements lists them. A cause seen at several viewports is one candidate too (viewports), and every
candidate counts the raw differences it stands for (occurrences). Console messages are compared one
message at a time (without the source position), so one message on many pages is one candidate. A
console message for an HTTP error response ("Failed to load resource: the server responded with a
status of 400") logged in one engine only is dismissed when every run of both engines recorded that
response status (http_errors, written by capture.py): the request failed alike, only the logging
differs (Firefox does not log fetch() error responses). Text whose computed font family differs between engines in every run
(a control that doesn't inherit the page font is drawn in the engine's own default face) is one
font_family_differs difference per pair of families, listing its elements; Playwright WebKit on
Windows draws such text in a serif face (webkit-windows-serif-controls, matched on the face the
engine drew, never on "serif" inside the reference's ui-sans-serif), so that one goes to needs
review until a real Safari confirms it. When a stylesheet failed to load in every run of the engine,
the detail says so and no limitation is matched: fix the request first. Text with the same computed
family drawn in another face (the faces widths differ: Windows WebKit resolves system-ui and
ui-sans-serif to another face than Chromium and Firefox) is a font_family_differs too, with the
webkit-windows-system-ui limitation, and the boxes it resized inherit it. Low ink on an <img> the
engine decoded and drew (5% ink or more) is dismissed as photographic content, and an SVG shape whose
box grew evenly by its stroke width (Firefox includes the stroke) is dismissed as SVG stroke. When comparisons were made and nothing at all came out (no candidate,
flaky entry or dismissal), candidates.json carries a self_check line saying how many element
boxes were compared and how many were identical: open one screenshot pair per engine and confirm the
pages really rendered alike before reporting no differences. Each screenshot is decoded once, in
parallel (--jobs, default up to 4; the validator takes the same option). Candidates matching a known
engine-build limitation carry its id in build_limitations, and so do the candidates that follow from an
undecoded image with a known limitation on the same page and engine (the <img> an error handler hides, the
fallback it shows beside it, a box resized around it): their detail says which image. A box or ink
difference of an element listed in a font_family_differs candidate with a build limitation (a key input
narrower because Windows WebKit draws it in MS Shell Dlg) inherits that limitation too. The Chromium
codec limitation is not matched by an .mp4 URL with ERR_ABORTED alone (not_if). Flag emoji are drawn
as letters by Chromium and WebKit on Windows and as colour flags by Firefox (windows-flag-emoji), and
Windows WebKit may draw default-face controls in MS Shell Dlg rather than a serif
(webkit-windows-serif-controls). Ratings follow RUBRIC.md.
5. Triage and write
Every candidate is promoted, needs_review (for example media that may play in branded browsers with
proprietary codecs, and every candidate with build_limitations), or dismissed with a checkable
reason. Each triage entry is {candidate_id, decision, difference_id, reason}: difference_id
names the difference a promoted candidate became and is null for needs_review and dismissed.
The reason says what was compared and what it showed ("the WebKit screenshot shows the label on one
line, wider by 6 px; nothing is hidden"). A box difference that repeats and shows in the screenshots
(a list wrapping at another word because its markers are narrower) can be promoted like any other:
its runs record that it differed in every run of the engine and in none of the reference's. a reason under 25 characters, or only "false positive",
"font metrics", "by design" and the like, fails validation. For a build limitation, the needs-review reason names it and what to check in the real
browser (its confirm text in data/engine-limitations.json); in the summary, say that WebKit
showed it and real Safari wasn't checked, never that Safari has it. For each difference write a title naming the
engines (never "Safari"), a remediation that addresses the cause (a feature check, a fallback, an
@supports block), and a verification step (rerun capture for that page and name what should pass).
When a site serves different scripts per engine (compare scripts across engines), say so for every
difference on that page.
A feature finding is possible, not a defect: say when it is a progressive enhancement that degrades
harmlessly (text-wrap: balance wraps normally, ::-webkit-scrollbar keeps default scrollbars)
rather than leaving it to read as a break.
6. Validate and render
python scripts/validate_report.py <run>/report.json --runs <run>/capture/runs.json --candidates <run>/candidates.json \
--features <run>/features.json --targets targets.json [--write-summary]
python scripts/render_report.py <run>/report.json [--brief]--brief shows the differences, the needs-review items and the limitations. The validator checks
feature_findings against the --features file passed (use for use), not only against the copy in
candidates.json.
Static only (static_only, or when no engine can run): skip the capture and write the
candidates file of the feature scan alone, then a report with run_status: "not_run", a
run_status_reason, every engine not_run, no differences, triage or flaky entries, runs_file:
null, verdict inconclusive, and a summary and limitation that say no engine ran:
python scripts/compare_engines.py --static-only --features <run>/features.json [--targets targets.json] --out <run>/candidates.json
python scripts/validate_report.py <run>/report.json --candidates <run>/candidates.json --features <run>/features.json [--targets targets.json] [--write-summary]--targets records the checks that were planned and not made; without it the plan is empty, and the
verdict is still inconclusive. examples/report.static.example.json is a static-only report.
Commercial device clouds (BrowserStack, Sauce Labs) aren't integrated; with an account, run the same targets there and compare by hand.
Output fields
scope: site, pages, viewports, engines with versions and status (ran,not_run,errored), reference engine, resolved browserslist,out_of_scope, reruns,authorised_by,third_party(as incandidates.json; required when the capture recorded it), andcapture,requestsandharness(copies ofruns.json's, required when the capture recorded them;harnessis the shared harness'smetadata()per engine: CSP bypass, read-only mode, request totals, retry policy).run_status(complete,partial,not_run) andrun_status_reason(unless complete): whether the capture ran in full;not_runis a static-only report.metadata: tools, data versions and hashes, runs file (nullwhen static only).headline: verdict (differences_found,no_differences_detected,inconclusive), summary, severity counts, flaky count, checks planned and run (checks.plannedandchecks.rancount runs: page or state × viewport × engine × rerun), engine comparisons planned and made (page or state × viewport × each engine other than the reference), evidence completeness (completeorpartial: partial when an engine or run didn't run, a comparison wasn't made, or anything is flaky, needs review or not assessed), overall confidence. Write the verdict and summary;--write-summarycomputes and fills the other fields, which a draft may leave out, and never changes the verdict. With every comparison made and nothing promoted,inconclusiveis accepted as the more cautious verdict (candidates held for review, a flaky page) when the summary says why.differences[]: kind and subkind, engines affected, reference, page and states, element, severity, confidence, priority, score, usage share, runs per state and engine, evidence per engine, linked feature cause, remediation and verification step.feature_findings[],flaky[],dismissed_automatically,needs_review[],triage[],limitations[].
What the validator enforces
Candidates, ratings, flaky list and feature links are recomputed from the recorded runs. Every
difference is consistent across its reruns, differs from the reference, and has screenshots (matching
their hashes) from both sides. Safari is never an engine or a title, and a summary sentence that
names Safari must say it wasn't tested (sentences split at a full stop and a space, so "Safari 16.4"
is one sentence). No compatibility claims. The verdict follows from the runs: differences_found
with a promoted difference; inconclusive whenever a planned comparison wasn't made, an engine didn't
run, or no comparison was made at all (an empty plan included), and nothing was promoted; otherwise
no_differences_detected or inconclusive; never no_differences_detected with zero
comparisons; an inconclusive summary must say so ("inconclusive", "not compared", "not made", "no
engine ran", "static only") and not claim "no differences", and a limitation must name what wasn't
compared. run_status, when given, agrees: complete only when every planned run ran and every
comparison was made; a not_run report is validated without --runs, from a --static-only
candidates file, with every engine not_run and no differences, triage or confirmed features. Confidence is low when no comparison, or under half of
them or of the runs, was made. scope.capture, scope.requests and scope.harness equal runs.json's; forced
reduced motion or frozen animations need out_of_scope to say motion wasn't compared; the feature
scan's limitations (utility CSS not scanned) must appear in the report's. Every
candidate is triaged once, with a reason that says what was checked, and none with
build_limitations is promoted. The schema is checked first: a schema error stops before any
screenshot is decoded. Checks planned come from
runs.json's recorded plan, --targets (which must agree with it), or for an older capture the
page and viewport pairs that ran, so the count is the same with and without --targets. Blocked
third-party requests need scope.third_party and a limitation saying they weren't compared. Counts the summary states ("3 need review", "5 differences", "2 flaky")
must equal the report's own lists. An environment cause in candidates.json (requests upgraded to
https on a plain-http target) must be named in a limitation, and screened_runs and environment
must follow from runs.json. A step or page error that failed alike in every engine
(same_everywhere in candidates.json) must be named, by its page or state, in a limitation or a
needs-review reason.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

