agentleFS
marksmayomarksmayo/qa-skills/wcag-auditor-skill/SKILL.md

wcag-auditor

Skill1 starsChanged yesterdayMITView on GitHub

Audit live web pages against WCAG (2.2 AA by default) without needing the source code. Runs three open-source engines inside a real browser (axe-core, IBM Equal Access, HTML_CodeSniffer), adds keyboard, focus, reflow, text-spacing, zoom, target-size, hover, motion, language and media probes, merges and de-duplicates the results by component, maps every issue to a success criterion, rates severity with a published rubric, and gives every SC at the target level an EARL outcome. Untested criteria become a manual test plan. The report never says the site conforms.

  • Installs packages

3 other projects use a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

9 rules. 0 are common, 2 are distinctive, and 7 are very specific to this project.

  • 0

    Common

    Most of the 3 edited versions we read keep these.

  • 2

    Distinctive

    This author's own choices. Worth a look.

  • 7

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    A success-criterion matrix: Every SC at the target level gets passed, failed, cantTell, inapplicable or untested, with the evidence for it.

    line 22
  2. Very specific:

    Test only what you are allowed to test: Before scanning, confirm the user owns the target or is authorised to test it, and record who …

    line 50
  3. Distinctive:

    If something essential is missing (the target, or authorisation for a non-loopback target), ask once, then proceed with the defaults.

    line 94
  4. Distinctive:

    Every candidate gets exactly one triage entry:

    line 229
  5. Very specific:

    Every SC at the target version and level gets one sc_results row.

    line 296
  6. Very specific:

    Every unreachable or blocked sample page is named in limitations.

    line 339
  7. Very specific:

    Strict CSP (Next.js and similar): leave bypass_csp on; with it off, expect engines and probes to be errored on pages whose CSP refuses …

    line 351
  8. Very specific:

    CAPTCHAs: stop, record the page as blocked, report affected SCs as untested.

    line 353
  9. Very specific:

    Login: use a captured session (capture-session) or a login recipe with env fields.

    line 365

“Common” is measured: kept by at least 6 in 10 of the 3 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

WCAG Auditor

Mission

Find the accessibility barriers on a website that real people will hit, explain who they hurt and how to fix them, and say plainly how much of WCAG the evidence actually covers.

What sets this apart from running one checker:

  • Three engines and a set of browser probes. axe-core, IBM Equal Access and HTML_CodeSniffer run inside the page; the probes test what engines can't: Tab traversal and traps, focus visibility and obstruction, reflow at 320 px, text spacing, 200% zoom, target size, hover content, motion and page language.
  • One issue, one finding. Results from every tool, viewport and state are merged per element, then per component, so a broken shared header is one finding listing every page.
  • A success-criterion matrix. Every SC at the target level gets passed, failed, cantTell, inapplicable or untested, with the evidence for it.
  • Honest about the limits. Automated tools can check only part of WCAG. The SCs nobody could test become a manual test plan with concrete steps.

Non-negotiable rules

  1. Never say the site conforms. WCAG conformance can't be established by automated and assisted testing. The verdict is does_not_conform (at least one high-confidence A or AA failure) or not_established. Never write "compliant", "conforms" or "accessible" as a conclusion. The best possible result is "No failures detected by automated and assisted testing; conformance not established."
  2. Use the EARL outcomes and never inflate them. An SC with no evidence is untested, never passed. A passing tool rule doesn't pass the SC: if axe passes color-contrast, only the text it could compute passed. An SC is passed only with probe, agent-review or manual evidence for the whole criterion on the sampled pages, and an SC no tool can check (automatable: none) needs agent-review or manual evidence.
  3. "Needs review" stays cantTell. axe incomplete, IBM POTENTIAL/MANUAL and HTMLCS warnings are never counted as failures and never discarded. Each one is promoted (with evidence), turned into a needs-review item, or dismissed with a reason someone can check.
  4. Best practice is not a WCAG failure. axe best-practice rules, IBM recommendations, and SCs above the target level or outside the target version are advisories.
  5. Every finding records where it came from: tool, version, rule id, ACT rule ids, selector, HTML snippet, URL, viewport, state and screenshot.
  6. Agent judgment gets a lower confidence. Alt-text quality, whether an error message helps, whether a heading is really a heading: tag these evidence_type: agent_judgment, rate them medium at most (low when nothing corroborates them), and say in resolution what you looked at.
  7. Test only what you are allowed to test. Before scanning, confirm the user owns the target or is authorised to test it, and record who confirmed it (authorised_by). Loopback targets need no confirmation. Respect rate limits; back off on 429 and 503.
  8. Change nothing. Form submission is off by default: every non-GET request is aborted. Third-party requests are blocked too (GETs included), so an audit never sends analytics hits or ad requests; service workers are blocked. Don't click destructive buttons, don't bypass CAPTCHAs (record the page as blocked), and never write credentials into any output.
  9. A probe that couldn't run is not a pass. The runner injects the engines, its probe helpers and a text-spacing stylesheet; a Content-Security-Policy with a hash or nonce refuses them silently. Pages therefore run with the CSP bypassed (bypass_csp, default true; it changes nothing for real users), every injection is checked, and one that didn't take makes that engine or probe errored, never "ran, nothing found". An SC passed on probe evidence needs its probe to have run on every planned check (the validator enforces it with --run). A probe cut short (partial: the keyboard traversal stopped at its press limit) is not "ran": it lowers evidence completeness, keeps overall confidence below high, blocks probe-based passes and must be named in limitations as partial. When a large focus- or hover-opened menu dominates the Tab order, add a state that closes it, or raise keyboard.max_presses, so the rest of the page is measured.

Inputs

InputDefaultNotes
targetsrequiredURLs, a sitemap URL, or a start URL plus a crawl limit. A target may be {"url": ..., "expect_status": 404}: an error template users really reach (the not-found page) is then audited like any page, with its status recorded in the sample; any other 4xx/5xx answer makes a page unreachable
wcag_version2.2Also 2.1 and 2.0. In 2.2, 4.1.1 Parsing is obsolete and recorded as inapplicable
levelAAA, AA or AAA. Requirements above the target are advisories
critical_pathsoptionalNamed flows as steps (url, optional state, label), or journey files from journey-mapper
authoptional{"storage_state": file}: a session a person captured by signing in by hand (python scripts/browser_harness.py capture-session --url <app> --out <file outside the repo>; its sessionStorage is restored too), or a login recipe whose fields read credentials from environment variables; run.json records only which kind was used
viewports1280x800, 320x640320 px also covers 1.4.10 Reflow
mobile_viewportsnoneViewports (also in viewports) audited as a touch phone: is_mobile, has_touch, coarse pointer
max_pages25Sampling cap for crawled and sitemap pages; listed targets are always audited
crawl.max_loads4 x max_pagesMost pages the crawl loads; it stops sooner once max_pages templates are found or max_pages loads in a row add no new template
crawl.per_pattern2Pages of one URL shape (/region/*, numbers and query values as placeholders) the crawl loads while links of an unseen shape wait, so a menu of hundreds of links of one shape on every page doesn't take every load before the footer's pages; run.json crawl.patterns counts the loads per shape and crawl.shapes_from says where the shapes came from
crawl.routes / crawl.next_appnoneThe app's route patterns (["/", "/about", "/{slug}"], [id] and :id also read), or a Next.js project folder whose page routes next_app.py reads: a URL takes the static route it equals, else the dynamic route it matches, so pages under a root-level dynamic segment (app/[slug]/page) share one shape while /about keeps its own. Without them a path rule can't tell /about from /heron-lake: once more than crawl.fold_root_after (8) root-level siblings are seen they all fold into /*, so list the root pages that are separate templates as targets
keyboard.max_presses2000Hard cap on Tab presses per keyboard traversal. The budget starts at 3 per visible focusable element (+20, at most 400) and grows by 20 each time a press reaches a new element, so a menu that focus reveals (hundreds of links) doesn't end the traversal early
settle_seconds10Most time spent waiting for each load to settle (network quiet for 500 ms, then a quiet DOM). Event streams, websockets, long-poll and hot-reload requests, and any request open longer than long_lived_seconds (4), don't hold it open; events names them
random_sample0.1Share of the remaining crawled pages added at random (seeded); 0 adds none
third_partyblockEvery request to a host that isn't loopback, a target's host or in allowed_hosts is aborted and logged. Third-party content (embeds, fonts, widgets from other hosts) is then absent: say so in limitations, or let a host through
allowed_hostsnoneFurther hosts to let through, such as a CDN the site needs ("*.cdn.example" allowed)
service_workersblockallow lets the site register them
allow_form_submissionfalseOnly the user can turn it on, for a test environment
bypass_csptrueRun pages with their CSP bypassed so engines and probes can be injected; false audits the CSP as served (injections it refuses are recorded as errored)
consentnoneAn overlay to dismiss (dismiss selector or steps), engines both (default), before or after, and probes_with_overlay (default ["keyboard"])
standards_mapoffen301549 adds EN 301 549 clause numbers (V3.2.1; WCAG 2.2 additions have none)

If something essential is missing (the target, or authorisation for a non-loopback target), ask once, then proceed with the defaults.

Workflow (WCAG-EM)

1. Define the scope

Record the version, level, targets, exclusions (third-party embeds, PDFs, out-of-scope subdomains) and critical paths. Write audit_config.json (python scripts/run_audit.py --help lists every field). Use states for UI that only appears after an action (open menus, dialogs, accordions, form errors), and reference them from critical-path steps.

2. Check tooling

pip install -r requirements.txt && python -m playwright install chromium   # ask first
python scripts/fetch_engines.py                       # pinned engines, SHA-512 and SHA-256 verified
python scripts/fetch_engines.py --include-optional    # also the Nu HTML Checker (needs Java; its jar SHA-256 is not pinned, so the pinned tarball is cached and re-checked)

Each engine is verified against data/engines.lock.json before it is cached, and again before it is injected; a file that fails either check is never used. A missing or failed engine is recorded as not_run or errored with the reason, lowers evidence completeness, and must be named in limitations. If installs aren't allowed, say what can't be done; don't simulate it.

3. Explore and sample

python scripts/run_audit.py --config audit_config.json --out <run-dir> --plan-only

The runner crawls (or reads the sitemap), printing each page it loads, groups pages into templates by URL pattern, page-level landmarks and the kinds of element in main (content landmarks such as regions, labelled sections, forms and dialogs, and repeat counts, are left out, so a docs page with four code-block regions shares a template with one that has none), and samples every critical-path page and every listed target (never dropped for sharing a template), one page per template not yet covered, and about 10% more of the rest at random (seeded). Each page records how it entered the sample (critical_path, listed, structured, random, or unreachable) and the HTTP status it answered. Check the sample makes sense before the full run; add targets for templates the crawl missed. A plan-only run records every engine as not_run with the reason "plan only": that is expected, not a failure. The crawl loads at most crawl.per_pattern pages of one URL shape while links of an unseen shape wait (listed targets always come first). The shape folds only the last segment of a nested path (/region/*); give crawl.routes or crawl.next_app for root-level dynamic routes. List the not-found page (any unknown URL) as a target with expect_status: 404 so its template is audited; an unreachable page is never audited, and the report must name it in limitations. Responses that aren't HTML (a linked .txt, XML or JSON file) are skipped, not sampled; run.json lists them under crawl.skipped_non_html with why the crawl stopped.

4. Run engines and probes

python scripts/run_audit.py --config audit_config.json --out <run-dir>

For every sampled page and viewport the runner takes a full-page screenshot and runs the engines with any consent banner or overlay showing (it's content too). With consent configured it then runs probes_with_overlay (the keyboard probe by default: does focus enter the overlay, is focus hidden behind it?) with it open (state consent_open), dismisses it, runs the engines again on the uncovered page (state consent_dismissed, unless engines is before), and runs the probes; run.json records which ran in each state. The motion probe watches fresh loads in their own pages, so the audited page is never reloaded under later probes.

ProbeSCsWhat it measures
keyboard2.1.1, 2.1.2, 2.4.3Tab order from the top of the document (an invisible starting point focused at the start of the body; nothing is clicked, so the first Tab reaches a skip link even under a fixed header); traps (no new element for 15 presses, then Shift+Tab and Escape tried); clickable elements that aren't focusable, with their click listeners; menu links that only :hover reveals and no key opens (focus, ArrowDown, and Enter and Space unless the control follows a link or submits a form). Clickable heading text whose heading holds a focusable # anchor link is not reported; a clickable element with a focusable sibling link or button is needs review, not failed. In a named state with a modal open and focus inside it, Tab starts from the focused element and only the modal's controls are expected (start: active_element); a state with "keyboard_start": "active_element" starts from wherever its actions left focus (after a client-side navigation: does the next Tab reach the new page's content?), and from the top, with a start_note, when nothing was focused (a start element focused by script with tabindex="-1" is not a Tab stop, so its indicator is not measured). A traversal that hits its press limit (keyboard.max_presses) before a full pass is partial, with a reason: its stops are evidence, what it never reached is not reported as unreachable, and the keyboard and focus checks count as not run
focus_visible2.4.7, 2.4.11, 2.4.12Pixels changed around each element when focused, against its perimeter (an inline link that wraps is measured per line box, line_boxes); whether fixed or sticky content covers it (a fixed header that contains the focused control doesn't count); focus guards (data-*focus-guard, 0 x 0 aria-hidden sentinels) are not measured. Stops with no or a weak indicator keep their focused and unfocused crops (indicator.screenshot, screenshot_unfocused, up to 30 per traversal): look at them before rating
reflow1.4.10Horizontal scrolling at 320 px and the element causing it (tables, video and other two-dimensional content flagged as possibly exempt); content past the viewport on a page that can't scroll sideways (cut off: reflow.clipped, failed), or cut off by an ancestor with overflow hidden (needs review)
text_spacing1.4.12Text clipped once the WCAG spacing overrides are applied (visually hidden .sr-only text excluded); the overrides are read back, and a refused stylesheet makes the probe errored
resize_text1.4.4Text clipped at 200% zoom (half the viewport at device scale 2; visually hidden text excluded); viewport meta that blocks zoom
target_size2.5.8Pointer targets (links, buttons, controls, ARIA widgets; never a tabindex container or a visually hidden link) under 24 x 24 px whose 24 px circle overlaps another target, measured at the top of the page; fixed or sticky targets are compared only with each other; inline links exempt
hover_focus1.4.13Content shown on hover: dismissible with Escape, hoverable, persistent. Anything that appears inside the hovered element's own box (an icon, a tint) is not additional content and is only counted (own_box_changes). Visually hidden elements (an .sr-only skip link) are not candidates; an element the pointer can't reach (covered by another) is named in skipped and makes the probe partial
motion2.2.2Content still changing 5 to 7.5 s after load with no pause control; again with reduced motion
a11y_tree1.3.1, 2.4.6, 4.1.2Accessibility-tree snapshot, headings and landmarks for the assisted review
language3.1.1lang present, valid, and matching the language of the text
media1.2.x, 1.4.2Video and audio elements, tracks, autoplay; embedded players

States from the config run the engines again, plus any probes listed for them. Each state records where focus was once its actions finished (focus_after_actions in its raw file: selector, snippet and a crop; null when nothing but the body was focused) and the URL it ended on (final_url); a {"press": "Tab"} state shows there which element it really focused. A state that navigates client-side to another sampled page (a framework <Link>) has its results filed under that page, with via naming the page and state it started from; one that ends on an unsampled URL keeps its page and says where it ended (ended_on). CAPTCHAs stop the page (blocked); cross-origin frames the engines can't reach are untested. The runner prints one progress line per probe ([p01__1280x800] keyboard: ran in 27s).

Full-page screenshots show only the first box of a panel that scrolls its own content (an app shell of height 100vh with a scrolling list and detail panel). The runner records such panels per page (inner_scroll) and writes the limitation into run.json limitations (also printed as a warning): copy it into the report's limitations, scroll the panels and check what they hide by hand, and don't cite a full-page screenshot as showing content below a panel's first box. When third-party requests were blocked, run.json limitations also names the hosts and counts: copy that too.

Pages without page-level landmarks are grouped into templates by file name and DOM shape, not by folder, so one generated page copied into many folders (a report.html per CI run) is one template.

5. Normalise and merge

python scripts/normalize.py <run-dir> --out <run-dir>/items.json
python scripts/merge_findings.py <run-dir>/items.json --run <run-dir> --out <run-dir>/candidates.json

normalize.py maps every result to SCs and ACT rules through data/rule-map.json, and classifies it as wcag, best_practice, above_level, outside_version, obsolete or experimental. The map is generated from the engines' own metadata; its overrides table corrects the few entries that metadata gets wrong (IBM img_alt_redundant is best practice, like axe's image-redundant-alt, not a 1.1.1 failure) and names the failure pattern a rule implies where its SC has several (axe link-in-text-block is a colour-only link). IBM's citation of ACT 7d6734 is kept only for an svg with an explicit graphics or img role, the rule's applicability. merge_findings.py merges results about the same element (or an image and its link) across tools, viewports and states, then groups them by component signature across pages. Each candidate carries its sources, outcome, a suggested confidence and the default failure pattern: the one its rule implies, or the only pattern of its SCs. When the rule implies none and the SC has several, pattern is null and pattern_choices lists them with their default severities: choose the one that matches what you see (never the most severe by default). sc_draft gives each SC a draft outcome; it never drafts passed.

Contrast "needs review" results (axe incomplete, HTMLCS .Alpha/.BgImage/.Abs, IBM potentials) can bury the real failures when a page has a faint background gradient. The runner records what lies behind each such element: the background image and the element carrying it, translucent layers, and the text colour and size. merge_findings.py then gives each candidate a contrast_estimate (the worst-case ratio against the gradient's colour stops, which bound a gradient's luminance) and collapses candidates with the same cause and the same background into one grouped candidate whose group lists the cause, the background, the member components, counts, text colours and sample selectors. Candidates estimated below the threshold are never grouped; they stay separate so they can be promoted. Visually hidden text is grouped on its own. Runs without that record (older runs) are not grouped. Colours in any CSS syntax are converted to sRGB (rgb(), hex, hsl(), hwb(), lab(), lch(), oklab(), oklch(), color(srgb …), display-p3, xyz; Tailwind 4 writes oklch() and Chromium reports some colours as lab()); one it can't parse gives no estimate rather than a wrong one (python scripts/merge_findings.py --self-test checks the conversions against known ratios). Text over an <img>, <video>, <canvas> or <svg> painted behind it has an unknown background: its estimate has no ratio (unknown_background), and the candidate is never grouped.

6. Triage every candidate

Every candidate gets exactly one triage entry:

  • finding: a WCAG A/AA failure you are confident of. One root cause is one finding, even when it spans several candidates and SCs (an unnamed logo link reported as both 1.1.1 and 2.4.4).
  • needs_review: a cantTell that a person must check, with what_to_check. A failed candidate kept as needs review needs a reason saying why the failure is not confirmed (what you looked at); repeating the needs-review text or the tool message is rejected.
  • advisory: best practice, above the target level, outside the target version, or obsolete (4.1.1 in WCAG 2.2): the advisory kind follows the candidate's kind.
  • dismissed: with a reason someone can check, such as the computed contrast ratio and the colours it came from, the probe measurement that contradicts the hint, or the exemption that applies. "False positive" alone is rejected.

A grouped contrast candidate is triaged once, like any other. A contrast_estimate comes from recorded colours, not pixels: dismiss on it only when the group's lowest estimate clears the threshold and you say so with the colours (#222 on a #fafafa-#eef4ee gradient: at least 14.2:1); otherwise send it for review with the sample selectors. Check a sample in the screenshot either way, because images, shadows and overlapping content aren't in the estimate; when the text colour isn't plain rgb() (an oklch() or lab() theme), check the screenshot pixels first. A contrast candidate with an estimate below the threshold, an unknown background, or a group with no estimate is never dismissed (the validator rejects it).

A finding built only from cantTell candidates is agent judgment and needs a resolution saying what you looked at.

A pointer_only keyboard candidate can be dismissed when a keyboard alternative elsewhere on the page does the same thing (a page-level shortcut that selects the item): name the key and what it did when you pressed it, so the reason can be checked.

Generated ids that differ only in numbers (flow-3-cmd-12-d2, row-7) are left out of the component signature, so one pattern repeated per row is one candidate.

7. Assisted review

Using the screenshots, the accessibility-tree snapshots and the page itself, review the SCs that need judgment: 1.1.1 alt-text quality, 1.3.1 visual structure against markup, 1.3.5 autocomplete, 1.4.1 colour-only meaning, 2.4.2 and 2.4.6 descriptive titles, headings and labels, 2.4.4 link purpose, 3.2.3/3.2.4/3.2.6 consistency across pages, and 3.3.1/3.3.2 errors and labels (use the form-error state). List what you reviewed in assisted_review.scs_reviewed; every SC passed or failed with agent_review in its basis must be listed there. Findings from this step are agent_judgment. When the site's source is available and a finding rests on reading it (a keydown handler, a template), give it a basis: [{"file", "line", "excerpt"}], paths relative to the source root, and validate with --repo <source> so each excerpt is re-read on its line. An agent_judgment finding whose text cites file:line without a basis for it is rejected.

8. Rate

Follow RUBRIC.md:

  • Severity starts at the default of the matching failure pattern in data/wcag-sc.json. It goes up one level when the finding is on a critical-path step and affects what that step needs, and it is always Critical for the non-interference SCs (1.4.2, 2.1.2, 2.2.2, 2.3.1). To deviate from the default, give severity_override_reason. Tool impact labels are inputs only.
  • Confidence: high needs two agreeing failing sources or a failing rule that implements an ACT rule; one heuristic engine or one probe is medium; agent judgment is medium at most. A failed SC's confidence is never above the most confident finding that fails it.
  • Critical path, precisely: a finding on a critical-path step that does not affect what the step needs (an unnamed progress bar that doesn't stop anyone) keeps its critical_path with "affects_step": false and a reason; it is then not raised a level and can't block the path.
  • Priority = severity weight x reach x leverage, with floors; see RUBRIC.md. Set blocks_critical_path when a user group can't complete the step at all; that makes it P0. An agent-judgment finding is at most P1 unless it blocks a critical path and can be re-checked: a basis (re-read with --repo) or a candidate it rests on (a tool's needs-review result).

9. Build the SC matrix and the manual test plan

Every SC at the target version and level gets one sc_results row. failed lists its findings; cantTell lists its needs-review items; inapplicable says why (no media, no authentication); passed says what evidence covers the whole criterion; everything else is untested and goes in the manual_test_plan with concrete steps and pages. A pass whose evidence is a probe's output (the keyboard probe, reflow, text spacing) has probe in its basis, so the validator can check the probe ran on every planned check: an agent_review pass whose evidence note cites probe results is rejected.

10. Report, validate and render

Write audit.json (described by schemas/audit.schema.json; its schema_version is "1.5"; "1.0" to "1.4" reports still validate; 1.3 adds the check status partial; 1.4 adds check rows for named states (state), critical_path.affects_step, metadata.untestable_surfaces and metadata.mobile_viewports; 1.5 adds an instance's via and ended_on and a sample page's status and expected_status). Copy metadata.blocked_requests and metadata.events from run.json as they are recorded there (lists of {method, url, reason, at} and {url, event, at} objects; plain strings are accepted too), or blocked_requests as its count; --run checks the count either way. Copy each instance's via or ended_on from its candidate. Save the report next to the run directory, then:

python scripts/validate_audit.py audit.json --run <run-dir> --candidates <run-dir>/candidates.json --write-summary [--repo <site source>]
python scripts/render_report.py audit.json [--brief] [--markdown audit.md]

--write-summary fills or recomputes the headline numbers before anything is checked, so the draft can leave them out; it never writes the verdict or its text. Fix every problem the validator reports, then give the user the rendered report: the verdict and its reason, the P0 and P1 findings, blocked critical paths, what was untested and the limitations.

Report fields that the validator checks

  • The verdict follows the evidence and the verdict text claims no conformance.
  • sc_results has every target SC exactly once, matching data/wcag-sc.json; failed SCs and findings agree; no tool-only passes; untested SCs are in the manual plan; cantTell SCs have needs-review items; 4.1.1 is inapplicable in 2.2.
  • Findings: SCs inside the target; a valid failure pattern; severity from the rubric; confidence rules; priority, reach and leverage recomputed; priority order; critical-path steps that exist and match the finding's pages; instances on sampled pages with screenshots that exist.
  • Triage: every candidate exactly once; references that exist and point back; checkable dismissals; no best-practice candidate promoted to a WCAG finding; finding sources come from their candidates.
  • Headline: severity counts, SC coverage, evidence completeness (ran / planned checks), overall confidence and blocked critical paths recomputed.
  • With --run: tool statuses, versions, the check matrix, the sample and the browser version match run.json, so nothing claims to have run that didn't; an SC passed on probe evidence had its probes run on every planned check of the pages that were reached (a page that was never reached is a missing page, named in limitations, not a probe that failed).
  • Every unreachable or blocked sample page is named in limitations; a page listed with expect_status answered that status.
  • An agent_review pass doesn't rest on probe results (see step 9).
  • With --candidates: no contrast candidate below the threshold, over an image or video, or grouped without an estimate is dismissed.

Edge cases

  • Consent banners and overlays: engines run with the overlay showing and again after consent.dismiss (or consent.steps) closes it; the keyboard probe also runs with it open; the other probes and states run after dismissal. A dismissal that fails is recorded (consent_dismissed: false) and the engines are not rerun.
  • Strict CSP (Next.js and similar): leave bypass_csp on; with it off, expect engines and probes to be errored on pages whose CSP refuses injected scripts or styles.
  • CAPTCHAs: stop, record the page as blocked, report affected SCs as untested.
  • SPAs: the runner waits for network idle and a quiet DOM; add wait_for actions for slow views. For focus management after a client-side navigation, add a state that clicks the link, with "keyboard_start": "active_element" and the keyboard probe: focus_after_actions shows where focus landed and the traversal shows what the next Tab reaches. A focus manager may move focus on the first load too, so a {"press": "Tab"} state can land past the skip link: check focus_after_actions.
  • Error templates: list a URL that does not exist as a target with "expect_status": 404 to audit the not-found page (and 410 or another code where users see that template).
  • Dev servers (hot reload, socket.io long-poll, event streams) never go network-idle: those connections are ignored and each load's settle is capped (settle_seconds); events records it.
  • Infinite scroll: capped by max_pages; don't scroll endlessly.
  • WAF or rate limits: the runner backs off; if it keeps failing, record it in events and limitations.
  • Login: use a captured session (capture-session) or a login recipe with env fields. Never put credentials in the config, and keep the session file outside the repository.
  • PDFs: out of scope; point to veraPDF.
  • Canvas, WebGL or plugin surfaces (a game, a map, a document viewer drawn on a <canvas>): the engines see an empty element, so canvas_content_described-style rules, best-practice region messages and contrast over the canvas are about the frame, not the content. Text over a canvas that read blank when audited is estimated against the CSS background (the estimate's basis says so); over a painted canvas it is an unknown background. When the main surface never loaded or can't be inspected, record it in metadata.untestable_surfaces (page, surface, reason) and in limitations: overall confidence is then never high, whatever the completeness.
  • Touch and mobile: narrow viewports are desktop Chromium unless listed in mobile_viewports (touch events, a coarse pointer, mobile layout). Add a phone viewport there when the site has touch-only controls or (pointer: coarse) styles.
  • Named states count: each state's engines and probes are rows of the check matrix (with state), so a state probe that was cut short or errored lowers completeness and confidence and must be named in limitations.
  • Text in another script: the language probe also counts letters per script, so a page of Cyrillic, Arabic, Hebrew, CJK, Greek, Devanagari or Thai text under a Latin-script lang is a 3.1.1 mismatch; a page in a Latin-script language with too few stop words stays undetected.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.