render-verify: look at what you rendered

Why this skill exists

A visual artifact can be technically valid and still tell the wrong story. A chart with a collapsed axis, a merged color scale, or an empty data binding renders cleanly. So does a report whose images 404, whose text is clipped, or whose stylesheet never loaded. No validator catches any of it. validate_chart proves a spec is well-formed, not that the picture is true. Only looking does.

This is the plugin's characteristic bug shape — see Known failure modes in the repo docs. Every other silent failure here is a config path; this one is a picture.

The rule: if you rendered it, opened it, or edited it, look at it before you say it is done.

For project deliverables, apply the shared storage contract: inspect the final repository copy and its resources, not only a temporary preview or MCP display. Preserve required supporting sources and report blocked delivery explicitly. This check grants no relocation or publication authority.

Scope. The method below — open, read the console first, walk a catalog, check the claim — is general. It applies to charts, generated HTML reports, SVG figures, dashboards, diagrams, and printable output. Charts are the deepest-worked case because this plugin produces them, and their catalog is the longest; a shorter general catalog follows it.

When to invoke

Mandatory:

• After any post-Flint Vega-Lite edit. The flint-chart skill forbids sending an edited spec back to render_chart, so the MCP server's own validation no longer protects you. You are flying without instruments.
• Before committing generated HTML, SVG, or PNG. Inline specs fail silently.
• When the artifact is layered, faceted, multi-series, or multi-figure. Most failures below come from layer, scale, or layout interaction.
• When requested interaction behavior or state changes: if gesture and reset acceptance is not already defined, load and follow flint-interactions. Otherwise apply the existing acceptance; do not reload that owner. This skill owns visual/data checks, not a second interaction procedure; a valid spec, screenshot, or accepted display call does not prove behavior.

Recommended:

• After the first render of anything that will be shown to someone other than the person who asked for it.
• Whenever you changed the data binding or the page structure, not just styling.

Skip:

• Single-layer chart, small embedded data, spec unchanged since a render you already looked at.
• The user is iterating rapidly on color or title only.

The failure catalog — charts

These render without error. Check each one explicitly — the list is the point of the skill, not the tooling.

The failure catalog — any rendered artifact

For generated HTML, SVG, dashboards, diagrams, and printable output. These also render without error, and a screenshot alone can look plausible.

Prose-coupling check (before shipping a published figure)

The failure catalogs pin what the figure SHOWS. This check pins what the surrounding PROSE CLAIMS about the figure. Numbers drift silently between them, and the reader reads both.

Before declaring a figure done in a document / chapter / report / worked solution, sweep these five surfaces:

Look for the non-data lever first

When numbers drift, the fix that preserves the argument is usually a NARRATIVE lever — a threshold, a target, or a round-number goal — that lives only in prose. Moving it does not break the dataset OR the contract test.

Example: prose says "the channel needs to clear a $2.00 target" but channel-romi.json contains no 2.00. Raising the target to $2.50 corrects the wrong-figure references and leaves the beat, the Big Idea, the dataset, and the contract test all untouched. Rewriting the numbers to match the prose is more expensive and touches more surfaces.

Priority order when the prose is wrong:

1. Number wrong in prose, right in data: fix the prose. Do not ask.
2. Two valid fixes, one preserves the rhetorical shape: take that one, even if the diff is larger.
3. Only correct fix changes what the passage argues: stop and ask.
4. Wording awkward but numbers right: out of scope for this skill.

Adapted from a published decision-analysis book's figure-authoring practice.

Step 1 — pick a verification capability

If the artifact is ASCII, stop here and use the ASCII branch below instead. Steps 1 through 3 assume a rendered artifact that a browser can open. An ASCII chart has no such artifact: the text in the code fence is both the source and the render, so there is nothing to screenshot and no console to read.

For ASCII, substitute ascii-chart Module 5, the alignment QA loop. It covers the same ground for a character grid: validate every line against the width constant, verify borders and padding by counting characters rather than eyeballing them, and re-check after each fix. That is the ASCII equivalent of opening the render and reading the errors.

Then rejoin at Step 4. Claim checking and honest reporting do not depend on a renderer, so the Storytelling read-back, the mutation check, and Step 5 apply unchanged. Geometry passing is not the same as the numbers being right, and on a character grid that gap is easy to miss: a chart can validate perfectly while displaying a value the data does not support.

This skill names the capability, not a product. Work down this ladder and stop at the first rung that works. Do not install a second MCP server for a job the host already does.

1. The host's own browser capability — always try this first. If your tool inventory contains anything that opens a page and returns a screenshot or a page snapshot to you, use it. Discover current capabilities rather than assuming a product-wide browser matrix. Local-file access, child-frame input, console access, and downloads are distinct, host-dependent capabilities. Follow browser-tools when input times out or reaches an iframe without reaching its chart; diagnose before changing the artifact or switching tools.

   Historical observation: Scout blocked file: in a 2026-09-07 test. That is not a permanent capability claim for Scout, CLI, or any other host. Verify the currently selected host and preserve sharing and authentication boundaries.

2. The optional alex-playwright MCP server — fallback, or the only path. Use when rung 1 is absent, when rung 1 cannot open file:// and the artifact is local, or when rung 1 lacks console-error access and the defect you are chasing needs a cause rather than a symptom. Do not substitute it when the user explicitly requires native-browser evidence without naming the gap. See Playwright MCP setup below. It carries real costs: a browser must already be installed, file:// needs --allow-unrestricted-file-access, and it writes artifacts into the working directory.
3. The human. Ask the user to open the artifact and describe what they see, or give them a specific checklist item to confirm. This is a legitimate outcome, not a failure — but it must be stated.

Never silently skip verification. If you reached rung 3, or if you have partial capability (see below), say so plainly in your report. An unverified chart described as verified is worse than an unverified chart.

Which capabilities you actually need

"Can open HTML" is not one capability, and hosts differ in which they provide. Establish what you have before interpreting what you see.

[!WARNING] Capability 5 can manufacture a defect that isn't there. Vega-Lite and ECharts draw after page load. A screenshot taken too early shows an empty container — which is visually identical to the empty binding row in the failure catalog. Before diagnosing "empty binding", re-capture at least once and confirm the emptiness is stable. Diagnosing a race as a data bug sends the fix upstream into a spec that was never wrong.

When file:// is not enough

Default to opening the artifact directly. The VS Code integrated browser supports http://, https://, and file:// URLs, and an agent-opened file:// page needs no server, no flags, and no deploy. Reach for something heavier only when one of these holds.

[!WARNING] A pass in the agent browser is not a pass for the human. The agent browser permits fetch() over file://; Chromium in a normal browser does not. An artifact that loads its own content — a shell that fetches markdown, a report that pulls a sibling JSON — can verify green for you and be blank for the reader. When the artifact fetches anything, verify over http:// before calling it done, and state which origin you verified against.

Probe by doing, not by asking: attempt the action against the real artifact and observe the result. Tool names vary between hosts; outcomes do not.

Step 2 — open it and read the errors first

1. Open the artifact. For local files use the canonical absolute file:///… form.
2. Read the console errors before looking at the picture. This is the check that finds the cause rather than the symptom — a silently-failing inline Vega-Lite spec throws to the console while still rendering a plausible-looking page. With the Playwright server this is browser_console_messages at level error.
3. Then screenshot it.

Console output alone does not establish a stale render or a spec defect. Check the actual artifact, readiness, frame target, and state before choosing a cause.

Step 3 — check the picture against the catalogs

Walk the chart table if it is a chart, and the general table for anything that is rendered as a page. Then, for multi-figure artifacts:

• Scroll each figure into view and capture it separately. A single full-page screenshot visually hides defects in unfocused figures.
• Check the axes have real domains — not [0, 0], not a collapsed range, no undefined ticks.
• Count the marks against what the data should produce, by count, not by eye. (1) Derive the expected count from the source: rows per series in the dataset or the spec's data, counted with a script or jq, with any dedup or aggregation the spec applies already accounted for. (2) Count the rendered marks in the browser snapshot: the mark group's DOM elements, or the mark entries in the accessibility tree. (3) Compare the two numbers. A mismatch is a data or spec problem (dedup, aggregation, empty binding) until proven otherwise; do not resolve it by looking harder at the picture.
• Search the rendered text for placeholders — TODO, undefined, NaN, {{, Lorem. Search what rendered, not the source that produced it.
• Decode embedded images. Require decode() success or naturalWidth > 0, including data-URI SVG. An image can be complete and still fail XML decoding; report/footer text outside its SVG root is not valid image content.
• Measure clipping after resize. Wait for the child frame and layout to settle; compare each plot's getBoundingClientRect() with its nearest clipping container at desktop and narrow widths. A page with no horizontal overflow can still hide part of a fixed-height board chart.
• Exercise requested behavior. Use the existing flint-interactions acceptance contract for themed controls, semantic identities, linked overlays, preserved populations, and neutral-state reset. A static preview or changed opacity alone does not prove that the correct recipient records responded.

Step 4 — check the claim, not just the render

Every artifact worth verifying exists to carry a claim. For charts that is the Big Idea from the chart-big-idea skill; for a report or diagram it is whatever the surrounding prose asserts. A correct render of a wrong claim is still a defect.

• Verify prose claims arithmetically against the plotted or tabulated values, with a script. If the caption says "less than a fifth of the noise", do not do the division in your head: run this skill's bundled scripts/check-claims.mjs --data <dataset.json> --part <value or field path> --whole <value or field path> --claim 1/5 --relation lt and quote its output. The model writes the check and interprets the result; it never does the arithmetic in prose. Order-of-magnitude overstatements in captions survive every automated check there is.
• Re-read the claim and ask whether the picture shows it. If the claim is about a gap and the eye goes to a trend, the chart type is wrong — go back to the owning flint-chart selection reference, do not patch the styling.

Storytelling read-back

Read the artifact once as its intended audience, without relying on surrounding prose to rescue it:

If the first focal point or reading order is wrong, return to chart selection, technique, or ThemeSpec. Do not compensate by writing a longer caption.

Mutation check (decision-bearing values)

Rendering checks confirm the picture drew. They cannot confirm the numbers are right, so a figure can pass every geometry, catalog, and accessibility check while asserting something false.

Before shipping a figure whose numbers drive a decision:

1. Change one decision-bearing value in the source data.
2. Regenerate the artifact.
3. Require the output to change visibly, and any coupled check to fail.
4. Restore the source bit-identically and regenerate once more.

If the output does not move, the figure is not reading the data it claims to read. Visual inspection does not find this, because the wrong number renders exactly as cleanly as the right one.

Check every surface carrying the value, not only the plotted marks: embedded data, visible labels, aria description, tooltip, caption, and any evidence-boundary text.

Composes with the mutation-testing skill, which applies the same idea to a test harness. Adapted from the Executable Example Contract in an earlier visual-storytelling curriculum, absorbed into this plugin.

Step 5 — report honestly

State which capability you used, that you looked, and what you checked. If you could not verify, say that instead. Never describe an unopened render as verified.

Playwright MCP setup

Fallback only — rung 2 of Step 1. Try the host's own browser capability first; if it provides the capabilities the task needs, you do not need this server. Check the actual capability inventory in Step 1: a host with a browser may still block file:// or lack child-frame input. Measured against @playwright/mcp@0.0.80:

Run setup-dependencies after installing or updating this plugin when the selected fallback runtime needs provisioning. Its --check-updates mode compares the reviewed Playwright pin with the stable dist-tag; a newer version still requires a browser-compatibility pass before the source pin changes.

The server key is alex-playwright, not playwright. Some hosts — Microsoft Scout among them — already ship a built-in server under the bare name, and a collision silently shadows one of the two. The distinct key lets both coexist: the host's for ordinary web pages, this one for file:// artifacts.

{
  "servers": {
    "alex-playwright": {
      "type": "stdio",
      "command": "node",
      "args": [
        "skills/setup-dependencies/scripts/runtime-launcher.mjs",
        "playwright",
        "--headless",
        "--isolated",
        "--browser",
        "msedge",
        "--allow-unrestricted-file-access"
      ]
    }
  }
}

Merge this additively into the existing server map — overwriting the file destroys the user's other servers, including flint. Same per-host path table as the flint-chart skill (.vscode/mcp.json for VS Code, .mcp.json for Claude Code, .cursor/mcp.json for Cursor, ~/.copilot/mcp-config.json with key mcpServers for Copilot CLI).

Why each flag

Security note — read before enabling the file-access flag

--allow-unrestricted-file-access lets the browser read any file the user can read. That is acceptable for verifying local artifacts you just produced. It is not acceptable in combination with browsing untrusted web pages: a malicious page can attempt to drive the agent into reading local files and sending them out. Keep this server for local verification. If you need general web browsing, use a separate configuration without the flag.

Do not enable arbitrary host-code execution just to inspect a chart. Screenshots and console access do not establish interactive behavior; prefer supported native input or narrowly scoped in-page evaluation when those capabilities are available. General-purpose host-code execution remains a separate capability and approval boundary, not a verification default.

Housekeeping

The server writes artifacts (accessibility snapshots, screenshots) into the working directory. Add .playwright-mcp/ to .gitignore or they land in commits.

Do not pass a bare filename to browser_take_screenshot. A bare name like shot.png is written to the working-directory root, which is outside the ignored folder and shows up as an untracked file in the user's repo. Either omit filename entirely — the server then writes into .playwright-mcp/ — or give a path inside that folder. Verified 2026-07-25: a bare filename leaked into git status while the snapshot beside it did not.

Troubleshooting

Anti-patterns

• Declaring an artifact done because the tool returned success. The tool reporting OK means bytes were written, not that the picture is true.
• Trusting a batch edit's summary line. When a multi-edit call reports "1 succeeded, 1 failed", verify which one landed by inspecting the file before re-rendering. The visible change is often not the one that succeeded.
• One full-page screenshot for a multi-figure report. Defects hide in the figures you did not focus.
• Verifying against sample data only. The failure mode this skill exists for appears when real data meets the spec.
• Installing the Playwright server on a host that already has a browser capability. Redundant dependency, extra config surface, no gain. Rung 1 before rung 2, always.
• Diagnosing an empty chart before re-capturing. JS-rendered charts draw after load; one early screenshot is not evidence of an empty binding.
• Screenshotting without reading the console. The console usually names the cause — a 404'd image, a failed font, a thrown spec error — while the picture only shows the symptom.
• Reporting a file:// pass for an artifact that fetches its own content. The agent browser is more permissive than the reader's browser. Serve it over http:// first, or name the origin you actually verified against.
• Standing up a server before trying file://. file:// is documented and usually sufficient. Escalate on evidence, not on habit.
• Fixing a data or chart-type problem with a style tweak. Recoloring a mark that is wrong because two scales merged hides the bug instead of fixing it.

Related skills

• chart-big-idea — Step 0.5 earn-a-figure gate + Step 4.5 focus discipline. Framing side of the same discipline.
• chart-vocabulary — Module 2 CSAR evaluation loop asks did the AI pick the right chart family. This skill's Prose-coupling check asks did the render match the message. Both fire on an AI-generated chart; different failure modes.
• flint-chart — spec authoring + Publication config preset. Verification runs against what this skill produces.
• print-svg-style-guide — the visual grammar the shipped SVG obeys. The failure-catalog entries in this skill fire when that grammar is violated.
• figure-generator — where the "fix in the generator, never in the SVG" rule for the SVG XML invalid catalog entry lives.
• corpus-qa-sweep — the corpus-scale companion. That skill sweeps every item against machine-checkable invariants and triages the flags; this one judges a single artifact against the failure catalog. Sweep to find candidates, then verify one.
• annotate-screenshot — inherits Step 2's open-and-read discipline for a different output. Where this skill judges whether a render says what it should, that one marks the answer onto the image, and its own failures (a badge covering the feature it points at, a clipped caption) are only visible by rendering and looking.

Would Revise If

Revise this skill if:

• The host's built-in browser tools gain or lose console-error access. Prefer native console inspection where available; current VS Code documentation includes it. Retain the optional Playwright server for required scenarios that are unavailable or unverified on the active host, not as a universal requirement.
• A host appears whose canvas renders HTML for the user but returns nothing to the agent. Capability 2 in Step 1 assumes "renders" and "agent can read it back" usually travel together. A surface that splits them would make rung 3 the common case rather than the exception, and Step 5's honesty requirement the most load-bearing part of this skill.
• @playwright/mcp changes its file:// default. The security note and the flag table both assume navigation is blocked unless the flag is set.
• The integrated browser changes its file:// posture — either dropping support, or tightening fetch() to match a normal browser. The When file:// is not enough table assumes agent-permissive and reader-strict; if they converge, both the table and the agent-browser-only catalog row collapse to one line.
• Agent-opened tabs gain the user's session. The auth row says to share a tab precisely because agent-opened tabs are ephemeral. If that changes, the row is wrong.
• @playwright/mcp ships a bundled browser by default. The troubleshooting row about installed-Chrome-by-channel would then be wrong.
• A failure mode recurs that is not in either catalog above. The tables are the load-bearing content; extend them rather than adding tooling.
• Verification is consistently skipped by users, indicating the step is too heavy and should collapse into the flint-chart render step instead of standing as its own skill.