agentleFS
doublejdoublej/consult-user-mcp/test-cases/CLAUDE.md

consult-user-mcp / test-cases

CLAUDE.md39 starsChanged 4 months agoMITView on GitHub

JSON fixtures for every dialog type, plus two harnesses over them. Two consumers of the fixtures, which is the thing worth knowing: the harnesses and the Tray App's debug menu all read from cases/. The two harnesses answer different questions, and the second one is the one that fails a build: A state is a fixture plus how to drive it: which pane is open, what keys to type, how long to settle. That is why states.tsv has more rows…

The file

We split this CLAUDE.md into the rules it gives an agent

1 rule. About 0 are common, 0 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈0

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    The debug menu loads from here: Never hardcode dialog JSON in macos-app/Sources/AppDelegate.swift.

    line 61

≈ “Common” is an estimate: at least 1 in 5 of the 27,641 CLAUDE.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
CLAUDE.md

Test Cases

What this is

JSON fixtures for every dialog type, plus two harnesses over them. Two consumers of the fixtures, which is the thing worth knowing: the harnesses and the Tray App's debug menu all read from cases/.

The two harnesses answer different questions, and the second one is the one that fails a build:

test-runner.shlayout-audit.sh
Questiondoes it look right?does it fit?
Methodspawns real dialogs, screencapture, OCRrenders off-screen, measures the view tree and the bitmap
Verdictword presence, margin stripsoverflow, overlap, text-fit, escapes
Takes the keyboardyes — one dialog per case, on screenno — windows are parked off-display
Exit codecapture failures onlyfails on any layout violation

Mental model

cases/<type>/<name>.json    a raw dialog request, exactly as the CLI receives it
test-runner.sh              iterates cases/, spawns each dialog, screenshots it
capture-dialog.swift        the screenshot helper
verify-checklist.md         what a human should look for per dialog type
screenshots/<timestamp>/    output (gitignored)

layout-audit.sh             the asserting harness — render every state, then check
skin-states/states.tsv      the state manifest: fixture + settle + pane + key script
skin-states/render/         off-screen renderer (main.swift) and the probe (probe.swift)
skin-states/check-layout.ts the rules; turns probe output into pass/fail
skin-states/waivers.json    known-bad states, each with a reason and an issue id
skin-states/audit/<skin>/   output: <state>.png beside <state>.layout.json (gitignored)

A state is a fixture plus how to drive it: which pane is open, what keys to type, how long to settle. That is why states.tsv has more rows than cases/ has files — one fixture shot four ways is four states.

States with no pane and no key script are rendered in parallel; the rest run one at a time. Their settle delays were measured on an idle machine, and under load the probe catches a surface mid-transition — every state that went red the first time the audit ran wide was a pane or a key script, and every one of them was actually correct. Parallelism is not allowed to change the verdict, so the split is by whether a state has anything to settle, not by a tuning knob.

A fixture is not a wrapper format — it is the literal JSON argument passed to the [[Dialog CLI]]. That is why the same file works for both the runner and the debug menu.

Directory names are the test-case names, not CLI commands. test-runner.sh maps between them around line 88:

DirectoryCLI command
confirm/confirm
choose/choose
text-input/textInput
questions/questions
notify/notify
tweak/tweak
sketch/(Sketch CLI)

Important invariants

  • The debug menu loads from here. Never hardcode dialog JSON in macos-app/Sources/AppDelegate.swift. A fixture added here is the way to get a dialog into the debug menu.
  • A new dialog type needs four things: a cases/<type>/ directory with at least one fixture, an entry in the test-runner.sh command mapping, a makeWindow case in skin-states/render/main.swift, and a debug-menu item. Miss the mapping and the runner skips the whole type without reporting anything; miss the makeWindow case and the audit reports unknown kind and moves on.
  • A fixture is not a state. Adding JSON to cases/ gets it into the debug menu and the screenshot runner. It does not get it into the layout audit — that needs a row in skin-states/states.tsv. A fixture with no row is never measured.
  • Fixtures cover shapes, not just happy paths. The value of long-body, unbreakable-tokens, forty-options, wide-glyphs, unbalanced is that they break layout. Keep adding the ugly ones — window sizing is the most fragile thing in the dialog CLI.
  • The skin is pinned, not inherited. Both harnesses default to caret, the New Interface. Before that was true, every visual run silently exercised only the classic chassis, and the skin people had actually switched on was the one nothing tested.
  • Some fixtures have companion setup files (a .css for tweak cases) that the runner copies into place. Keep them beside the JSON.

What the audit can and cannot see

Worth knowing before trusting a green run, and before writing a rule:

  • fittingSize can never report overflow on its own. createAutoSizedWindow sizes the window from fittingSize, so the two agree by construction. Overflow only means something once the window is clamped at the height cap. The rule is kept because that clamped case is real, not because it fires often.
  • SwiftUI Text is invisible to the view walk. It is drawn into the hosting view's layer, not into a child NSView. Only the NSViewRepresentable widgets — CaretTarget, CaretNoteEditor, the Focusable* family — can be measured. A clipped SwiftUI label is caught by its consequences (an overlap, a box that stopped fitting), not directly.
  • Overlap between interactive views is the highest-signal rule. It is what catches a note pane opening across the footer: nothing is clipped, the geometry stays self-consistent, and two things a person can click simply land on the same pixels.
  • The pixel probe derives the background from the image, so it works on any skin and theme without being told the palette. It cannot see a collision — pixels do not record what drew them.
  • The rules are calibrated for caret and only caret. Telling a row scrolled out of a list from a control the layout pushed outside the window has no general answer. Caret lays an oversized list out at full height and hangs the scroll view outside the surface, which is what the probe keys on; classic keeps it inside and clips at the layer with a translucent footer over the top. Run the audit against classic or bracket and every long-list state fails while looking perfectly correct in the screenshot — 14 and 8 respectively. That is why SKINS defaults to caret. Tracked in cum-6ql.
  • There is no edge-ink rule, though the probe still measures it. It produced zero true positives across two skins and a wall of false ones. A rule nobody can trust is worse than no rule.

Common change patterns

Reproducing a layout bug → add a fixture that triggers it and a states.tsv row, then run the audit and confirm it fails before fixing. visual-bugs/ collects known-bad renders.

Adding a fixture → drop the JSON in the right cases/ directory for the debug menu and the screenshot runner; add a states.tsv row to get it measured.

A bug you are not fixing today → add it to waivers.json with the rules it trips, a reason, and the issue id. A waiver that stops matching fails the run, so the fix gets noticed rather than quietly widening the waiver list.

Verification

Run these in the container. bun run test:container — or a subset with SUITES="unit layout" bun run test:container. See ../container/README.md.

bun run test:container                     # everything, in the VM — the verdict
bun run test:layout                        # assert: caret, every state
SKINS="caret classic" bun run test:layout  # more than one — expect noise, see below
bun run test:visual                    # capture + OCR   ⚠ takes the screen
bun run test:keyboard                  # keyboard contract ⚠ takes the keyboard

test:layout is the exception that is safe on the host: it renders off-screen, so nothing appears and nothing takes focus. The other two spawn real dialogs. On a machine somebody is using they seize the screen for minutes, and a stray focus change silently corrupts the run — the visual suite spent months photographing one stuck window because of exactly that.

From here: ./layout-audit.sh [name-filter], ../container/launch.sh run.

The audit prints a FAIL/WAIVED/STALE line per state and exits non-zero. The screenshot runner captures, OCRs each shot against the words that should be on it, and now fails when it cannot read them — a shot of the wrong window is a red run rather than a green one. Read verify-checklist.md against its output.

A pass is not a clean surface. The audit waives every state with a known open bug and prints the list each run. bun run test:layout currently comes back green with a dozen states waived across six open issues. Read that list before treating a pass as a release signal.

Related context

  • verify-checklist.md — per-dialog-type review checklist
  • ../dialog-cli/CLAUDE.md — what consumes these fixtures
  • ../macos-app/CLAUDE.md — the debug menu
  • .claude/rules/dialog-parity.md

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.