ui-audit-and-predicates
Skill22 starsChanged 3 months agoApache-2.0View on GitHub
QA a screen with the deterministic audit action and target elements with structured predicates instead of hand-parsing describe-ui trees or eyeballing screenshots. Use for fleet QA sessions checking touch targets, clipping, alignment, spacing, safe-area violations, and missing accessibility labels.
In this repository8 other files it gives its agents.
- broker-federationSkill
- build-and-testSkill
- drive-a-mirrored-iphoneSkill
- journal-evidence-exportSkill
- lease-and-drive-a-simSkill
- park-thaw-semanticsSkill
- run-evalsSkill
- run-the-daemonSkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 1 is distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 60
Never compute tap coordinates from a describe-ui dump.
- Very specific: line 76
If a matcher misses, call ui_tree (manzanas observe) to see what is actually on screen, then tighten — do not fall back to coordinates.
- Very specific: line 101
Add --capture-logs [--log-process MyApp] (payload capture_logs / log_process) to tap-element, type-into-element, or scroll-to-element to …
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
UI audit + predicates: stop hand-parsing trees
Two tools replace manual tree/screenshot inspection during QA:
audit— deterministic geometry checks over the accessibility tree, returning findings (measured evidence, never pass/fail verdicts) plus an annotated screenshot with a labeled red box per finding. Both are journaled automatically.- Predicates — strict element matching for
tap_element,type_into_element,wait_for_element,scroll_to_element. A predicate must resolve to exactly one element; ambiguity fails loudly with every candidate listed instead of silently guessing.
Audit a screen
manzanas audit --lease $LID -o annotated.png # all six checks
manzanas audit --lease $LID --checks touch_target,missing_labels
manzanas audit --lease $LID --label "Sign up" # scope to one subtree
manzanas audit --lease $LID --region 0,0,393,400 # scope to a screen areaMCP agents call the audit tool with the same fields (checks, region,
matcher fields, min_touch_pt, alignment_tolerance_pt,
spacing_tolerance_pt, safe_area_insets).
Checks: touch_target (< 44x44pt interactive), clipping (past screen or
non-scrolling parent bounds), alignment (near-miss edge deltas ≤ 4pt),
spacing (sibling gaps deviating > 4pt from the median), safe_area
(interactive elements in the insets), missing_labels (interactive
elements a screen reader cannot name).
Reading results:
- Each finding has
check,ref(F1, F2, ... — matches the red box on the annotated screenshot), the element's role/label/id/frame,measuredvalues, and anevidencesentence. - Findings are EVIDENCE, not verdicts. You decide what matters: a 30x30 close button is probably a real defect; a deliberately compact stepper may not be.
- Dense repeated grids (keyboards, emoji pickers, calendar cells) are
auto-suppressed;
suppressed_elementsreports how many. - System chrome is auto-suppressed too: scroll-indicator pseudo-elements,
the status bar, and the keyboard never produce findings by default
(
--include-system-chromerestores them; individual keyboard keys stay under the dense-group rule), andtouch_targetwithholds small controls covered by a full-size tappable list row (Cell or Button) — stock Settings' ~28pt row buttons — reported assuppressed_covered_controls(--include-covered-controlsrestores them). So a noisy stock-app screen audits clean; remaining findings are the app's own layout. - Both artifacts land in the journal:
manzanas journal export $LID -o evidence.mdincludes them — paste into the PR.
Target elements with predicates, not coordinates
Never compute tap coordinates from a describe-ui dump. Use matchers:
manzanas tap-element --lease $LID --label "Continue" # flat matcher, best-ranked
manzanas tap-element --lease $LID --predicate '{
"type": "Button", "text": "Delete",
"near": {"predicate": {"text": "Alice"}, "direction": "right"}}'Predicate fields: text / text_contains / text_regex (one text form),
type, accessibility_id, bounds_hint (top_half/bottom_half/...),
near ({predicate, direction, max_distance?}), parent_of (predicate on a
descendant), index (explicit 0-based disambiguator). Prefer predicates
whenever a screen repeats controls (one Delete per row) — a flat matcher
would silently pick the "best" one.
If a matcher misses, call ui_tree (manzanas observe) to see what is
actually on screen, then tighten — do not fall back to coordinates.
Timeout errors now tell you when matching elements exist off-screen
(... exist off-screen: Button label="Save" (below the viewport) —
scroll to bring them into view): that means scroll_to_element, not a
new matcher. If instead the error says candidates are on screen but
outside the requested in_frame/bounds_hint region, fix the region —
scrolling will not help.
Keep ui_tree cheap on busy screens
manzanas observe --lease $LID --compact # one line per element, [i] indexes
manzanas observe --lease $LID --interactive-only --exclude-system-chrome
manzanas observe --lease $LID --roles Button,Cell --scope '{"type":"Table"}'MCP ui_tree takes the same fields (format:"compact",
interactive_only, roles, scope, exclude_system_chrome). Filters
compose; hash always digests the full tree, so change detection is
unaffected. The compact [i] indexes are stable depth-first positions —
the same order the predicate index field uses.
Correlate app logs with actions
Add --capture-logs [--log-process MyApp] (payload capture_logs /
log_process) to tap-element, type-into-element, or
scroll-to-element to capture the simulator's os_log lines emitted
during the action window. They come back as result.logs and are
journaled as action-logs.txt next to the step, so the journal export
shows what the app logged at each action.
QA loop recipe
lease_acquire→ launch the app →wait_tree_stable.auditthe screen; note refs worth acting on.- Drive to the next screen with
tap_element/predicates; repeat. journal export $LID -o evidence.mdand attach findings + annotated screenshots to the PR.
Full docs: docs/mcp.md (audit + predicate DSL sections).
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

