agentleFS
BariBariGoodBariBariGood/manzanas/.agents/skills/ui-audit-and-predicates/SKILL.md

ui-audit-and-predicates

Skill22 starsChanged 3 months agoApache-2.0View on GitHub

QA a screen with the deterministic audit action and target elements with structured predicates instead of hand-parsing describe-ui trees or eyeballing screenshots. Use for fleet QA sessions checking touch targets, clipping, alignment, spacing, safe-area violations, and missing accessibility labels.

The file

We split this skill into the rules it gives an agent

3 rules. About 0 are common, 1 is distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Never compute tap coordinates from a describe-ui dump.

    line 60
  2. Very specific:

    If a matcher misses, call ui_tree (manzanas observe) to see what is actually on screen, then tighten — do not fall back to coordinates.

    line 76
  3. Very specific:

    Add --capture-logs [--log-process MyApp] (payload capture_logs / log_process) to tap-element, type-into-element, or scroll-to-element to …

    line 101

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

UI audit + predicates: stop hand-parsing trees

Two tools replace manual tree/screenshot inspection during QA:

  1. audit — deterministic geometry checks over the accessibility tree, returning findings (measured evidence, never pass/fail verdicts) plus an annotated screenshot with a labeled red box per finding. Both are journaled automatically.
  2. Predicates — strict element matching for tap_element, type_into_element, wait_for_element, scroll_to_element. A predicate must resolve to exactly one element; ambiguity fails loudly with every candidate listed instead of silently guessing.

Audit a screen

manzanas audit --lease $LID -o annotated.png            # all six checks
manzanas audit --lease $LID --checks touch_target,missing_labels
manzanas audit --lease $LID --label "Sign up"           # scope to one subtree
manzanas audit --lease $LID --region 0,0,393,400        # scope to a screen area

MCP agents call the audit tool with the same fields (checks, region, matcher fields, min_touch_pt, alignment_tolerance_pt, spacing_tolerance_pt, safe_area_insets).

Checks: touch_target (< 44x44pt interactive), clipping (past screen or non-scrolling parent bounds), alignment (near-miss edge deltas ≤ 4pt), spacing (sibling gaps deviating > 4pt from the median), safe_area (interactive elements in the insets), missing_labels (interactive elements a screen reader cannot name).

Reading results:

  • Each finding has check, ref (F1, F2, ... — matches the red box on the annotated screenshot), the element's role/label/id/frame, measured values, and an evidence sentence.
  • Findings are EVIDENCE, not verdicts. You decide what matters: a 30x30 close button is probably a real defect; a deliberately compact stepper may not be.
  • Dense repeated grids (keyboards, emoji pickers, calendar cells) are auto-suppressed; suppressed_elements reports how many.
  • System chrome is auto-suppressed too: scroll-indicator pseudo-elements, the status bar, and the keyboard never produce findings by default (--include-system-chrome restores them; individual keyboard keys stay under the dense-group rule), and touch_target withholds small controls covered by a full-size tappable list row (Cell or Button) — stock Settings' ~28pt row buttons — reported as suppressed_covered_controls (--include-covered-controls restores them). So a noisy stock-app screen audits clean; remaining findings are the app's own layout.
  • Both artifacts land in the journal: manzanas journal export $LID -o evidence.md includes them — paste into the PR.

Target elements with predicates, not coordinates

Never compute tap coordinates from a describe-ui dump. Use matchers:

manzanas tap-element --lease $LID --label "Continue"     # flat matcher, best-ranked
manzanas tap-element --lease $LID --predicate '{
  "type": "Button", "text": "Delete",
  "near": {"predicate": {"text": "Alice"}, "direction": "right"}}'

Predicate fields: text / text_contains / text_regex (one text form), type, accessibility_id, bounds_hint (top_half/bottom_half/...), near ({predicate, direction, max_distance?}), parent_of (predicate on a descendant), index (explicit 0-based disambiguator). Prefer predicates whenever a screen repeats controls (one Delete per row) — a flat matcher would silently pick the "best" one.

If a matcher misses, call ui_tree (manzanas observe) to see what is actually on screen, then tighten — do not fall back to coordinates. Timeout errors now tell you when matching elements exist off-screen (... exist off-screen: Button label="Save" (below the viewport) — scroll to bring them into view): that means scroll_to_element, not a new matcher. If instead the error says candidates are on screen but outside the requested in_frame/bounds_hint region, fix the region — scrolling will not help.

Keep ui_tree cheap on busy screens

manzanas observe --lease $LID --compact                  # one line per element, [i] indexes
manzanas observe --lease $LID --interactive-only --exclude-system-chrome
manzanas observe --lease $LID --roles Button,Cell --scope '{"type":"Table"}'

MCP ui_tree takes the same fields (format:"compact", interactive_only, roles, scope, exclude_system_chrome). Filters compose; hash always digests the full tree, so change detection is unaffected. The compact [i] indexes are stable depth-first positions — the same order the predicate index field uses.

Correlate app logs with actions

Add --capture-logs [--log-process MyApp] (payload capture_logs / log_process) to tap-element, type-into-element, or scroll-to-element to capture the simulator's os_log lines emitted during the action window. They come back as result.logs and are journaled as action-logs.txt next to the step, so the journal export shows what the app logged at each action.

QA loop recipe

  1. lease_acquire → launch the app → wait_tree_stable.
  2. audit the screen; note refs worth acting on.
  3. Drive to the next screen with tap_element/predicates; repeat.
  4. journal export $LID -o evidence.md and attach findings + annotated screenshots to the PR.

Full docs: docs/mcp.md (audit + predicate DSL sections).

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.