agentleFS
ZSeven-WZSeven-W/dsh-qa/skills/qa-explore/SKILL.md

qa-explore

Skill4 starsChanged 56 days agoMITView on GitHub

Explore an app autonomously with dsh-qa, preserve evidence and driver safety decisions, export only freshly observed outcomes, and replay the exact exported scenario.

4 other projects use a version of this.

Adopted by

Most recently taken up by

  • Other versions4 projects use a version of this.

    Word for word · 0

    None found.

    Edited · 0

    None found.

    How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.

The file

We split this skill into the rules it gives an agent

6 rules. 0 are common, 1 is distinctive, and 5 are very specific to this project.

  • 0

    Common

    Most of the 3 edited versions we read keep these.

  • 1

    Distinctive

    This author's own choices. Worth a look.

  • 5

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use this loop: start → observe → choose one semantic target → act → inspect the fresh observation → assert → capture evidence when a …

    line 8
  2. Very specific:

    Keep one explicit owner from qa_session_start through qa_record_export.

    line 12
  3. Very specific:

    Call qa_evidence at the moment a problem appears, before navigating away or changing the state.

    line 28
  4. Very specific:

    Call qa_record_export with the same owner and a .json output_path whose parent already exists under the current workspace or temporary …

    line 40
  5. Very specific:

    every exported step gets an assertion synthesized from and evaluated against the immediate fresh observation after that action.

    line 45
  6. Very specific:

    if no proven step remains, export returns NO_PROVEN_STEPS and writes no file.

    line 49

“Common” is measured: kept by at least 6 in 10 of the 3 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Explore with dsh-qa

Use this loop: start → observe → choose one semantic target → act → inspect the fresh observation → assert → capture evidence when a problem appears → export → replay → stop.

Method

  • Keep one explicit owner from qa_session_start through qa_record_export.
  • A heavy site can widen the settle budget at start: pass settle_budget_ms (and settle_quiet_ms) to qa_session_start, clamped to the schema bounds (budget ≤ 15000ms). Separately, the session widens its budget ONCE automatically (the adaptive budget, settle_adaptive_budget_ms / env DSHPLUGIN_QA_SETTLE_ADAPTIVE_BUDGET_MS, default 6000ms, 0/off disables) when a settle window is still churning at the starting budget, OR when a positive-existence assertion retry exhausts its budget without finding its target: the same window keeps polling until the adaptive budget, and every settle result reports settle.widened ({ fromMs, toMs, cause } or null, cause "unstable" or "assertion-retry"). Whatever effective policy Explore ran with (the widened budget when it widened) is what qa_record_export records into meta.settle, and qa_replay_run applies it (env/host defaults otherwise), printing the effective policy plus settleWidened in report.json / report.md.
  • Begin with qa_observe. Prefer a unique role plus accessible name. Treat refs, coordinates, indices, observation ids, and generated ids as ephemeral live-session handles.
  • Perform exactly one qa_act, then inspect its fresh post-action observation before deciding what happened. Re-observe when diagnosing, and never reuse an old ref.
  • An unknown receipt is never success. It needs a semantic delta or URL change in the fresh observation. A rejected or failed receipt is a hard stop; honor every driver safety rejection and never route around it.
  • Mobile text input is conditional and driver-native: iOS fill/type use the dsh-ios element-bound fillTarget/typeTarget only when the live driver exposes them (never raw global type or invented focus), Android type is append-faithful after real focus verification, and Android fill remains unavailable.
  • A fill is proven by its OWN target's value: when the fresh observation shows the target carrying the typed text, the exporter synthesizes a node-value assertion on that target — including when the fill REWROTE the target's accessible name OR role (aria-label following the value: "Search" → "Search: async", or textbox → combobox once suggestions open); the exporter then follows the same identity rule the echo mask uses (match by name role-agnostic OR by role name-agnostic, unique among candidates) and binds the assertion to the node's MOST STABLE predicate — the unique accessible name alone (role omitted) when the role changed, so a fast replay still matches before the role switch; role+name only when the name alone is ambiguous — never node-present of the renamed field alone. A secret-bearing control (valueWithheld: password / one-time-code / cc autocomplete) never carries a value, so no value assertion is synthesized for it.
  • The settle window echo-masks the action's own value write (fill/type/select, and key/press on a uniquely identified target), so a downstream consequence that lands after the echo is still waited for — a fresh observation that only shows the echo never proves the action alone.
  • After the first non-echo change is observed, the quiet window lengthens to postChangeQuietMs (default 2× quietMs), measured from the last change, so an outcome landing after early unrelated churn (a sibling mirroring the typed value, a late hydration rename) is still captured rather than cut off by one short quiet window.
  • Assert resulting state with qa_assert; do not repeat the action to test whether it landed. node-value (expected: { role?, name?, tag?, value }) asserts a node's exact current value and is deterministic, like node-present / node-absent / page-url. It demands the predicate identify EXACTLY ONE node (TARGET_NOT_UNIQUE otherwise — a twin already holding the value proves nothing), and a valueWithheld/secure/valueTruncated node can never satisfy it (VALUE_WITHHELD/VALUE_SECURE/VALUE_TRUNCATED).
  • A POSITIVE existence assertion (node-present / node-value / node-in-viewport / page-url) that is first "not found" is re-observed within the settle budget before failing — the node may just be slow to render — and the result records attempts / elapsedMs. When that retry exhausts its budget without finding the target, the session widens its budget ONCE through the same gate (settle.widened with cause: "assertion-retry") and keeps retrying until the adaptive budget, so a page that settles fast but renders slowly is still found. node-absent is never retried into a pass: absence is never proven by waiting, only by having seen the whole view.
  • A view can also be UNSTABLE: when a fresh observation's settle.stable is false the page never stopped changing inside the settle budget, so nothing in it proves anything. qa_assert then returns passed: false with inconclusive: true and code: "INCONCLUSIVE_UNSTABLE" (the same non-result vocabulary as INCONCLUSIVE_TRUNCATED) — never a false green; wait for the page to stop changing, then re-observe. A qa_act on an unstable proof window keeps its receipt honest (confirmed / unknown) but adds proven: false plus code: "INCONCLUSIVE_UNSTABLE": the dispatch happened, the consequence is unproven.
  • A view can be TRUNCATED at the node budget, and a node outside that window still exists. An absence therefore cannot be proven from a truncated view: node-absent re-observes once at a raised budget and then fails closed with completeness.reason: "INCONCLUSIVE_TRUNCATED" instead of reporting a false "gone". Read completeness before believing any negative result — "we did not see it" is not "it is not there". A node that WAS returned is sound evidence of presence either way.
  • A view can also be SCOPED (browser, driver contract v8): pass within_ref — an opaque ref from your CURRENT (latest, unexpired) qa_observe result — to observe only the composed subtree rooted at that element. Budgets, the byte ceiling, the scan window, and the iframe marker become SUBTREE-relative, so a container whose subtree fits reports truncated: false with no truncationReasons: absence inside it is PROVABLE once the deciding observation's coverage is verified (see the coverage rule), and a deep target unreachable in the whole-page window becomes reachable. The observation's scope field ({ ref, rootRef, role, name, tag }) echoes the root the driver observed; rootRef is the fresh per-observation ref that chains the next scoped read; it is absent for whole-page observations. An unknown, expired, consumed, non-element, or detached ref REFUSES the call with its driver code (REF_UNKNOWN / REF_EXPIRED / TARGET_CHANGED / …) — never a whole-page fallback and never a "not found". The computer driver does not support scoping and refuses within_ref. A scoped proof is exported as a scoped assertion (scope: { role, name }), which Replay re-derives in the whole-page view (UNIQUE predicate, TARGET_NOT_UNIQUE when ambiguous) and decides inside the container; the completeness block names the scope, so "absent from this container" is never read as "absent from the whole page". A browser scroll-by-ref whose settled proof view is truncated and lacks the target is re-read ONCE at record time, preferring an identity-anchored SCOPED read rooted at the target's nearest container-role ancestor (its parentRef chain): the escalated view is accepted only when the driver's identity anchor reports the ORIGINAL acted element connected, contained in that container, and in the viewport — identity comes from the anchor, never from matching role/name/tag. A scroll (or any act) whose ref came from a SCOPED baseline takes its PROOF settle inside that scope instead (withinRef: the baseline scope.rootRef + anchorLastAction), accepted only on the same identity-anchor rule — the result then carries proofScope: { role, name } plus anchor (no escalation happened, so no proofEscalated), and export carries the scope (provisional at replay on long pages). On driver >= d069f4f the dispatched action RETAINS the consumed scope root (contract v9), so the scoped proof read RESOLVES through it: the FIRST poll is keyed by the EXPLICIT baseline scope.rootRef (never 'last-scope', so a stale baseline is refused instead of silently rebinding to whatever root was last acted), then the loop re-keys each later poll to the fresh rootRef that poll minted — the scroll is PROVEN inside that scope. When the driver refuses the root (a pre-retention driver, the acted ref IS the scope root itself — the single-owner handle is never retained, while the anchor still works — or a released retention after navigation: OBSERVATION_REQUIRED / SCOPE_UNAVAILABLE), the refusal is DISCLOSED and the proof falls back to the whole-page read. Every non-acceptance exit is disclosed as escalationRefused: { reason, code? } with a fixed vocabulary — target-not-in-baseline, container-not-in-view, escalated-window-unstable, target-not-returned, target-not-in-viewport, anchor-not-connected, anchor-not-contained, anchor-unavailable — plus the driver's code when it threw; already-in-viewport is NOT a refusal (the result then simply has no escalation fields), and the refusal rides on the exported step so report.md prints it on the step line.
  • qa_assert takes the SAME within_ref: the assertion is decided INSIDE that container (the deciding read is a fresh settled scoped observation), so node-absent means "absent FROM THE CONTAINER" — completeness.scope names it ({ role, name }) and the PASS wording says "within the {role} named "{name}""; WITHOUT within_ref the decision stays whole-page and carries no completeness.scope. The ref must come from the LATEST observation: a container ref from the latest whole-page qa_observe, or the scope.rootRef (any node ref works) echoed by the immediately preceding scoped qa_observe. Every observe REPLACES the driver's current observation, so a ref from an earlier observation — including a whole-page container ref captured before an intervening scoped observe — is REFUSED as { ok:false, code: "REF_UNKNOWN" | "REF_EXPIRED" | …, error }, never silently re-decided against the whole page. All scoped rules apply (subtree budgets, in-scope escalation and coverage probe, the coverage.verified gate); the recorder exports a passed scoped assert with its scope (SCOPE_NOT_DURABLE exclusion when the container is not unique in a complete recorded baseline); the computer driver refuses within_ref and kind: "visual" does not take it.
  • Absence and coverage (contract v9): node-absent PASSES only when nothing matched AND the deciding view is complete (truncated: false) AND coverage.verified: true — the terminal absence decision requests the driver's bounded closed-shadow-root probe on its ONE deciding re-observation (never on settle polls; ~5ms scoped, possibly over-budget whole-page on very large pages, and then the absence is UNPROVEN). A probe that found closed shadow roots or did not complete keeps the result INCONCLUSIVE_TRUNCATED / COVERAGE_UNVERIFIED, naming closed-shadow-root / shadow-coverage-unverified; a passing absence prints "No driver-observable semantic node matching {predicate} was found within {scope|the whole page}; coverage verified (N nodes probed). K hidden candidates excluded."
  • Call qa_evidence at the moment a problem appears, before navigating away or changing the state. Missing permissions, truncation, and driver rejection are boundaries, not passes.

Visual assertions: trust the verdict, never the narration

  • qa_assert kind:"visual" is ADVISORY and never changes pass/fail. Trust verdict and confidence; they are the model's answer.
  • Never quote details from reasoning as observed fact. It is unverified model narration and it invents detail: in a live run the model correctly answered yes (confidence 1.00) to "is the serif WIKIPEDIA wordmark present" and then narrated "with the puzzle globe logo", which was not on the page. Every advisory record carries reasoningTrust: "unverified-model-narration", and report.md renders the text as a labelled "model narration" blockquote.
  • Need a detail confirmed? Ask a separate visual question about it, or prove it with node-present / node-absent / page-url.
  • A visual assertion takes its own fresh settled observation before capturing, so it works right after qa_act or qa_evidence; no extra qa_observe is needed. A capture you pinned with visual_fingerprint is never silently refreshed, so a stale pin is refused by design.
  • A visual finding carries settle: { stable, passes, budgetMs } from the observation it captured from. When settle.stable is false the finding also carries captureSettled: false: the advisory verdict is over a view that never stopped changing, so it proves nothing about the page. A qa_evidence visual capture carries the SAME settle + captureSettled: false vocabulary.

Export rule

Call qa_record_export with the same owner and a .json output_path whose parent already exists under the current workspace or temporary directory.

Export is deliberately strict:

  • an action target must resolve uniquely in its preceding observation by non-empty role + accessible name — except that role drift on real pages (a server-rendered control replaced by a hydrated component that renders the same accessible name under a DIFFERENT role: Wikipedia's search input textbox -> combobox once the typeahead mounts) makes the LIVE role a fragile identity, so the action target is exported NAME-only whenever the accessible name is non-empty and unique in the recorded baseline view, with the live role kept only as an advisory roleHint (accepted and ignored for matching; drifted node-value assertions follow the same name-only rule);
  • every exported step gets an assertion synthesized from and evaluated against the immediate fresh observation after that action; a fill is proven by its own target's value (node-value);
  • unknown receipts require an observable semantic delta or URL change (a fill's own value counts);
  • rejected, failed, undispatched, unobserved, redacted, unnamed, duplicate, volatile, and Replay-unsupported actions are excluded with a structured reason;
  • a delta whose accessible name is a concatenation of its children's text (an ordering-fragile container such as a search/list/listbox whose name exceeds ~80 characters) is never exported as a proof — skipped in favour of a sound delta, or FRAGILE_PROOF_ONLY when it is the only change;
  • if no proven step remains, export returns NO_PROVEN_STEPS and writes no file.

Inspect excludedActions: exclusion is not success. Run qa_replay_run on the exact exported file without hand editing it. Only a pass report closes Explore→Replay. A failed run carries a machine failure.code for recognized non-results — INCONCLUSIVE_UNSTABLE (a view that never settled) and TARGET_NOT_UNIQUE (an ambiguous action target) — so you never have to parse prose to tell them from an ordinary assertion failure.

Role drift on replay (QA-BL-064): a recorded role+name ACTION target that matches NOTHING in the deciding view (after the one escalated read when that view is truncated) falls back to a NAME-only match iff exactly one node carries the same non-empty accessible name — the node is present under a drifted role, not outside the window — and the step discloses targetResolution: { mode: "name-only", recordedRole, observedRole } in report.json/report.md. The fallback is a refusal, never a guess, when the name is empty or matches two or more nodes (TARGET_NOT_UNIQUE with "present under a different role: recorded X, observed Y"); a zero-match failure otherwise says "the target is absent from the returned window" (truncated, INCONCLUSIVE_TRUNCATED) or "the target is absent from a complete view". The same hydration swap can land BETWEEN resolution and dispatch: a TARGET_CHANGED refusal (identity staleness — nothing was dispatched) is retried WITHIN the settle budget, never once (QA-BL-070): a fresh settled observation, a re-resolution of the SAME semantic target, and a re-dispatch, repeated until the pair lands, widening the budget ONCE through the shared session gate when the retry exhausts it (the same machinery assertion retries use; cause assertion-retry). The step discloses targetChangedRetries: N (the refusal count) in report.json/report.md, and the driver's last refusal receipt (verbatim reason plus any additive fields such as changed) rides on the step so triage sees WHAT changed. Every other rejection — a policy/safety refusal, TARGET_NOT_UNIQUE, a target absent from a COMPLETE view — stays a hard stop with zero retries. When the budget is exhausted with the target still changing identity, the step is inconclusive with reason INCONCLUSIVE_UNSTABLE and assertionPassed: false (the run is inconclusive, NEVER fail): "the target kept changing identity between resolution and dispatch for the whole settle budget (N retries): the page did not hold still, so the step is unproven". The SAME refusal on a within read of the scoped path walk (QA-BL-073: an ancestor changed identity between its parent read and the scoped read) runs the SAME bounded retry — re-resolving the level from the level above with a fresh settled read, counting into the SAME targetChangedRetries — and its exhaustion is the same inconclusive / INCONCLUSIVE_UNSTABLE classification with a message naming the level ("path level N (<what>) kept changing identity …") and the driver's changed/before/after verbatim as scopeIdentityRefusal; only TARGET_CHANGED is ever retried (policy refusals, REF_UNKNOWN, OBSERVATION_REQUIRED, SCOPE_UNAVAILABLE stay hard). A CONTENT-named container whose aggregated name changed between reads is never a refusal at all: the driver reports it informationally (scope.nameChanged), and the step records scopeNameChanged: true.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.