agentleFS
tardigrdetardigrde/obsidian-agentic-chat/.agents/skills/verify-obsidian-chat/SKILL.md

Verify Obsidian Chat

Skill8 starsChanged yesterdayMITView on GitHub

Use when you need to prove behavior in the real Obsidian app for this plugin: launch a throwaway vault, drive a user journey, capture evidence, and check every entry point in the feature map.

  • Reads credentials

The file

We split this skill into the rules it gives an agent

7 rules. About 0 are common, 6 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈6

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Before trusting a run, check:

    line 42
  2. Distinctive:

    Use stable handles, not coordinates.

    line 71
  3. Distinctive:

    Verify side effects (note exists, content changed, JSONL written) alongside what is visible.

    line 72
  4. Distinctive:

    When a change adds, removes, or changes a user-visible entry point, record two clips:

    line 91
  5. Very specific:

    Delete or trash notes the run created.

    line 110
  6. Distinctive:

    Async saves: After changing a setting, poll the plugin settings value.

    line 123
  7. Distinctive:

    Start with the index, then update the map when entry points change:

    line 130

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Verify Obsidian Chat

The unit layer fakes Obsidian. This skill drives the real app. It is adapted from pstack's create-verification-skill by Lauren Tan (MIT), narrowed to this repo.

Launch

Two ways in:

npm run probe -- --scenario settings-gate     # recorded exploratory run
npm run verify -- e2e test/e2e/specs/smoke.e2e.ts

probe starts an isolated Xvfb display and records the whole display with ffmpeg, so Obsidian settings popouts are in frame. verify -- e2e runs a committed spec without recording.

Both build first. Scripted probes use the E2E stream bundle; live probes use the production bundle. wdio.conf.mts loads test/e2e/probe/** only when AGENTIC_CHAT_PROBE=1.

verify -- e2e runs on the host display by default. Every Obsidian window asks the window manager to activate it, which steals the cursor. Run the lane on a nested Xephyr display instead:

npm run nested -- npm run verify -- e2e test/e2e/specs/smoke.e2e.ts

This is a real GUI Obsidian, not AGENTIC_CHAT_E2E_HEADLESS=1. --display :N reuses an Xephyr; --screen 390x844 matches the narrow viewport. Probes already own Xvfb, so they do not need the wrapper.

Doctor

Before trusting a run, check:

  • The plugin under test is the one you just built. probe rebuilds by default; verify -- e2e runs scripts/build-e2e.mjs first.
  • The vault is the throwaway sandbox, not ~/MyTestVault.
  • Session ID and Obsidian version appear in the reporter header.
  • For live runs, AGENTIC_CHAT_API_KEY is present. A missing key skips tests and still exits 0; read the counts.
  • DISPLAY points at Xvfb for probes. Never record the host desktop by accident.

Drive

Verified selectors and helpers:

surfacehandle
open chatbrowser.executeObsidianCommand("agentic-chat:open-chat")
composer.agentic-chat-input, .agentic-chat-send
transcript.agentic-chat-messages, .agentic-chat-assistant .agentic-chat-text
tool card.agentic-chat-tool, .agentic-chat-step-diff-line.is-add
approval.agentic-chat-approval, button=Allow, button=Deny
info block.agentic-chat-info
settingstest/e2e/support/settings-ui.ts
vault statebrowser.executeObsidian(...)

Settings on Obsidian 1.13+ open in a popout window. settings-ui.ts switches handles for you. Do not assume the settings DOM is in the main window, and do not assume a single window when recording.

Use stable handles, not coordinates. Drive the user path, not an internal setter. Verify side effects (note exists, content changed, JSONL written) alongside what is visible.

Evidence

For a UI claim, produce:

  • session.mp4 (default; GIF only when you pass --gif) or a committed e2e assertion
  • keyframes at each mark (shots/)
  • metrics.json: viewport, overflow, element rects, console errors
  • timeline.json: one entry per record.mark(label)
  • report.md: hypothesis, observed, decision, open questions

The video is evidence for a human. The assertion is the regression guard. Do not promote a recording into CI as a pixel comparison; timing, model text, and cursor position differ every run.

Entry-point changes

When a change adds, removes, or changes a user-visible entry point, record two clips:

  • desktop 1280x1024
  • mobile-width 390x844

The 390x844 clip is a narrow desktop window, not real Obsidian Mobile. State that honestly. Each clip needs a “what to watch” list with labels or timestamps.

A removal also needs absence proof: the command is gone from /help and the command registry, the UI entry point is gone, and the docs and feature map no longer claim it. Use @break-change removal mode. If clips are impossible, put a waiver in the PR.

State the mode. mode: live means a real model answered. mode: scripted uses the __AGENTIC_CHAT_E2E_TURNS__ hook. Never present scripted as live.

Cleanup

Delete or trash notes the run created. Stop the instance you started; never kill by process name. Keep evidence after teardown. probe owns Xvfb and ffmpeg and cleans both up.

Gotchas

  • Popout settings. Capture the display, not a window. Without a window manager the popout can be placed oddly; if it is off-screen, run openbox inside Xvfb.
  • Headless is not enough. --ozone-platform=headless has no accessibility tree, so getText() lies. Use DOM text or executeObsidian when headless.
  • Proxy. Chromedriver must stay off the proxy; wdio.conf.mts strips it.
  • Stale bundle. main.js is gitignored. Rebuild before asserting.
  • Live skip. An empty key makes the live spec skip, not fail.
  • Async saves. After changing a setting, poll the plugin settings value; do not assume the save landed.
  • Scripted turn order. One tool call consumes one turn, and a denied call still needs its follow-up text turn. Keep the queue aligned.

Feature map

Start with the index, then update the map when entry points change:

  • Index and proof rules
  • Open and compose
  • Chat turn
  • Approval and permissions
  • Sessions and tabs
  • Commands and skills
  • Settings
  • MCP and OAuth
  • Subagents and context

Each entry names every way a user reaches the feature and how to prove it. A proof that tests one entry point is incomplete when the map lists others. The status values and proof rules are in the index.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.