Verify Obsidian Chat
Skill8 starsChanged yesterdayMITView on GitHub
Use when you need to prove behavior in the real Obsidian app for this plugin: launch a throwaway vault, drive a user journey, capture evidence, and check every entry point in the feature map.
- Reads credentials
In this repository15 other files it gives its agents.
- AGENTS.mdAGENTS.md
- Break ChangeSkill
- cavecrewSkill
- caveman-commitSkill
- caveman-compressSkill
- caveman-helpSkill
- caveman-reviewSkill
- cavemanSkill
- caveman-statsSkill
- Chaos MonkeySkill
- factorySkill
- Probe FeatureSkill
- Ship ChangeSkill
- Verify PRSkill
- WorktreeSkill
The file
We split this skill into the rules it gives an agent
7 rules. About 0 are common, 6 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈6
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 42
Before trusting a run, check:
- Distinctive: line 71
Use stable handles, not coordinates.
- Distinctive: line 72
Verify side effects (note exists, content changed, JSONL written) alongside what is visible.
- Distinctive: line 91
When a change adds, removes, or changes a user-visible entry point, record two clips:
- Very specific: line 110
Delete or trash notes the run created.
- Distinctive: line 123
Async saves: After changing a setting, poll the plugin settings value.
- Distinctive: line 130
Start with the index, then update the map when entry points change:
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Verify Obsidian Chat
The unit layer fakes Obsidian. This skill drives the real app. It is adapted from
pstack's create-verification-skill by Lauren Tan (MIT), narrowed to this repo.
Launch
Two ways in:
npm run probe -- --scenario settings-gate # recorded exploratory run
npm run verify -- e2e test/e2e/specs/smoke.e2e.tsprobe starts an isolated Xvfb display and records the whole display with ffmpeg,
so Obsidian settings popouts are in frame. verify -- e2e runs a committed spec
without recording.
Both build first. Scripted probes use the E2E stream bundle; live probes use the
production bundle. wdio.conf.mts loads test/e2e/probe/** only when
AGENTIC_CHAT_PROBE=1.
verify -- e2e runs on the host display by default. Every Obsidian window asks
the window manager to activate it, which steals the cursor. Run the lane on a
nested Xephyr display instead:
npm run nested -- npm run verify -- e2e test/e2e/specs/smoke.e2e.tsThis is a real GUI Obsidian, not AGENTIC_CHAT_E2E_HEADLESS=1. --display :N
reuses an Xephyr; --screen 390x844 matches the narrow viewport. Probes already
own Xvfb, so they do not need the wrapper.
Doctor
Before trusting a run, check:
- The plugin under test is the one you just built.
proberebuilds by default;verify -- e2erunsscripts/build-e2e.mjsfirst. - The vault is the throwaway sandbox, not
~/MyTestVault. Session IDand Obsidian version appear in the reporter header.- For live runs,
AGENTIC_CHAT_API_KEYis present. A missing key skips tests and still exits 0; read the counts. DISPLAYpoints at Xvfb for probes. Never record the host desktop by accident.
Drive
Verified selectors and helpers:
| surface | handle |
| open chat | browser.executeObsidianCommand("agentic-chat:open-chat") |
| composer | .agentic-chat-input, .agentic-chat-send |
| transcript | .agentic-chat-messages, .agentic-chat-assistant .agentic-chat-text |
| tool card | .agentic-chat-tool, .agentic-chat-step-diff-line.is-add |
| approval | .agentic-chat-approval, button=Allow, button=Deny |
| info block | .agentic-chat-info |
| settings | test/e2e/support/settings-ui.ts |
| vault state | browser.executeObsidian(...) |
Settings on Obsidian 1.13+ open in a popout window. settings-ui.ts switches
handles for you. Do not assume the settings DOM is in the main window, and do not
assume a single window when recording.
Use stable handles, not coordinates. Drive the user path, not an internal setter. Verify side effects (note exists, content changed, JSONL written) alongside what is visible.
Evidence
For a UI claim, produce:
session.mp4(default; GIF only when you pass--gif) or a committed e2e assertion- keyframes at each mark (
shots/) metrics.json: viewport, overflow, element rects, console errorstimeline.json: one entry perrecord.mark(label)report.md: hypothesis, observed, decision, open questions
The video is evidence for a human. The assertion is the regression guard. Do not promote a recording into CI as a pixel comparison; timing, model text, and cursor position differ every run.
Entry-point changes
When a change adds, removes, or changes a user-visible entry point, record two clips:
- desktop
1280x1024 - mobile-width
390x844
The 390x844 clip is a narrow desktop window, not real Obsidian Mobile. State
that honestly. Each clip needs a “what to watch” list with labels or timestamps.
A removal also needs absence proof: the command is gone from /help and the
command registry, the UI entry point is gone, and the docs and feature map no
longer claim it. Use @break-change removal mode. If clips are impossible, put a
waiver in the PR.
State the mode. mode: live means a real model answered. mode: scripted uses
the __AGENTIC_CHAT_E2E_TURNS__ hook. Never present scripted as live.
Cleanup
Delete or trash notes the run created. Stop the instance you started; never kill
by process name. Keep evidence after teardown. probe owns Xvfb and ffmpeg and
cleans both up.
Gotchas
- Popout settings. Capture the display, not a window. Without a window manager
the popout can be placed oddly; if it is off-screen, run
openboxinside Xvfb. - Headless is not enough.
--ozone-platform=headlesshas no accessibility tree, sogetText()lies. Use DOM text orexecuteObsidianwhen headless. - Proxy. Chromedriver must stay off the proxy;
wdio.conf.mtsstrips it. - Stale bundle.
main.jsis gitignored. Rebuild before asserting. - Live skip. An empty key makes the live spec skip, not fail.
- Async saves. After changing a setting, poll the plugin settings value; do not assume the save landed.
- Scripted turn order. One tool call consumes one turn, and a denied call still needs its follow-up text turn. Keep the queue aligned.
Feature map
Start with the index, then update the map when entry points change:
- Index and proof rules
- Open and compose
- Chat turn
- Approval and permissions
- Sessions and tabs
- Commands and skills
- Settings
- MCP and OAuth
- Subagents and context
Each entry names every way a user reaches the feature and how to prove it. A proof that tests one entry point is incomplete when the map lists others. The status values and proof rules are in the index.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

