agent-desktop-windows
Skill2 starsChanged 17 days agoApache-2.0View on GitHub
Windows platform guide for the agent-desktop CLI. Lists per command group what works on Windows and what refuses, the first-contact gotchas (PowerShell ref quoting, minimized windows, headed-only input), and where to find shell surfaces, notifications, the cursor overlay, UIPI elevation, Chromium/Electron settle behavior and troubleshooting. Triggers on: "windows desktop automation", "UIA tree", "automate Windows app", "agent-desktop on Windows", "elevated window input blocked", "E_ACCESSDENIED", or any Windows GUI interaction task.
2 other projects use a version of this, 2 of them edited.
Adopted by
Most recently taken up by
- vercel-labs/agent-desktopedited, first seen Nov 1, 2026
- lahfir/agent-desktopedited, first seen Oct 25, 2026
Other versions2 projects use a version of this.
Word for word · 0
None found.
Edited · 2
- lahfir/agent-desktop★ 24k97% same
- vercel-labs/agent-desktop★ 24k97% same
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository8 other files it gives its agents.
- AGENTS.mdAGENTS.md
- architecture-diagramSkill
- technical-writerSkill
- agent-desktop-ffiSkill
- agent-desktop-linuxSkill
- agent-desktop-macosSkill
- agent-desktopSkill
- jev-desktopSkill
The file
We split this skill into the rules it gives an agent
2 rules. About 0 are common, 0 are distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 54
Every element of a minimized window reports offscreen and ref actions fail.
- Very specific: line 110
Read any file from the binary: agent-desktop skills get agent-desktop-windows references/troubleshooting.md.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
agent-desktop-windows
This file covers only what differs on Windows. The observe-act loop, refs, JSON envelope, sessions and tracing are the same on every platform.
Requires Windows 10 1809+ / Windows Server 2019+ (x64 or ARM64). Run it in the interactive desktop session of the signed-in user. Session 0 (services and SYSTEM tasks), Server Core, the secure desktop (UAC prompts, the sign-in screen), a locked desktop and another user's session are not supported: observation, actions and capture there fail or return nothing.
Capability Table
| Group | Commands | Status on Windows |
| Observation | snapshot, find, get, is, screenshot | Works. Surfaces are window, focused, a Chromium modal as sheet, and an open application menu as menu |
| Surfaces | list-surfaces | Works. Lists window, focused and sheet per process, plus menu with item_count while a menu is open |
| Shell surfaces | open-system-surface | Works for start-menu, taskbar, system-tray, system-tray-overflow and action-center, and returns the window snapshot --surface <kind> reads. quick-settings returns INVALID_ARGS naming action-center |
| Ref interaction | click, right-click, type, set-value, clear, select, toggle, check, uncheck, expand, collapse, scroll, scroll-to | Works, semantic in headless and --headed mode. right-click and type need --headed and return POLICY_DENIED headless |
| Multi-click and focus | double-click, triple-click, focus | Works with global --headed only |
| Keyboard and mouse | press, hover, drag, mouse-click, mouse-move, mouse-wheel | Works. Cursor-moving input needs --headed. First Contact covers press --app |
| Held input | key-down, key-up, mouse-down, mouse-up | Fails closed on every platform with ACTION_NOT_SUPPORTED. A headless mouse-down or mouse-up returns POLICY_DENIED first |
| App and window | launch, close-app, list-windows, list-apps, focus-window, resize-window, move-window, minimize, maximize, restore | Works. First Contact covers the name launch takes |
| Displays | list-displays | Works |
| Clipboard | clipboard-get, clipboard-set, clipboard-clear | Works; typed text and image content |
| Wait | wait | Works for ms, --element, --window, --text, --menu, --menu-closed, --event and --notification |
| Notifications | list-notifications, dismiss-notification, dismiss-all-notifications, notification-action | Works over the Action Center. Pass --headed when the center is closed |
| Cursor overlay | cursor-overlay | Works. enable returns data.rendered: true only after the renderer acknowledges over its pipe. See references/shell-and-overlay.md |
| System | status, permissions, version, batch, skills, session, trace | Works |
First Contact
- Quote every ref in PowerShell. PowerShell reads a bare
@tokenas its splatting operator and deletes the argument before the binary sees it, soset-value @s8f3k2p9:e1 hifailsINVALID_ARGS. Writeset-value '@s8f3k2p9:e1' hi. cmd.exe and bash need no quoting. - A window behind another window is drivable. A minimized one is not.
Every element of a minimized window reports
offscreenand ref actions fail. Runrestore --app <image>and snapshot again. Do not usefocus-windowby reflex: it steals the user's foreground. - No permission dialog exists.
permissionsreportsautomationasnot_required. Elevation boundaries are the only access control. - Chromium and Electron apps read thin before they settle. Pass a longer
--timeout-mstosnapshotinstead of concluding the tree is empty. - Blocked shortcuts.
alt+f4,win+l,win+d,alt+tab,win+r,win+x,win+e,ctrl+shift+escand their modifier supersets are refused without--force. In combos,cmdandmetaare the Windows key, someta+cis a shell shortcut. Writectrl+cto copy. typeneeds--headed. It is keystroke synthesis throughSendInput, so a strict-headlesstypereturnsPOLICY_DENIED. Useset-value, or pass--headed. It endsdelivered_unverified(verification_scope: "unavailable") when the field exposes no readable Text selection, and can verify when one is readable: confirm with a freshget.press --appwrites to the foreground queue with no per-process targeting, and takes the machine-wide interaction lease becauseSendInputholds modifier keys in the session keyboard. A headless press to a non-frontmost app fails closed. A non-interactive caller (service, scheduled task, CI job) getsdelivered_unverified.launchtakes an absolute path or a bare name under System32 or the Windows directory, with or without.exe(notepadornotepad.exe), not a display name. Uselist-appsto find a running app.- Notifications:
list-notificationswith the Action Center closed needs--headed. Single-itemdismiss-notificationandnotification-actionneed--headedand--expected-appor--expected-titlefor the entry at INDEX. - Surfaces: an open or context menu is the
menusurface. Windows does not advertise amenubarsurface. - Commands that raise shell chrome take the foreground. Strict headless
refuses them with
POLICY_DENIEDbefore anything is raised. - Stateful actions run once, then verify by re-reading.
delivered_verifiedmeans the readback matched.ACTION_FAILEDwithdetails.kind: "post_action_verification"means the action landed but the readback differs: readdetails.post_stateand do not retry. Aset-valuethe app reformats reads as failed. Seereferences/troubleshooting.md. - Unsigned binary warnings are expected. See
references/troubleshooting.md.
What differs on Windows
- Ref actions are semantic UI Automation calls in headless and
--headedmode, exceptright-click,double-click,triple-click,typeandfocus, which are physical or take focus and need--headed. - A ref resolves only when it was the only element in the window its snapshot captured with its role, native id, name and description, and exactly one live element still matches. Uniqueness counts the whole window, not the shallow tree a
--skeletonor--max-depthsnapshot presents, so a twin below the cut still refuses the ref. Identical twins, such as several identicalDeletebuttons, are refused withAMBIGUOUS_TARGET,details.kind: identity_not_unique, before anything reaches the app; a live second match issaved_ref_ambiguous, and a removed element isSTALE_REF,saved_ref_absent. Take a fresh snapshot after a list changes, and use--headed mouse-click --xywhen you must pick one of two identical controls. - The cursor overlay plays its per-action travel only for headless semantic actions. A
--headedaction moves the real cursor instead.
References
references/shell-and-overlay.md: open it foropen-system-surface, tray and taskbar surfaces, notifications, the cursor overlay, hosted (UWP) window identity and menu detection coverage.references/permissions-and-elevation.md: integrity levels, UIPI, blocked combos, protected processes, the cross-process interaction lease.references/chromium-and-electron.md: settle timing, covered-window hazard, WPF wrong-provider trap, unnamed web content.references/troubleshooting.md: symptom-to-cause map, action verification outcomes, saving a document headless, install warnings.
Read any file from the binary:
agent-desktop skills get agent-desktop-windows references/troubleshooting.md.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

