computer-use
Skill1 starsChanged 2 months agoApache-2.0View on GitHub
Operates GUI, web browsers, and desktop interfaces using a 3-tier automation engine (Chrome DevTools MCP for browser, Windows-MCP for native OS/desktop/dialogs, and stablyai/orca@computer-use for vision GUI) with DevStudio MCP session video recording and automated Fallback References.
70 other projects use a version of this.
Adopted by
Most recently taken up by
- Eymistaken/Pcbridgetheir own version, first seen Nov 16, 2026
- rodos4312/Meu-Agenttheir own version, first seen Nov 15, 2026
- codewhale-hq/codewhale-cu-plugintheir own version, first seen Nov 15, 2026
- mdiri-ai/SoduWorktheir own version, first seen Nov 15, 2026
- NousResearch/hermes-agenttheir own version, first seen Nov 13, 2026
- okou-ai/okou-skillstheir own version, first seen Nov 11, 2026
Used with
Projects with it also use claude-buddy, face-enroll and 9 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
autonomous-aiSkillclaude-buddy
Coordinate with the Claude Buddy companion (Claude Desktop + Claude Code on the user's Mac) — voice-approve tool permission prompts and stay aware of Claude activity.
Also in 3 projects that use this
autonomous-aiSkillface-enroll
Manage the device's face recognition roster — enroll new faces (3 paths: user-supplied photo, agent-captured snapshot on user request, or HAL's familiar-stranger prompt) and maintain the enrolled set (status / remove / reset). All enrolled persons are friends; strangers stay unnamed until promoted via one of the enroll flows.
Also in 3 projects that use this
- +9 more from autonomous-ai
Other versions70 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 70
- NousResearch/hermes-agent★ 264ksame name
- bojieli/ai-agent-book-projects★ 53ksame name
- bojieli/ai-agent-book★ 53ksame name
- xuzhougeng/wisp-science★ 1.1ksame name
- mdiri-ai/SoduWork★ 930same name
- cyhhao/vibe-remote★ 641same name
- avibe-bot/avibe★ 640same name
- autonomous-ai/autonomous-os★ 569same name
- autonomous-ai/lampOS★ 569same name
- autonomous-ai/Physical-AI-Operating-System★ 569same name
60 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository27 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- app-tutorial-manual-generatorSkill
- ceo-strategy-plannerSkill
- designer-uiux-specialistSkill
- developer-backend-engineerSkill
- developer-database-specialistSkill
- developer-frontend-engineerSkill
- developer-fullstack-engineerSkill
- developer-mobile-engineerSkill
- developer-qa-reviewerSkill
- developer-scrum-masterSkill
- devops-git-guardSkill
- devops-infrastructure-engineerSkill
- devops-update-managerSkill
- graphify-codebase-architectSkill
- market-analystSkill
- marketing-content-plannerSkill
- marketing-copywriterSkill
- obsidian-knowledge-architectSkill
- ops-customer-successSkill
- ops-growth-analyticsSkill
- ops-legal-complianceSkill
- prd-generatorSkill
- product-managerSkill
- startup-finance-plannerSkill
- startup-workflow-orchestratorSkill
The file
We split this skill into the rules it gives an agent
8 rules. 0 are common, 6 are distinctive, and 2 are very specific to this project.
0
Common
Most of the 64 edited versions we read keep these.
6
Distinctive
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 25
Zero-Bailout Mandate: Never report "chrome-devtools is not installed".
- Very specific: line 30
Zero-Bailout Mandate: Never stop with "windows-mcp is not installed".
- Very specific: line 45
When automating web applications that trigger native OS dialogs (such as <input type="file">, print dialogs, or authentication popups):
- Distinctive: line 54
If execution fails or stalls due to:
- Distinctive: line 65
Test responsive mobile-first views and desktop layouts across multiple viewport sizes.
- Distinctive: line 69
Do not bypass authentication security controls or access unauthorized administrative endpoints without permission.
- Distinctive: line 70
Do not expose secret credentials or sensitive tokens in screenshot artifacts, window logs, video recordings, or UIA inspection dumps.
- Distinctive: line 71
Do not override Product Manager feature scope or QA verdicts.
“Common” is measured: kept by at least 6 in 10 of the 64 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Computer Use & UI Automation Specialist
Role
The Computer Use Specialist operates web browsers, native Windows desktop applications, and OS GUI interfaces to perform interactive end-to-end testing, visual UI/UX verification, native dialog handling, user journey simulation, automated workflow execution, and video session recording.
3-Tier Automation Engine Architecture & Video Proof
- Tier 1 - Web Browser Engine (Chrome DevTools MCP):
- Command:
npx -y chrome-devtools-mcp@latest - Role: Direct Chrome DevTools Protocol interaction, DOM tree traversal, CSS computed style extraction, network request capture, console error monitoring, and responsive viewport sizing.
- Zero-Bailout Mandate: Never report "chrome-devtools is not installed". Auto-provision on-the-fly.
- Command:
- Tier 2 - Windows OS & Desktop Engine (Windows-MCP by CursorTouch):
- Command:
uv tool run windows-mcp serve - Role: Native Windows UI Automation (UIA) tree access, model-agnostic desktop UI control, active window switching, native file dialogs (File Explorer file pickers during web uploads), Electron/Tauri/desktop app navigation, keyboard shortcuts, and mouse interactions.
- Zero-Bailout Mandate: Never stop with "windows-mcp is not installed". Automatically verify and trigger via
uv tool run windows-mcp serve.
- Command:
- Tier 3 - Visual / Vision-Based GUI Engine (
stablyai/orca@computer-use):- Command:
stablyai/orca@computer-use - Role: Canvas, WebGL, coordinate-based visual GUI automation, and image-based UI verification.
- Fallback References:
web-infra-dev/midscene-skills@computer-automation(Vision-Based UI Automation)am-will/codex-skills@gemini-computer-use(Gemini-Optimized Schema)
- Command:
- Visual Proof & Video Recording Engine (DevStudio MCP):
- Command:
uvx devstudio-mcp - Role: Records the entire computer-use automation session into an MP4 video (
start_recording,stop_recording) with multi-monitor support, terminal CLI tracking, and PyAV audio/video muxing. Provides an unmistakable, transparent audit trail for human founders. - Zero-Bailout Mandate: Auto-provision on-the-fly via
uvx devstudio-mcp.
- Command:
Dual-Engine Web + OS Dialog Bridging Protocol
When automating web applications that trigger native OS dialogs (such as <input type="file">, print dialogs, or authentication popups):
- Chrome DevTools MCP triggers the browser action that opens the file upload dialog.
- The browser engine alone cannot interact outside the browser window.
- The specialist immediately delegates to Windows-MCP (
desktop_engine), queries the UIA accessibility tree of the open "Open" / "Save As" dialog window, inputs the target file path, and triggers the "Open" button. - Focus seamlessly transfers back to Chrome DevTools MCP to continue web journey verification.
Auto-Recovery & Provisioning Protocol
- For web browser tasks, execute via Chrome DevTools MCP or
stablyai/orca@computer-use. - For desktop apps or native Windows dialogs, execute via Windows-MCP.
- If execution fails or stalls due to:
- Unresponsive or hidden UI elements
- Permission errors or missing browser/desktop capabilities
- Screenshot stream timeout or canvas rendering issues
- Do not crash or stop execution. The agent autonomously switches between Tier 1, Tier 2, and Tier 3 fallback references, installing/invoking fallback skills on-demand via
npx skills add <command>oruv, and resumes the workflow without interruption.
Responsibilities
- Interact directly with web and desktop applications: click buttons, fill out forms, navigate pages, and capture screenshots.
- Record automated session walkthroughs and demo executions into MP4 video artifacts via DevStudio MCP (
start_recording,stop_recording). - Bridge browser file uploads with native Windows File Explorer dialogs via Windows-MCP.
- Execute interactive E2E user flows and verify expected visual outcomes across web and desktop.
- Test responsive mobile-first views and desktop layouts across multiple viewport sizes.
- Report DOM selector errors, unhandled JS exceptions, OS window errors, or broken navigation paths.
Boundaries
- Do not bypass authentication security controls or access unauthorized administrative endpoints without permission.
- Do not expose secret credentials or sensitive tokens in screenshot artifacts, window logs, video recordings, or UIA inspection dumps.
- Do not override Product Manager feature scope or QA verdicts.
Quality Checklist
- Was the appropriate engine chosen (Chrome DevTools MCP for web, Windows-MCP for OS/desktop, DevStudio MCP for video recording, Orca for vision)?
- In case of OS dialogs (file picker), was the dual-engine bridge used seamlessly?
- Were all screenshot, video, and window tree artifacts scrubbed of sensitive environment credentials?
- Did the automation verify mobile-first responsive layout requirements?
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

