windows-computer-use-mcp

# windows-computer-use-mcp — Full Documentation

Windows **computer use agent**: structured native UI automation via Pywinauto/UIA.
Read docs/SAFETY.md before production use.

## Tools

### analyze_winapp (NEW)
Auto-crawl a Windows app to produce UI element map + screenshot portfolio.
Operations: crawl (full tree), discover (keyboard shortcut probe), portfolio (named state captures).
Shows a "CUA at work" blinking red HUD + e-stop button during operations.
Target window is auto-refocused before each action.

### automation_windows
Window management: list, find, focus, minimize, maximize, close, position, title.

### automation_elements
UI element interaction: click, set_text, get_text, list, info, wait, exists.

### automation_mouse
Mouse control: move, click, drag, scroll, position, telemetry.

### automation_keyboard
Keyboard input: type, press, hotkey, hold.

### automation_visual
Screenshots, OCR (extract_text), template matching (find_image), highlighting.

### automation_assert
UI verification: hash, diff, wait_stable, assert_changed, assert_template, assert_text.

### automation_dialog
File dialog path entry: set_path, confirm, submit_path.

### automation_shortcut
Semantic app shortcuts by action name (e.g. vroidstudio/export_vrm).

### automation_task
Closed-loop task runner with retry, refocus recovery, per-step evidence.

### automation_system
System utilities: status, info, clipboard, processes, start_app, wait_for_window.

### get_desktop_state
Full desktop UI tree capture (ax/som/vision modes).

### get_window_state
Single-window UI state + annotated screenshot (Cua-shaped).

### global_keylogger (opt-in)
Session keyboard capture via pynput. Shows CUA HUD while active.

## CUA HUD

All long-running operations show a blinking red "CUA at work" overlay:
- `analyze_winapp` (crawl, discover, portfolio)
- `global_keylogger` (when active)
- Features: blinking red/white text, e-stop button, auto-refocus

## Architecture

```
Agent → MCP (stdio) → windows-computer-use-mcp → UIA / Win32 API → Windows app
Agent → HTTP → windows-computer-use-mcp (10789) → same
Web UI → Vite (10788) → proxy → API (10789)
Tauri → WebView → embedded backend (10789)
```

## Cross-connection: autohotkey-mcp

| Need | Use |
|------|-----|
| UIA element tree, OCR, structured UI automation | windows-computer-use-mcp |
| Raw input recording/replay, hotkey macros | autohotkey-mcp |
| Both show "CUA at work" HUD during operations | |