vision-memory-mcp
Skill24 starsChanged 3 months agoMITView on GitHub
Teaches the agent to use the Visual Memory MCP server to cache webpage and application screenshots, matching layout states and avoiding redundant LLM vision calls.
- Reads credentials
6 other projects use a version of this, 6 of them edited.
Adopted by
Most recently taken up by
- putervision/vision-memory-mcpedited, first seen Oct 8, 2026
- putervision/state-memory-mcpedited, first seen Sep 28, 2026
- putervision/world-model-mcpedited, first seen Sep 26, 2026
- putervision/behavior-mcpedited, first seen Sep 26, 2026
- putervision/agent-reasoning-mcpedited, first seen Sep 26, 2026
- putervision/spcedited, first seen Aug 20, 2026
Used with
Projects with it also use agent-reasoning-mcp, behavior-mcp and 1 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
putervisionSkillagent-reasoning-mcp
Teaches the agent to use the Strategic Agent Reasoning MCP server for BDI goals, utility scoring, risk evaluation, and replanning.
Also in 5 projects that use this
- putervisionSkill
behavior-mcp
Teaches the agent to use the Behavior MCP server for ~60Hz in-browser behavior trees, triggers, and recordings.
Also in 5 projects that use this
- +1 more from putervision
Other versions6 projects use a version of this.
Word for word · 0
None found.
Edited · 6
- putervision/state-memory-mcp★ 114100% same
- putervision/world-model-mcp★ 50100% same
- putervision/vision-memory-mcp★ 48100% same
- putervision/spc★ 41100% same
- putervision/behavior-mcp★ 24100% same
- putervision/agent-reasoning-mcp★ 22100% same
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository9 other files it gives its agents.
- .agents/AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- .github/copilot-instructions.mdCopilot instructions
- .cursor/rules/state-graph-mcp.mdcCursor rule
- .cursor/rules/state-memory-mcp.mdcCursor rule
- .cursor/rules/vision-memory-mcp.mdcCursor rule
- state-memory-mcpSkill
- video-ingestSkill
- webcrypt-mcpSkill
The file
We split this skill into the rules it gives an agent
8 rules. 1 is common, 0 are distinctive, and 7 are very specific to this project.
1
Common
Most of the 6 edited versions we read keep these.
0
Distinctive
This author's own choices. Worth a look.
7
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 14
Orient: Call get_session_context to load active transitions and recent visual states.
- Very specific: line 15
Search (Optional): Call recall_memory to search past visual states by text query or image query.
- Very specific: line 16
Ingest/Verify: Call analyze_screenshot with the base64 screenshot before calling any vision LLMs.
- Very specific: line 17
Cache Hit: If is_known is true, read the returned description and do NOT call your vision LLM.
- Very specific: line 18
Cache Miss: If is_known is false, inspect the image with your vision model, summarize the layout, and register it back by calling …
- Very specific: line 20
Snapshotting: Call manage_snapshot (action: "save") when reaching milestones, and manage_snapshot (action: "diff") to check for visual …
- Very specific: line 46
Google Antigravity (
~/.gemini/config/config.json): Add these rules to your "globalPermissionGrants" -> "allow" list: - Common: line 53
Run these commands in the terminal for management and analytics:
“Common” is measured: kept by at least 6 in 10 of the 6 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Visual Memory (vision-memory-mcp)
This project utilizes vision-memory-mcp to cache visual states, record layout transitions, and avoid repetitive LLM vision calls.
1. Priority Order & Checklist
Whenever you capture a screenshot, examine a webpage, or need to verify a visual state, you MUST run this sequence:
- Orient: Call
get_session_contextto load active transitions and recent visual states. - Search (Optional): Call
recall_memoryto search past visual states by text query or image query. - Ingest/Verify: Call
analyze_screenshotwith the base64 screenshot before calling any vision LLMs. - Cache Hit: If
is_knownistrue, read the returneddescriptionand do NOT call your vision LLM. - Cache Miss: If
is_knownisfalse, inspect the image with your vision model, summarize the layout, and register it back by callinganalyze_screenshotwith both thescreenshotanddescriptionparameters. - Log Transitions: Right after taking any UI action (click, type, navigate, scroll), call
record_outcometo build the navigation graph. - Snapshotting: Call
manage_snapshot(action: "save") when reaching milestones, andmanage_snapshot(action: "diff") to check for visual regressions.
2. Complete Tool Reference
| Tool Name | Key Inputs | Description |
analyze_screenshot | screenshot? (base64), file_path?, description?, items? | Main ingestion (single or batch) and visual state retrieval tool. |
recall_memory | query?, screenshot?, file_path?, strategy?, limit? | Search visual memory by text query or image query (read-only). |
record_outcome | from_state_id, to_state_id?, action, action_type? ('blocker' | 'click' | etc.) | Record UI action transitions or log visual blockers for state-memory. |
get_navigation_paths | from_state_id?, to_state_id?, to_description?, max_hops? | Find historical path or instructions between states. |
predict_next_action | current_state_id, goal_description?, goal_state_id? | Predict best next UI action and grounded element handles (target_selector, target_coords). |
compare_states | state_a_id & state_b_id OR video_a_id & video_b_id | Compare two states visually (has_layout_change) or compare video runs. |
get_session_context | include_recent?, include_frequent? | Get recent/frequent states, transition graphs, disk stats, cache metrics, and version info. |
manage_snapshot | action ('save' | 'diff' | 'export' | 'restore'), name?, archive_json? | Unified snapshot management for visual checkpoints and regression detection. |
manage_visual_spec | action ('set' | 'verify' | 'list'), name?, screenshot?, tolerance? | Register and verify visual design contract baselines (Visual SDD). |
manage_video | action ('ingest' | 'search' | 'timeline'), file_path?, query?, video_id? | Ingest WebM/MP4 recordings, search video keyframes, or retrieve timelines. |
create_evidence_pack | keyframe_state_ids, source_video_id?, linked_state_memory_nodes? | Package immutable evidence packs linking video keyframes to state-memory DAGs. |
export_trajectories | format? ('json' | 'llava' | 'qwen2_vl' | 'joint'), trace_id? | Export multimodal trajectories for model fine-tuning or joint workflow exports. |
undo_visual_mutation | type? ('state' | 'transition' | 'any') | Revert the last visual state ingestion or transition edge addition. |
forget_state | state_id | Purge a specific state and vector embedding for privacy. |
wait_for_visual_state | target_state_id, timeout_ms? | Poll for target visual state until present or timeout occurs. |
3. Agent Permissions & Auto-Run Configuration
To bypass confirmation dialogs when running CLI cache commands or reading/writing brain images, add these allows to your configuration:
- Google Antigravity (
~/.gemini/config/config.json): Add these rules to your"globalPermissionGrants"->"allow"list:"command(vision-memory-mcp)"(Allows running any query/ingest command prefix)"read_file(.*\\.gemini/antigravity/brain/.*)"(Allows reading brain screenshots)"write_file(.*\\.gemini/antigravity/brain/.*)"(Allows saving brain snapshots)
4. CLI Commands Reference
Run these commands in the terminal for management and analytics:
vision-memory-mcp init [-y|--yes]: Scaffold workspace .vision-memory-mcp/, .gitignore, .env, and IDE agent rules.vision-memory-mcp init-global: Re-initialize across all projects registered in ~/.vision-memory-mcp/projects.json.vision-memory-mcp doctor: Health check storage writability, sharp bindings, Node runtime, and sub-directory Git repos.vision-memory-mcp audit: Audit sub-directory Git repos, submodules, database locations, and total visual states.vision-memory-mcp inspect: Display stored visual states in an ASCII table.vision-memory-mcp metrics: Calculate cache hit rate, token savings, and ROI.vision-memory-mcp view: Open an interactive force-directed graph visualizer in the browser.vision-memory-mcp export --format [json\|mermaid\|html] --out [file]: Export the memory graph.vision-memory-mcp prune: Purge expired or low-access states.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.