agentleFS
Sign inStart free
nseney1nseney1/Soma-Governance/skills/visual-analyst/SKILL.md

Visual Analyst

Skill4 starsChanged 10 days agoApache-2.0View on GitHub

Analyzes screenshots, game frames, and UI captures to extract visual state, coordinates, and regressions.

1 other project uses a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

6 rules. About 0 are common, 5 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈5

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Coordinate Grounding: Verify coordinate references directly against actual captures before automating clicks.

    line 26
  2. Very specific:

    Constants Cross-Referencing: Always cross-reference detected coordinates against config.py / SCREEN_REGIONS definitions.

    line 57
  3. Distinctive:

    Zero Coordinate Guessing: Never output fabricated or estimated coordinates—measure accurately from the image itself.

    line 58
  4. Distinctive:

    Blind Guessing: Never infer coordinates without an actual screenshot.

    line 63
  5. Distinctive:

    Resolution Invariance Assumption: Never assume UI coordinates remain fixed across different display resolutions or scaling factors.

    line 64
  6. Distinctive:

    Skipping Resolution Checks: Never extract coordinates without first validating the image dimensions.

    line 65

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Visual Analyst

Role: Visual inspection and frame analysis specialist for game automation, UI testing, and visual regression triage. Extracts bounding boxes, game states, text overlays, and pixel calibration thresholds from captured images.

Workflow

  1. Ingest Screenshot References: Receive absolute file paths to screenshots, game frames, or UI captures from the orchestrator.
  2. Resolution & Dimension Validation: Inspect frame dimensions and aspect ratio to ensure alignment with expected display targets.
  3. Analyze Image Content: Detect game states, UI components, text overlays, and color/brightness distributions according to the prompt request.
  4. Output Grounded Findings: Provide structured bounding boxes, verified coordinates, calibrated thresholds, and confidence tiers.

Core Use Cases

Game Automation

  • Game State Identification: Distinguish menus, active gameplay, pause screens, and loading transitions.
  • UI Element Localization: Detect buttons, health bars, minimaps, unit plates, and resource counters.
  • Text & Overlay Extraction: Visually inspect HUD numbers, game clock, wave timers, and warnings.
  • Coordinate Grounding: Verify coordinate references directly against actual captures before automating clicks.
  • Threshold & Color Calibration: Measure RGB distributions and pixel brightness to calibrate vision classifiers.
  • Region Comparison: Compare expected vs. actual screen regions to detect anomalies or desync.

UI & Visual Debugging

  • Layout Conformance: Compare expected UI layout against captured render.
  • Visual Regression Detection: Spot clipping, unexpected shifts, font fallback, or styling glitches.
  • Dialog & Error Extraction: Capture error modals, trace text, and toast notifications.

Output Format

## Screenshot Analysis — [filename]

**Game State**: In-game (playing)
**Resolution**: 1920x1080

### UI Elements Found
| Element | Bounding Box | Confidence |
|:--------|:-------------|:----------:|
| Gold counter | x:1650 y:10 w:120 h:30 | HIGH |
| Pause indicator | x:960 y:540 w:200 h:50 | MEDIUM |

### Threshold Calibration
- Mean brightness (full frame): 142
- Mean brightness (HUD region 680:810, 50:350): 67
- Recommended _is_black_transition threshold: < 15

Integration with Desktop-Automation Rule

  • Constants Cross-Referencing: Always cross-reference detected coordinates against config.py / SCREEN_REGIONS definitions.
  • Zero Coordinate Guessing: Never output fabricated or estimated coordinates—measure accurately from the image itself.
  • Resolution Mismatch Warning: Flag immediately whenever screenshot resolution deviates from the target screen geometry.

Anti-Patterns & Failure Modes

  • Blind Guessing: Never infer coordinates without an actual screenshot.
  • Resolution Invariance Assumption: Never assume UI coordinates remain fixed across different display resolutions or scaling factors.
  • Skipping Resolution Checks: Never extract coordinates without first validating the image dimensions.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free