agentleFS
upex-galaxyupex-galaxy/agentic-qa-boilerplate/.agents/skills/bug-screenshot-annotation/SKILL.md

bug-screenshot-annotation

Skill26 starsChanged 9 days agoMITView on GitHub

Turns a raw bug screenshot into a QA-style annotated evidence image — circles/ovals around the broken region, arrows, callout text boxes, a 'BUG — <KEY>' corner badge, and axis tick-marks for date/offset bugs. Use whenever a bug is visual or positional (overlapping elements, misalignment, wrong date/offset on a chart axis, a UI element in the wrong place) and a plain screenshot would need a paragraph to explain what's wrong — the annotated image should make the defect obvious at a glance, the way a QA engineer would mark it up with Snagit or Markup. Triggers on: annotate bug screenshot, mark up evidence, add circles/arrows to screenshot, clarify this bug visually, anota este bug, marca la captura, resalta el bug en la imagen. Runs 100% locally (HTML+CSS overlays rendered via a loopback-only HTTP server and captured with playwright-cli) — do NOT use for photos of physical objects or documents where the fix is inherent to an external image-editing/generative service; this skill only knows how to overlay shapes on a screenshot you already have on disk. Do NOT use for filing the bug itself (that's sprint-testing Execution and Reporting) or for routine before/after screenshots that already read clearly without markup — this skill is for the specific case where a raw screenshot alone doesn't communicate the defect.

The file

We split this skill into the rules it gives an agent

11 rules. About 0 are common, 7 are distinctive, and 4 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈7

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    DO NOT route a QA screenshot through ANY external image service, generative or otherwise — real product/customer data is in the frame.

    line 16
  2. Distinctive:

    DO: work from a screenshot that already exists on disk.

    line 18
  3. Very specific:

    DO: produce exactly ONE evidence file — the final annotated PNG in the ticket's evidence/ folder, named {KEY}-BUG-{BUG-KEY}-annotated.png.

    line 19
  4. Very specific:

    DO: copy the commented overlay blocks from references/shapes.html instead of designing from scratch, and keep both its z-index scale (base …

    line 20
  5. Very specific:

    DO NOT: load the annotation HTML over file:// — the browser-automation CLI refuses it before rendering.

    line 21
  6. Distinctive:

    DO: size the capture viewport equal to or larger than the HTML canvas.

    line 22
  7. Distinctive:

    DO: read the rendered PNG back and expect at least one adjustment pass (move a circle, rewrap callout text, nudge the badge out of a …

    line 23
  8. Distinctive:

    DO: state the final PNG's repo-relative path in chat the moment it lands, unprompted, and repeat it leading the "Bug annotations" group in …

    line 24
  9. Distinctive:

    WHEN embedding the annotated PNG into the bug issue: offer it and let the human confirm first.

    line 25
  10. Distinctive:

    Inspect and iterate: Read the PNG back and check it reads cleanly.

    line 75
  11. Very specific:

    file:// doesn't work: Always serve over loopback HTTP (step 4).

    line 95

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Bug Screenshot Annotation

A workflow skill (metadata.kind: workflow) with no dispatch of its own: it is loaded INSIDE the subagent that is already executing a testing stage, spawns no agents, and is not a stage owner. Typical caller: /sprint-testing Execution, when a bug found during exploration is visual/positional (see ../sprint-testing/references/exploration-patterns.md §"Bugs found during exploration").

Compact Rules

  • DO NOT route a QA screenshot through ANY external image service, generative or otherwise — real product/customer data is in the frame. Explicit user authorization in chat does NOT lift this; everything renders locally over a loopback HTTP server and a local browser capture.
  • WHEN a bug is visual or positional (overlap, misalignment, wrong date/offset on an axis, an element in the wrong place) and a raw screenshot would need a paragraph to explain: annotate it. DO NOT use this skill to file the bug itself, or on before/after shots that already read clearly raw.
  • DO: work from a screenshot that already exists on disk. This skill overlays shapes on an existing image; it never generates or edits an image from a text description.
  • DO: produce exactly ONE evidence file — the final annotated PNG in the ticket's evidence/ folder, named {KEY}-BUG-{BUG-KEY}-annotated.png. The crop and the annotation HTML are scratchpad working files, never written to evidence/ and never cited from a ticket.
  • DO: copy the commented overlay blocks from references/shapes.html instead of designing from scratch, and keep both its z-index scale (base image → shapes → callout boxes → corner badge topmost) and its utf-8 meta tag; a copied block that drops either produces a hidden badge or mojibake.
  • DO NOT: load the annotation HTML over file:// — the browser-automation CLI refuses it before rendering. Serve over loopback HTTP, and kill that server before the session ends.
  • DO: size the capture viewport equal to or larger than the HTML canvas. A smaller viewport clips callouts.
  • DO: read the rendered PNG back and expect at least one adjustment pass (move a circle, rewrap callout text, nudge the badge out of a collision). It is not one-shot.
  • DO: state the final PNG's repo-relative path in chat the moment it lands, unprompted, and repeat it leading the "Bug annotations" group in the session-close screenshot list.
  • WHEN embedding the annotated PNG into the bug issue: offer it and let the human confirm first; once published it leads the bug's Evidence section, ahead of the raw capture.
  • DIRECTION: the evidence page (the evidence-artifact skill) keeps marks as data on one page per ticket and per bug; when a ticket has one, link it and keep this PNG only where a tracker needs an inline image.

Read full SKILL.md when: building the annotation HTML, choosing shape types, or handling a case the local render cannot cover (e.g. a photo of physical signage that would need anonymization).


Why 100% local — the security rationale (binding)

QA screenshots carry real product, customer and competitor data. A screenshot sent to an external image service has already landed in that service's public bucket, and the agent runtime's data-exfiltration classifier blocks such uploads even when the user authorizes them in chat.

So: QA evidence containing real product or customer data never routes through an external image service, generative or otherwise. This skill sidesteps the risk entirely — everything happens with HTML+CSS rendered by an HTTP server bound to 127.0.0.1, captured by a local browser-automation CLI; nothing leaves the machine. If a genuinely unhandleable case appears (e.g. annotating a photo of physical signage), go back to the user and talk through anonymization first — NEVER quietly pipe a real screenshot to an external service.

Scope

  • Input: a raw screenshot that already exists on disk (typically in the ticket's PBI evidence/ folder). This skill overlays shapes; it does not generate or edit images from a text description.
  • Output: exactly ONE file that counts as evidence — the final rendered annotated PNG, in the ticket's evidence/ folder. The crop and the annotation HTML are Bucket C working files (see ../agentic-qa-core/references/evidence-conventions.md §1): session scratchpad only, never referenced from Jira/ATR/bug tickets.
  • Not for: filing the bug (reporting-templates owns that), plain before/after shots that read clearly raw, photos of physical objects/documents.

Process (8 steps + optional embed)

  1. Identify the region. From the raw screenshot (or its accessibility snapshot), work out the pixel crop box and roughly where each annotation shape lands relative to it.
  2. Crop with Python + PIL — scratchpad only.

    from PIL import Image
    img = Image.open("<raw-screenshot-path>")
    img.crop((left, top, right, bottom)).save("<scratchpad>/crop.png")

    The crop is a working intermediate, not evidence — never write it to the ticket's evidence folder.

  3. Build the annotation HTML — scratchpad only. The crop goes in as a background <img>; each annotation is one absolutely-positioned <div> overlay. Do NOT design from scratch — copy/adjust the commented blocks in references/shapes.html (circle-callout, arrow-to-region, callout-box, badge-corner, axis-tick + axis-tag, before-after) and keep its z-index scale.
  4. Serve it over loopback. The browser-automation CLI refuses file:// URLs (errors out before rendering) — a local HTTP server is the only way, and loopback binding is also what keeps this approach exfiltration-safe, not just a workaround:

    python3 -m http.server <port> --bind 127.0.0.1 --directory <scratchpad>
  5. Capture with the browser-automation CLI. Load /playwright-cli first (CLI → skill auto-load rule) for exact verbs/flags. Flow: open http://127.0.0.1:<port>/annotation.html in a named session of its own (-s=<KEY>-annotate; anonymous, case (a) of ../agentic-qa-core/references/browser-sessions.md), close it when the PNG is on disk, resize the viewport to the HTML's real dimensions (equal or larger — a smaller viewport clips callouts), then screenshot with an explicit destination path into the ticket's evidence/ folder, named:

    {KEY}-BUG-{BUG-KEY}-annotated.png

    {KEY} = ticket under test, {BUG-KEY} = the filed bug/defect/improvement issue key (they differ in the normal case). This is the ONE file of the whole process that belongs in evidence/.

  6. Inspect and iterate. Read the PNG back and check it reads cleanly. Not optional and not one-shot — expect at least one adjustment pass: move/resize a circle or callout box, rewrap/shorten callout text, nudge the badge or arrow out of a collision.
  7. Clean up. Kill the HTTP server process (kill <pid> or pkill -f "http.server <port>"). Never leave it running past the session.
  8. Report the path — immediately, unprompted. The moment the final PNG lands in evidence/, state its repo-relative path in chat, in that same turn (standing contract: ../agentic-qa-core/references/session-footer-contract.md Part 1). The path must also reappear in the session-close consolidated screenshot list, leading the "Bug annotations" group.
  9. (Optional — this repo's upgrade over the manual-attach model.) Embed into the bug issue. Jira accepts inline images via the bundled helper — offer to publish the annotated PNG directly as an evidence comment on the bug (human confirms first):

    bun .agents/skills/acli/scripts/jira-attach-media.ts <BUG-KEY> \
      ./evidence/<KEY>-BUG-<BUG-KEY>-annotated.png \
      --caption "Annotated evidence — <one-line defect summary>" --publish

    The annotated image is the "money shot": it leads the bug's Evidence section, ahead of the raw capture. When the bug has an evidence page (the evidence-artifact skill), the image goes into the bug's pack as the reproduction's first defect image, so it leads the page and becomes the bug's one tracker thumbnail; the Stage 3 Evidence Handoff (../sprint-testing/references/reporting-templates.md §3.5) links that page, not image files.

Known gotchas

  • Badge hidden behind a callout box. Both are position: absolute; without explicit stacking, DOM order (not visual intent) decides paint order. references/shapes.html fixes this with an explicit z-index scale — base image 1 → circles/arrows/ticks 10 → callout boxes 20 → corner badge 30 (always topmost). Skipping the scale is the #1 source of annotation bugs; don't let a copy-pasted block quietly drop its z-index.
  • Callout text clipped at the edge. Happens when the resize viewport is smaller than the HTML canvas, or a callout box lacks max-width + white-space: normal. Size the viewport to the real image dimensions or larger — never smaller; keep callout boxes wrapped, not nowrap.
  • file:// doesn't work. Always serve over loopback HTTP (step 4).
  • Mojibake on em-dashes / accented characters. Without <meta charset="utf-8"> in <head>, — and accents render as garbage when captured. references/shapes.html already carries the meta tag — keep it when copying.

Evidence rules

Binding conventions: ../agentic-qa-core/references/evidence-conventions.md — bucket model (§1: intermediates are Bucket C, final PNG is Bucket B), annotated-name suffix (§2), verify-on-disk discipline (§3).

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.