agentleFS
timurgaleevtimurgaleev/vibestack/skills/canary/SKILL.md

canary

Skill8 starsChanged 6 months agoMITView on GitHub

Post-deploy canary monitoring: watch the live app for console errors, slowdowns and page failures.

42 other projects use a version of this, 4 of them edited.

Adopted by

Most recently taken up by

The file

Show its rules and text

We split this skill into the rules it gives an agent

22 rules. 0 are common, 10 are distinctive, and 12 are very specific to this project.

  • 0

    Common

    Most of the 35 edited versions we read keep these.

  • 10

    Distinctive

    This author's own choices. Worth a look.

  • 12

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    If this session was compacted, the copy of this skill in context may be cut off.

    line 17
  2. Very specific:

    Read ${CLAUDE_SKILL_DIR}/SKILL.md from disk and resume at the step you are on.

    line 18
  3. Distinctive:

    Use when: "monitor deploy", "canary", "post-deploy check", "watch production", "verify deploy".

    line 22
  4. Distinctive:

    If the URL contains "github.com" → platform is GitHub

    line 60
  5. Distinctive:

    If the URL contains "gitlab" → platform is GitLab

    line 61
  6. Very specific:

    When the user types /canary, run this skill.

    line 97
  7. Very specific:

    If the user passed --baseline, capture the current state BEFORE deploying.

    line 122
  8. Very specific:

    Write the filtered absolute URLs, one per line, to .vibestack/canary-reports/links-<page-name>.txt with the Write tool, then:

    line 184
  9. Very specific:

    If no --pages were specified, auto-discover pages to monitor:

    line 253
  10. Very specific:

    If no baseline.json exists, take a snapshot now as the monitoring-start reference.

    line 277
  11. Very specific:

    Run the link check on the links output.

    line 294
  12. Very specific:

    Run the link check on the links output and the asset and stream probes.

    line 314
  13. Very specific:

    Cadence: Before the first round, record the start time and the deadline (start plus the duration) with date -u +%s, and note each round's …

    line 317
  14. Very specific:

    After a round, wait max(0, 60 − <round seconds>) seconds in a bounded background wait — foreground sleep is blocked on some hosts, so run …

    line 319
  15. Very specific:

    After each check, compare results against baseline.json (or pre-monitor.json):

    line 326
  16. Distinctive:

    Don't cry wolf: Only alert on patterns that persist across 2 or more consecutive checks.

    line 338
  17. Distinctive:

    After monitoring completes (or if the user stops early), produce a summary:

    line 382
  18. Distinctive:

    If the deploy is healthy, offer to update the baseline:

    line 457
  19. Distinctive:

    If the user chooses A, copy the latest screenshots to the baselines directory and save each page's current text:

    line 464
  20. Distinctive:

    Speed matters: Start monitoring as soon as the pages are chosen.

    line 482
  21. Distinctive:

    Screenshots are evidence: Every alert includes a screenshot path.

    line 484
  22. Distinctive:

    Transient tolerance: Only alert on patterns that persist across 2+ consecutive checks.

    line 485

“Common” is measured: kept by at least 6 in 10 of the 35 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

If this session was compacted, the copy of this skill in context may be cut off. Read ${CLAUDE_SKILL_DIR}/SKILL.md from disk and resume at the step you are on.

When to invoke

Use when: "monitor deploy", "canary", "post-deploy check", "watch production", "verify deploy".

Preamble

eval "$("${VIBESTACK_HOME:-$HOME/.vibestack}/bin/vibe-slug" 2>/dev/null)" 2>/dev/null || SLUG="unknown"
_LEARN_FILE="${VIBESTACK_HOME:-$HOME/.vibestack}/projects/${SLUG:-unknown}/learnings.jsonl"
if [ -f "$_LEARN_FILE" ]; then
  _LEARN_COUNT=$(wc -l < "$_LEARN_FILE" 2>/dev/null | tr -d ' ')
  echo "LEARNINGS: $_LEARN_COUNT entries loaded"
  if [ "$_LEARN_COUNT" -gt 5 ] 2>/dev/null; then
    "${VIBESTACK_HOME:-$HOME/.vibestack}/bin/vibe-learnings-search" --limit 5 2>/dev/null || true
  fi
else
  echo "LEARNINGS: none yet"
fi

{{include lib/snippets/session-host.md}}

{{include lib/snippets/decision-brief.md}}

{{include lib/snippets/working-protocols.md}}

{{include lib/snippets/state-protocols.md}}

SETUP

{{include lib/snippets/browse-detect.md}}

Step 0: Detect platform and base branch

First, detect the git hosting platform from the remote URL:

git remote get-url origin 2>/dev/null
  • If the URL contains "github.com" → platform is GitHub
  • If the URL contains "gitlab" → platform is GitLab
  • Otherwise, check CLI availability:
    • gh auth status 2>/dev/null succeeds → platform is GitHub (covers GitHub Enterprise)
    • glab auth status 2>/dev/null succeeds → platform is GitLab (covers self-hosted)
    • Neither → unknown (use git-native commands only)

Determine which branch this PR/MR targets, or the repo's default branch if no PR/MR exists. Use the result as "the base branch" in all subsequent steps.

If GitHub:

  1. gh pr view --json baseRefName -q .baseRefName — if succeeds, use it
  2. gh repo view --json defaultBranchRef -q .defaultBranchRef.name — if succeeds, use it

If GitLab:

  1. glab mr view -F json 2>/dev/null and extract the target_branch field — if succeeds, use it
  2. glab repo view -F json 2>/dev/null and extract the default_branch field — if succeeds, use it

{{include lib/snippets/base-branch-detect.md}}

/canary reads no diff, so a BASE_CONTAINS_HEAD line (a merged branch still checked out) is not a stop here: take the base from its origin/<name>, say so in one line, and carry on.

Print the detected base branch name. In every subsequent git diff, git log, git fetch, git merge, and PR/MR creation command, substitute the detected branch name wherever the instructions say "the base branch" or <default>.


/canary — Post-Deploy Visual Monitor

You are a Release Reliability Engineer watching production after a deploy. You've seen deploys that pass CI but break in production — a missing environment variable, a CDN cache serving stale assets, a database migration that's slower than expected on real data. Your job is to catch these in the first 10 minutes, not 10 hours.

You use the browse daemon to watch the live app, take screenshots, check console errors, and compare against baselines. You are the safety net between "shipped" and "verified."

User-invocable

When the user types /canary, run this skill.

Arguments

  • /canary <url> — monitor a URL for 10 minutes after deploy
  • /canary <url> --duration 5m — custom monitoring duration (1m to 30m)
  • /canary <url> --baseline — capture baseline screenshots (run BEFORE deploying)
  • /canary <url> --pages /,/dashboard,/settings — specify pages to monitor
  • /canary <url> --quick — single-pass health check (no continuous monitoring)
  • /canary <url> --sse <endpoint-url> — also probe a server-sent events endpoint for its first event

Instructions

Phase 1: Setup

eval "$("${VIBESTACK_HOME:-$HOME/.vibestack}/bin/vibe-slug" 2>/dev/null || echo "SLUG=unknown")"
mkdir -p .vibestack/canary-reports
mkdir -p .vibestack/canary-reports/baselines
mkdir -p .vibestack/canary-reports/screenshots

Parse the user's arguments. Default duration is 10 minutes. Default pages: auto-discover from the app's navigation.

Phase 2: Baseline Capture (--baseline mode)

If the user passed --baseline, capture the current state BEFORE deploying.

The browse console buffer is shared across every page and every check, and only console --clear empties it — navigation does not. Every block below that reads console --errors therefore clears the buffer first, so the errors it reports belong to that one page load. --errors also returns warnings: keep only lines tagged [error] and drop [warning] lines before any comparison. Each line reads [<timestamp>] [error] <message>; store and compare only <message> — the timestamp differs on every load, so keeping it would make every error look new.

For each page (either from --pages or the homepage):

B='<BROWSE_BIN>'
$B console --clear
$B goto <page-url>
$B snapshot -i -a -o ".vibestack/canary-reports/baselines/<page-name>.png"
$B console --errors
$B perf
$B text > ".vibestack/canary-reports/baselines/<page-name>.txt"
$B links

Then run the link check (below) on the links output, and the asset and stream probes (below).

Collect for each page: the screenshot path, the list of [error] message strings (the messages themselves, not a count), the broken links, the saved text snapshot path and the page load time from perf. A baseline that already serves HTML for an asset, or an SSE endpoint that already sends no first event, is reported to the user at capture time — it is a live failure, not a reference to compare against.

Save the baseline manifest to .vibestack/canary-reports/baseline.json:

{
  "url": "<url>",
  "timestamp": "<ISO>",
  "branch": "<current branch>",
  "pages": {
    "/": {
      "screenshot": "baselines/home.png",
      "console_errors": ["Uncaught TypeError: x is undefined"],
      "broken_links": ["https://example.com/old-pricing"],
      "text_snapshot": "baselines/home.txt",
      "load_time_ms": 450
    }
  }
}

Then STOP and tell the user: "Baseline captured. Deploy your changes, then run /canary <url> to monitor."

Link check

Used by Phase 2, Phase 4 and every round of Phase 5. From the $B links output keep same-origin links only, and drop any URL whose path matches logout|signout|delete|remove|cancel|unsubscribe (case-insensitive). Filter BEFORE fetching anything: such links are never visited with goto or HEAD, because the browse session may carry imported cookies and a request to one of them changes real production state.

Write the filtered absolute URLs, one per line, to .vibestack/canary-reports/links-<page-name>.txt with the Write tool, then:

grep -Eiv '^[a-z]+://[^/]+/.*(logout|signout|delete|remove|cancel|unsubscribe)' ".vibestack/canary-reports/links-<page-name>.txt" | while IFS= read -r u; do
  code=$(curl -sI -o /dev/null -w '%{http_code}' "$u" 2>/dev/null) || code=000
  printf '%s %s\n' "$code" "$u"
done

404 or 410 is broken. Any other code outside 2xx/3xx, or a curl failure (000), is unknown — report it, never alert on it.

Asset and stream probes

Used by Phase 2 and every round of Phase 5: the asset probe per page, right after that page's $B goto; the SSE probe once per round. Both are absolute checks — a broken asset or a silent stream is a failure whatever the baseline held — but they still need 2 consecutive checks before an alert is confirmed.

Static-asset content type. A single-page app whose server falls back to index.html answers a missing bundle with an HTML 200, so the status code looks fine while the page is dead. For every <script src> and stylesheet <link> on the loaded page, send a HEAD from the page itself and read the content type:

B='<BROWSE_BIN>'
$B js "Promise.all([...document.querySelectorAll('script[src],link[rel~=stylesheet][href]')].map(e=>{const u=e.src||e.href;const k=e.tagName==='SCRIPT'?'js':'css';return fetch(u,{method:'HEAD'}).then(r=>({url:u,kind:k,status:r.status,type:r.headers.get('content-type')})).catch(()=>({url:u,kind:k,status:0,type:null}))})).then(a=>JSON.stringify(a))"

HEAD reads headers only and changes nothing. A js entry passes when its status is 2xx and its type names javascript or ecmascript; a css entry when its status is 2xx and its type names text/css. The status decides first, whatever the type: a 404, 410 or 5xx is a broken asset — HIGH. A 2xx whose type is text/html is an SPA fallback serving HTML for an asset — HIGH. Status 0 (a cross-origin asset whose server sends no CORS headers, or a network error), a 401 or 403, and any other status or type are unknown: report them, never alert on them.

SSE first event — only when the user named an SSE endpoint with --sse. Never guess one from the page. Write that URL, one line, to .vibestack/canary-reports/sse-url.txt with the Write tool, then:

sse_url=$(head -n1 ".vibestack/canary-reports/sse-url.txt")
hdr=".vibestack/canary-reports/sse-headers.txt"
case "$sse_url" in
  http://*|https://*)
    : > "$hdr"
    if curl -N -sS --max-time 10 -D "$hdr" -- "$sse_url" 2>/dev/null | head -c 65536 | grep -q -m1 -E '^(event|data):'; then
      echo "SSE_FIRST_EVENT: yes"
    else
      status=$(awk 'toupper($(1)) ~ /^HTTP\// {s = $(2)} END {print s}' "$hdr")
      case "$status" in
        3[0-9][0-9]|401|403) echo "SSE_FIRST_EVENT: unknown (HTTP $status)" ;;
        *) echo "SSE_FIRST_EVENT: no" ;;
      esac
    fi ;;
  *) echo "SSE_URL_INVALID: not an http(s) URL" ;;
esac

The connection is bounded at 10 seconds and 64 KB. SSE_FIRST_EVENT: no — the stream did not connect, or sent no event: or data: line inside those bounds — is CRITICAL. SSE_FIRST_EVENT: unknown — a redirect, 401 or 403 — means the stream wants a login the probe cannot send: curl carries none of the browse session's cookies. Report it once, never alert on it, the same as an unknown link. SSE_URL_INVALID is not a probe result: tell the user and skip the SSE probe.

Phase 3: Page Discovery

If no --pages were specified, auto-discover pages to monitor:

B='<BROWSE_BIN>'
$B goto <url>
$B links
$B snapshot -i

From the links output keep same-origin links only and drop any URL whose path matches logout|signout|delete|remove|cancel|unsubscribe (case-insensitive) before building the list — those pages are never proposed and never visited, because an imported session would change real production state. Take the top 5 internal navigation links from the filtered set. Always include the homepage. Present the page list via AskUserQuestion:

  • Context: Monitoring the production site at the given URL after a deploy.
  • Question: Which pages should the canary monitor?
  • RECOMMENDATION: Choose A — these are the main navigation targets.
  • A) Monitor these pages: [list the filtered pages]
  • B) Add more pages (user specifies)
  • C) Monitor homepage only (quick check)

Phase 4: Monitoring-Start Reference (if no baseline exists)

If no baseline.json exists, take a snapshot now as the monitoring-start reference. It is taken after the deploy, not before it: an error the deploy already introduced is in it and will never read as new. The report says so (see Phase 6).

For each page to monitor:

B='<BROWSE_BIN>'
$B console --clear
$B goto <page-url>
$B snapshot -i -a -o ".vibestack/canary-reports/screenshots/pre-<page-name>.png"
$B console --errors
$B perf
$B text > ".vibestack/canary-reports/screenshots/pre-<page-name>.txt"
$B links

Run the link check on the links output. Collect the same fields as the baseline (screenshot, console_errors as the list of [error] messages, broken_links, text_snapshot, load_time_ms) and write them in the same shape to .vibestack/canary-reports/pre-monitor.json. Never write this snapshot to baseline.json. It is the reference for detecting regressions during monitoring.

Phase 5: Continuous Monitoring Loop

Monitor for the specified duration in rounds; each round checks every page:

B='<BROWSE_BIN>'
$B console --clear
$B goto <page-url>
$B snapshot -i -a -o ".vibestack/canary-reports/screenshots/<page-name>-<check-number>.png"
$B console --errors
$B perf
$B links

Run the link check on the links output and the asset and stream probes. Keep only [error] console lines.

Cadence. Before the first round, record the start time and the deadline (start plus the duration) with date -u +%s, and note each round's start and end the same way. After a round, wait max(0, 60 − <round seconds>) seconds in a bounded background wait — foreground sleep is blocked on some hosts, so run the sleep as a background command whose timeout is a few seconds longer, and start the next round when it exits. Rounds never overlap: a round that took 60 seconds or more is followed straight away by the next. Once the deadline has passed, finish the current round and stop; never start a round after it, except the one confirming round a pending CRITICAL or HIGH finding gets. The report gives the cadence actually achieved, not the nominal one.

After each check, compare results against baseline.json (or pre-monitor.json):

  1. Page load failure — goto returns error or timeout → CRITICAL ALERT
  2. New console errors — an [error] message not in the page's console_errors list → HIGH ALERT
  3. Performance regression — load time exceeds 2x baseline → MEDIUM ALERT
  4. Broken links — a 404/410 URL not in the page's broken_links list → LOW ALERT
  5. Broken asset — a script or stylesheet URL answering 404, 410 or 5xx, or 2xx with text/html → HIGH ALERT
  6. SSE stream silent — SSE_FIRST_EVENT: no for an endpoint named with --sse → CRITICAL ALERT

Alert on changes, not absolutes (items 5 and 6 excepted — they are absolute checks). Errors are compared by message, not by count. A page whose baseline holds 3 error messages is fine if it still shows those 3. One NEW message is an alert, even if another one went away.

Don't cry wolf. Only alert on patterns that persist across 2 or more consecutive checks. A single transient network blip is not an alert.

Notify once confirmed. A CRITICAL or HIGH finding reaches the user only when it is confirmed on two consecutive checks. A first occurrence is pending: record its time, page and evidence, say nothing, and run the next check; if that check clears it, it goes under observations. A finding still pending at the deadline gets one more confirming round, so a short run can still alert. Under --quick there is no second check: report each CRITICAL or HIGH finding of the single pass as a finding and set the status from it. On confirmation, notify via AskUserQuestion:

CANARY ALERT
════════════
Time:     [timestamp, e.g., confirmed on check #4 at 240s]
Seen:     [check #3 and check #4]
Page:     [page URL]
Type:     [CRITICAL / HIGH]
Finding:  [what changed — be specific]
Evidence: [screenshot path]
Baseline: [baseline value]
Current:  [current value]
  • Context: Canary monitoring detected an issue on [page] after [duration].
  • RECOMMENDATION: Choose based on severity — A for CRITICAL, B to keep watching a HIGH.
  • A) Investigate now — stop monitoring, focus on this issue
  • B) Continue monitoring — watch whether it clears
  • C) Rollback — revert the deploy immediately
  • D) Dismiss — false positive, continue monitoring

Phase 6: Health Report

An alert is confirmed once it is seen on 2 or more consecutive checks. Derive the status from confirmed alerts only:

  • BROKEN — any CRITICAL alert was confirmed
  • DEGRADED — any other alert was confirmed
  • HEALTHY — no alert was confirmed

A confirmed alert that later clears stays in the report with resolved: true; it still counts toward the status. Findings seen on a single check never confirm — they go under observations. Under --quick the single pass is the whole run: its findings count as confirmed for the status (a CRITICAL makes it BROKEN), and the report says they were seen once.

After monitoring completes (or if the user stops early), produce a summary:

CANARY REPORT — [url]
═════════════════════
Duration:     [X minutes]
Pages:        [N pages monitored]
Checks:       [N total checks performed]
Cadence:      [N rounds, mean interval between round starts Xs]
Reference:    [baseline.json from <timestamp> / monitoring-start reference]
Status:       [HEALTHY / DEGRADED / BROKEN]

Per-Page Results:
─────────────────────────────────────────────────────
  Page            Status      Errors    Avg Load
  /               HEALTHY     0         450ms
  /dashboard      DEGRADED    2 new     1200ms (was 400ms)
  /settings       HEALTHY     0         380ms

Alerts Fired:  [N] (X critical, Y high, Z medium)
Screenshots:   .vibestack/canary-reports/screenshots/

VERDICT: [DEPLOY IS HEALTHY / DEPLOY HAS ISSUES — details above]

With the monitoring-start reference, add one line under Reference:: "Errors already present when monitoring started cannot be detected without --baseline."

Save report to .vibestack/canary-reports/{date}-canary.md and .vibestack/canary-reports/{date}-canary.json. The JSON report has this shape:

{
  "skill": "canary",
  "timestamp": "<ISO>",
  "url": "<url>",
  "duration_min": 10,
  "status": "DEGRADED",
  "pages": {
    "/dashboard": {
      "checks": 10,
      "new_errors": ["Uncaught TypeError: x is undefined"],
      "new_404s": ["https://example.com/old-pricing"],
      "load_avg_ms": 1200,
      "load_baseline_ms": 400
    }
  },
  "alerts": [
    {
      "type": "new_console_error",
      "page": "/dashboard",
      "severity": "HIGH",
      "first_seen": "<ISO>",
      "confirmed_at": "<ISO>",
      "resolved": false,
      "evidence": "screenshots/dashboard-3.png"
    }
  ],
  "observations": ["transient findings that never confirmed"]
}

Log the result for the review dashboard by appending one line to the project's canary history. The line holds model-authored summary fields only — skill, timestamp, status, url, duration_min, alerts (the confirmed alert count) — never page text. The file is appended to, never overwritten:

eval "$("${VIBESTACK_HOME:-$HOME/.vibestack}/bin/vibe-slug" 2>/dev/null || echo SLUG=unknown)"
_H="${VIBESTACK_HOME:-$HOME/.vibestack}/projects/$SLUG"
mkdir -p "$_H"
printf '%s\n' '{"skill":"canary","timestamp":"<ISO>","status":"<HEALTHY/DEGRADED/BROKEN>","url":"<url>","duration_min":<N>,"alerts":<N>}' >> "$_H/canary-history.jsonl"

Phase 7: Baseline Update

If the deploy is healthy, offer to update the baseline:

  • Context: Canary monitoring completed. The deploy is healthy.
  • RECOMMENDATION: Choose A — deploy is healthy, new baseline reflects current production.
  • A) Update baseline with current screenshots
  • B) Keep old baseline

If the user chooses A, copy the latest screenshots to the baselines directory and save each page's current text:

B='<BROWSE_BIN>'
$B goto <page-url>
$B text > ".vibestack/canary-reports/baselines/<page-name>.txt"

Then rewrite baseline.json with the full field set per page: screenshot, console_errors (the current [error] messages), broken_links, text_snapshot and load_time_ms.

{{include lib/snippets/capture-learnings.md}} Canary runs learn things nothing else sees — which page is slow only under real traffic, which console error the CDN produces after every deploy, which route needs a warm-up request before it answers. Log those, so the next canary starts with the last one's map instead of a blank baseline.

Important Rules

  • Speed matters. Start monitoring as soon as the pages are chosen. Don't over-analyze before monitoring.
  • Alert on changes, not absolutes. Compare against baseline, not industry standards.
  • Screenshots are evidence. Every alert includes a screenshot path. No exceptions.
  • Transient tolerance. Only alert on patterns that persist across 2+ consecutive checks.
  • Baseline is king. Without a baseline, canary is a health check. Encourage --baseline before deploying.
  • Performance thresholds are relative. 2x baseline is a regression. 1.5x might be normal variance.
  • Read-only. Observe and report. Don't modify code unless the user explicitly asks to investigate and fix.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.