agentleFS
jeremyinthebayjeremyinthebay/relay-skills/skills/alert-routing/SKILL.md

alert-routing

Skill1 starsChanged 4 months agoMITView on GitHub

Stop your automation from paging a human for routine events. Route every alert through one front door so agents triage first and humans are only woken for real emergencies. Use when an autonomous system texts/emails/pages a person too often, when alerts get ignored because they're noisy, when a human is manually relaying alarms back to an agent, or when the same alert policy is duplicated across several scripts.

The file

We split this skill into the rules it gives an agent

5 rules. About 0 are common, 4 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈4

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Default is normal. Paging is opt-in, and the bar is: *would they want to be woken for this?*

    line 25
  2. Distinctive:

    If that returns anything but your router, you have not shipped the policy.

    line 44
  3. Distinctive:

    Every caller collapses to one line:

    line 81
  4. Distinctive:

    Exactly one page: If the routine one paged too, a caller is still bypassing the front door.

    line 99
  5. Distinctive:

    If the second one logs, your dedup key still carries a volatile substring and the throttle is a no-op.

    line 108

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Alert routing: the human is the last responder, not the first

The problem this solves

Your automation detects a problem and texts a human. The human reads it, and then has to tell an agent to go look — even though an agent runs every five minutes and could have seen it first.

The human has become the message bus for a system built to avoid exactly that.

Meanwhile, routine states ("a PR is open and waiting", "retrying a task") page them too — so the alerts get muted, and then the one that mattered is missed.

The rule

SeverityGoes toWakes a human?
urgentlog + chat + agent queue + pageYes — prod is down or money is burning
normallog + chat + agent queueNo

Default is normal. Paging is opt-in, and the bar is: would they want to be woken for this?

An agent triages the queue on its next cycle, fixes what's safe, and escalates with a diagnosis rather than a raw alarm. A person is contacted only if the agent can't resolve it.

The mistake almost everyone makes

Do not add severity to each script's alert() function. We did. We changed one of them, declared the policy shipped, and the very next routine event paged the human anyway — because four other scripts each had their own copy of alert() calling the pager directly.

A policy implemented in five places is not a policy. It's a coincidence waiting to end.

Collapse them into one front door every script must call, then prove no other path survives:

grep -rn "notify.sh" *.sh   # who can page a human? Must be exactly ONE file.

If that returns anything but your router, you have not shipped the policy. Make the wrong thing structurally impossible, not merely discouraged.

Reference implementation

alert.sh — the only script permitted to call the pager:

#!/bin/zsh
SEV="${1:-normal}"; shift
MSG="$*"; [ -n "$MSG" ] || exit 0

# Dedupe on the alert CLASS, not the exact bytes. "Fires every 60s = muted" is only true if the
# throttle actually catches it — and a live "12m" age or an HTTP code baked into the message changes
# the hash every tick and defeats the throttle entirely. We shipped exactly this: a watchdog put the
# minute-count in the text and fired 78 identical "stuck PR" alerts in one hour, evicting the one
# alert that mattered. Strip volatile digit runs BEFORE hashing so "open 12m" and "open 13m" collapse.
KEY=$(printf '%s' "$MSG" | sed -E 's/[0-9]+/N/g' | shasum | cut -c1-12)
STAMP="$STATE/$KEY"
if [ -f "$STAMP" ] && [ $(( $(date +%s) - $(stat -f %m "$STAMP") )) -lt 3600 ]; then exit 0; fi
date +%s > "$STAMP"

printf '[%s] [%s] %s\n' "$(date '+%F %T')" "$SEV" "$MSG" >> "$BASE/alerts.log"   # agents read this
printf '%s\t%s\n' "$SEV" "$MSG" >> "$BASE/.chat-queue"                           # agent posts it
printf '%s\n'     "$MSG"        >> "$BASE/.pending-alerts"                       # agent triages it

# A queue nothing drains is a silent leak. If an agent is not GUARANTEED to drain these every cycle,
# bound them (ring-buffer to the last N) and surface their depth where a human sees it. An undrained
# backlog must never be invisible — "silence must not look like health" applies to your own plumbing.
for f in "$BASE/.chat-queue" "$BASE/.pending-alerts"; do
  [ -f "$f" ] && [ "$(wc -l < "$f")" -gt 500 ] && { tail -n 500 "$f" > "$f.tmp" && mv "$f.tmp" "$f"; }
done

[ "$SEV" = "urgent" ] && { printf '%s\n' "$MSG" >> "$BASE/outbox.txt"; "$BASE/notify.sh"; }
exit 0

Every caller collapses to one line:

alert(){ local sev=normal; [ "$1" = urgent ] && { sev=urgent; shift; }; "$BASE/alert.sh" $sev "$*"; }

Shell can't call your chat API? Don't try. Queue to a file and let the agent — which does have chat tools — drain it on its next run. Queue, don't block.

Test it for real

Stub the pager so it records instead of sending, fire both severities, assert the count:

./alert.sh normal "a PR is open and waiting"     # expect: NO page
./alert.sh urgent "production is serving 500s"   # expect: page

Exactly one page. If the routine one paged too, a caller is still bypassing the front door.

Then fire the same class twice with only a number changed — assert it logs once:

./alert.sh normal "PR #7 open 12m, no build"     # logs
./alert.sh normal "PR #7 open 13m, no build"     # must NOT log again — same class

If the second one logs, your dedup key still carries a volatile substring and the throttle is a no-op. A test that fires a class only once can never catch this — vary the number and watch. (This is the bug that flooded us; the test that would have caught it didn't exist until after.)

The last mile: don't chatter either

Once alerting is trustworthy, delete the periodic "everything's fine" heartbeat. Ours ran every 10 minutes until the owner killed it: "I trust the process is working, we have a watcher, and you'll alert me when I need to do work."

A status update nobody asked for is a tax on attention. A watchdog is what makes silence trustworthy. Chattering to prove you're alive is just noise with good intentions.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.