skill-health
Skill789 starsChanged 24 days agoMITView on GitHub
Fleet skill observability with two views - health audits per-skill metrics and files/resolves issues in memory/issues/; analytics ranks the fleet by 7d runs, success rates, and anomaly flags.
- Reads credentials
8 other projects use a version of this, 2 of them edited.
Adopted by
Most recently taken up by
- shimo4228/akc-cycletheir own version, first seen Nov 8, 2026
- aaronjmars/miroshark-aeonedited, first seen Oct 21, 2026
- MiroShark/miroshark-aeonedited, first seen Oct 17, 2026
- shimo4228/claude-harnesstheir own version, first seen Oct 10, 2026
- shimo4228/skill-healththeir own version, first seen Oct 10, 2026
- ayutaz/piper-plustheir own version, first seen Oct 8, 2026
Other versions8 projects use a version of this.
Word for word · 1
- aaronjmars/aeon★ 789100% same
Edited · 2
- aaronjmars/miroshark-aeon★ 20100% same
- MiroShark/miroshark-aeon★ 20100% same
Wrote their own version · 5
- ayutaz/piper-plus★ 240same name
- shimo4228/claude-harness★ 4same name
- shimo4228/akc-cycle★ 2same name
- shimo4228/skill-health★ 1same name
- TheKREATORS/my-claude-skills★ 1same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository92 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- llms.txtllms.txt
- aeonSkill
- action-converterSkill
- aeon-doctorSkill
- aeon-updateSkill
- anatomySkill
- arc-studioSkill
- articleSkill
- auto-mergeSkill
- autoresearchSkill
- auto-workflowSkill
- base-mcpSkill
- bd-radarSkill
- canister-auditSkill
- changelogSkill
- competitor-monitorSkill
- compute-resellSkill
- cortx-reliabilitySkill
- create-proveSkill
- create-skillSkill
- defi-overviewSkill
- deploy-prototypeSkill
- deploy-uni-hookSkill
- digestSkill
- distribute-tokensSkill
- executor-mcpSkill
- featureSkill
- feedback-builderSkill
- fetch-tweetsSkill
- finance-district-mcpSkill
- fleet-controlSkill
- floviaSkill
- fork-fleetSkill
- github-filing-registrySkill
- github-monitorSkill
- github-trendingSkill
- glim-mcpSkill
- heartbeatSkill
- higgsfieldSkill
- hunter-22Skill
- idea-forgeSkill
- idea-pipelineSkill
- inbox-triageSkill
- install-skillSkill
- investigation-reportSkill
- last30Skill
- memory-flushSkill
- mention-radarSkill
- miroshark-matchdaySkill
- monitor-polymarketSkill
- narrative-convergenceSkill
- narrative-trackerSkill
- onchain-monitorSkill
- operator-scorecardSkill
- pack-submitSkill
- picks-trackerSkill
- pm-manipulationSkill
- posthog-errorsSkill
- The first 60 shown.
The file
Show its rules and text
We split this skill into the rules it gives an agent
27 rules. 0 are common, 6 are distinctive, and 21 are very specific to this project.
0
Common
Most of the 6 edited versions we read keep these.
6
Distinctive
This author's own choices. Worth a look.
21
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 29
Read memory/MEMORY.md for high-level context and scan the last ~3 days of memory/logs/ for recent activity — drop anything already reported …
- Very specific: line 62
Run ./scripts/skill-runs --hours 168 --failures --json 2>/dev/null || echo '{}' → extract any skill with failures in the last 7d that isn't …
- Distinctive: line 89
Do not duplicate the same error across per-skill rows — reference the systemic line.
- Very specific: line 107
If gh is unavailable or the lookup errors, leave the issue untouched and log it - never resolve on an unverified assumption.
- Very specific: line 111
Remove the skill from that issue's affected_skills.
- Very specific: line 117
Write memory/issues/ISS-NNN.md with YAML frontmatter:
- Distinctive: line 143
All issue writes must be atomic per file — never partial updates mid-run.
- Very specific: line 149
If current_hash == prev_report.hash AND now - prev_report.last_notified_at < 24h → do not notify.
- Very specific: line 188
Always end with Open issues: X · Resolved this run: Y.
- Very specific: line 189
If NO CRITICAL/DEGRADED/FLAPPING and no new/resolved issues → body is just HEALTH: OK — N skills healthy.
- Very specific: line 193
If the gate in step 5 said notify → ./notify "<report body>".
- Very specific: line 194
If gate said skip → do not call ./notify.
- Very specific: line 212
If all skills healthy, the body-only shortcut from step 6 still fires (once per 24h, per gate) so the operator gets confirmation the audit …
- Very specific: line 216
Never file two open issues for the same (skill, root_cause) pair — always check INDEX.md first.
- Very specific: line 217
Never edit a Resolved issue.
- Distinctive: line 218
Do not notify on pure HEALTHY runs more than once per 24h.
- Distinctive: line 219
If in single-skill mode (selector was a skill slug), skip INDEX.md updates only if the single skill is HEALTHY — otherwise file/resolve as …
- Very specific: line 220
Never touch memory/issues/INDEX.md Resolved section except to move rows into it.
- Distinctive: line 246
If the script fails (auth, rate limit, network error) or the JSON is empty:
- Very specific: line 261
Read aeon.yml and build SCHEDULED_SKILLS: dict {skill_name -> {enabled: bool, schedule: str}} for every entry under skills:.
- Very specific: line 332
If gate says skip, still write the article and JSON spec, and log SKILL_ANALYTICS_QUIET (no anomalies).
- Very specific: line 459
If anomaly_count >= 1, prepend an Alert block before the verdict:
- Distinctive: line 465
If the file write fails (filesystem read-only, missing directory), log a warning but do not abort — the article is the canonical artifact, …
- Very specific: line 495
Keep the message body tight for signal.
- Very specific: line 529
Never invent runs: If skill-runs returns empty, exit SKILL_ANALYTICS_NO_DATA — do not synthesise data from cron-state alone (cron-state's …
- Very specific: line 541
Health view: if ./scripts/skill-runs fails, log SKILL_HEALTH_PARTIAL — skill-runs unavailable and continue with cron-state only.
- Very specific: line 542
Analytics view: if gh api is rate-limited or the runner's network is degraded, ./scripts/skill-runs exits non-zero.
“Common” is measured: kept by at least 6 in 10 of the 6 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
${var} — View selector.
- empty → health check across all scheduled skills (default).
- a skill slug (e.g.
token-movers) → health check for that one skill.analyticsormetrics(optionallyanalytics:HOURS, e.g.metrics:72) → fleet metrics view over the last HOURS (default 168 = 7d, cap 720).- a bare integer (e.g.
168) → metrics view with that window in hours (legacy shorthand).
<!-- autoresearch: variation C — more robust: memory/issues integration per CLAUDE.md health-skill contract, state-change-gated notifications, graceful missing-data; folds in B's TL;DR+action-directives+top-5 and A's skill-runs fallback. Analytics view absorbed from skill-analytics: ranked fleet view, exit-taxonomy distribution, significance-gated notify + article + dashboard JSON. -->Overview
This skill provides two views over the same GitHub-Actions skill-run data. They share a preamble but branch into distinct logic:
- health (default): per-skill classification, issue filing/resolution against
memory/issues/, and a state-change-gated notification. This is the load-bearing self-healing view — its issue contract,memory/skill-health/scoring, and### skill-healthlog shape are depended on by the health loop and other skills. Do not weaken it. - analytics (metrics): a fleet-wide ranked view — top runners, failure rates, exit-taxonomy distribution, silent-scheduled detection, and anomaly flags — with a significance-gated notification plus an article and a dashboard JSON spec.
heartbeatgives binary ok/not-ok per run and the health view audits skills one degradation-band at a time; the analytics view is the only place the operator sees the entire fleet ranked side-by-side.
Shared preamble (run for either view)
- Read
memory/MEMORY.mdfor high-level context and scan the last ~3 days ofmemory/logs/for recent activity — drop anything already reported so you don't re-report the same signal. - Compute
${today}(UTC date,YYYY-MM-DD). - Parse
${var}→ selector (trim whitespace first):- empty →
VIEW=health,TARGET=all(all scheduled skills). - lowercase first token is
analyticsormetrics→VIEW=analytics. Parse an optional window argument after a:or a space (analytics:72,metrics 336): if it is a positive integer,WINDOW_HOURS = min(that, 720); otherwiseWINDOW_HOURS = 168. - a bare positive integer (e.g.
168) →VIEW=analytics,WINDOW_HOURS = min(that, 720)(legacy skill-analytics shorthand). - anything else (a non-keyword, non-integer slug) →
VIEW=health,TARGET=<that slug>(single-skill health check).
- empty →
- Dispatch: if
VIEW=health, run Health view; ifVIEW=analytics, run Analytics view.
Health view
VIEW=health. Audit skill quality metrics, detect API degradation, file issues for new failures and resolve them when skills recover, and notify only when fleet health state actually changes. If TARGET is a single skill slug, only check that skill.
Data sources
memory/cron-state.json— Per-skill quality metrics (as before).memory/skill-health/*.json— Per-skill quality analysis (Haiku post-run).memory/skill-health/last-report.json— Last run's classification snapshot (this skill writes it). Used to dedup notifications and detect flapping.aeon.yml— Enabled skills and schedules.memory/issues/INDEX.mdandmemory/issues/ISS-*.md— Open issues tracker. Check before filing, update on recovery../scripts/skill-runs --hours 168 --failures --json— Fallback source for failures that never wrote to cron-state (runs that crashed before writing, etc.). Run once, parse JSON.memory/logs/YYYY-MM-DD.md(last 3 days) — Grep forSKILL_*_ERRORorEMPTYsignatures keyed to skills missing from skill-health/*.json.
Steps
1. Gather state
- Parse
aeon.yml→ list of enabled skills with schedules. IfTARGETis a single skill, filter to just that skill. - Load
memory/cron-state.json(if missing or unparseable, treat as empty — first run, not failure). - Load every
memory/skill-health/*.json(exceptlast-report.json). - Load
memory/skill-health/last-report.jsonif present →prev_report. If missing,prev_report = {}. - Run
./scripts/skill-runs --hours 168 --failures --json 2>/dev/null || echo '{}'→ extract any skill with failures in the last 7d that isn't in cron-state (runs that failed before writing state). - Parse
memory/issues/INDEX.md→ extract open issues withdetected_by: skill-healthand their affected skills. If missing, treat as empty.
2. Classify each enabled skill
For each enabled skill, assign one status using the first matching rule:
| Status | Trigger |
| CRITICAL | consecutive_failures >= 3 OR (status==failed AND days_since_last_success >= 3) |
| DEGRADED | success_rate < 0.6 OR (latest skill-health/*.json avg_score < 2.5 over ≥3 runs) |
| FLAPPING | 3+ status transitions (success↔failed) in last 7 days per cron-state history or skill-runs output |
| WARNING | success_rate < 0.8 OR consecutive_failures >= 1 |
| HEALTHY | success_rate >= 0.8 AND consecutive_failures == 0 AND (no skill-health data OR avg_score >= 3) |
| NO DATA | no entry in cron-state AND never seen in skill-runs |
Compute severity score for sorting: consecutive_failures × (1 + days_since_last_success/7). Ties broken by days_since_last_success desc.
For each CRITICAL/DEGRADED/FLAPPING skill, record:
last_error(from cron-state or nearest log signature)api_hostif the error clearly names one (e.g.api.coingecko.com,api.github.com)suggested_action— one of:FIX CONFIG(missing secret, bad arg),WAIT-API(rate limit, 5xx, timeout on third-party host),INVESTIGATE(unrecognised error),DISPATCH-SKILL(NO DATA but scheduled — scheduler gap)
3. Detect systemic patterns
Group non-HEALTHY skills by shared api_host OR shared last_error signature. If ≥2 skills share one:
- Emit a single
SYSTEMIC:callout (e.g.SYSTEMIC: 3 skills failing on api.coingecko.com (rate_limit)). - Do not duplicate the same error across per-skill rows — reference the systemic line.
4. Reconcile with memory/issues/
Precondition guard: only perform issue filing/resolution if memory/issues/INDEX.md already exists. If it is missing, the operator has not opted into the issue-tracker contract yet — log SKILL_HEALTH_ISSUE_TRACKER_MISSING to memory/logs/${today}.md, skip this entire step (and the reconciliation side of step 5), and continue with classification + notification only. Do not auto-create INDEX.md.
For each CRITICAL or FLAPPING skill, check if an issue with status: open or status: fix-pending already has this skill in affected_skills AND a matching root_cause signature:
- Matching issue exists, same root cause → do nothing (no new file, no notification for this skill).
- Matching issue exists, different root cause → append a note to the existing ISS file's body:
Update YYYY-MM-DD: new signature: <error>. Do not file a new issue. - No matching issue → file a new one (see below).
Reconcile fix-pending issues first. skill-repair sets status: fix-pending with a fix_pr when it opens a repair PR - it cannot know whether that PR merges, so it deliberately does not claim resolved. This step closes that loop. For each issue with status: fix-pending and a non-null fix_pr, check the PR's real state (gh pr view <fix_pr> --json state,mergedAt) and reconcile:
- Merged (
mergedAtnon-null) → the fix shipped. Setstatus: resolved,resolved_at: <mergedAt>, move the row from Open to Resolved in INDEX.md. - Closed without merging (
state: CLOSED,mergedAt: null) → the fix did not ship. Setstatus: open, setfix_pr: null, and appendUpdate <YYYY-MM-DD>: fix PR <url> was closed unmerged; issue reopened.to the body. The skill is still broken; leaving itfix-pendingwould hide that. - Still open → leave untouched. The repair is in flight.
If gh is unavailable or the lookup errors, leave the issue untouched and log it - never resolve on an unverified assumption.
For each skill now HEALTHY whose name appears in any status: open issue's affected_skills:
- Skip
status: fix-pendingissues. Those wait for the reconcile step above; a lucky HEALTHY classification must not close a repair that has not merged. - Remove the skill from that issue's
affected_skills. If the list becomes empty, setstatus: resolved, setresolved_at: <now ISO>, and move the row from Open to Resolved in INDEX.md.
For each skill in a status: open critical issue that is still DEGRADED or WARNING only because of historical metrics, pipe its cron-state JSON object to python3 scripts/skill_health_recovery.py '<detected_at>'. If it prints recovered, remove the skill from affected_skills and resolve an empty issue exactly as above. A successful run after detection proves that specific failure incident recovered even when lifetime success_rate remains low. Do not apply this shortcut to FLAPPING/high issues (one successful run does not prove flapping stopped) or to status: fix-pending issues (those wait for the reconcile step above). Invalid or missing state/timestamps print active and fail closed.
Filing a new issue:
- Find next ID: scan
memory/issues/ISS-*.md, take maxNNN, add 1. Format as zero-padded 3 digits (ISS-042). - Write
memory/issues/ISS-NNN.mdwith YAML frontmatter:--- id: ISS-NNN title: <skill> <concise failure> status: open # open | fix-pending | resolved. fix-pending = repair PR open, unmerged. Health resolves fix-pending only after merge (or reopens if closed unmerged). HEALTHY recovery still resolves status: open issues with no pending PR. severity: critical | high | medium | low # critical=CRITICAL status, high=FLAPPING, medium=DEGRADED category: rate-limit | timeout | missing-secret | config | api-change | sandbox-limitation | unknown detected_by: skill-health detected_at: <ISO-8601 UTC, e.g. 2026-09-03T12:00:00Z> # must carry Z or a +00:00 offset; a naive stamp fails closed to active affected_skills: [<skill>, ...] # may grow later root_cause: <error signature, 1 line> fix_pr: null --- ## What happened <2-3 line summary> ## Signal - consecutive_failures: N - days_since_last_success: N - last_error: "<error>" - related skills: <list or "none"> - Append a row to
memory/issues/INDEX.mdunder Open:| ISS-NNN | title | severity | category | YYYY-MM-DD | skill-a, skill-b |.
All issue writes must be atomic per file — never partial updates mid-run.
5. Decide whether to notify
Build a stable signature from the current classification: sorted list of CRITICAL+FLAPPING+DEGRADED skill names + SYSTEMIC callouts. SHA-256 it → current_hash.
- If
current_hash == prev_report.hashANDnow - prev_report.last_notified_at < 24h→ do not notify. State unchanged. - Otherwise → notify (there's new signal or the daily reminder cadence elapsed).
Always write memory/skill-health/last-report.json:
{
"hash": "<current_hash>",
"last_notified_at": "<ISO if notified this run, else previous value>",
"last_run_at": "<ISO now>",
"classification": { "critical": [...], "degraded": [...], "flapping": [...], "warning": [...], "healthy_count": N, "no_data": [...] }
}6. Format the report
Top line: HEALTH: OK | HEALTH: WARNING(W) | HEALTH: DEGRADED(D) | HEALTH: CRITICAL(C) — most severe wins.
Body (notify-channel format, max 1 message):
*Skill Health - ${today}*
HEALTH: CRITICAL(2) [systemic: api.coingecko.com rate_limit - 3 skills]
🔴 CRITICAL
- token-movers - 5 fails, 3d down - WAIT-API (rate_limit) → ISS-042
- defi-monitor - 4 fails, 2d down - WAIT-API (rate_limit) → ISS-042
🟡 DEGRADED / FLAPPING
- digest - 52% success (14d), avg quality 2.1 - INVESTIGATE → ISS-043
⚪ NO DATA (2): skill-x, skill-y - DISPATCH-SKILL
🟢 HEALTHY: 34
Open issues: 2 · Resolved this run: 1 (rss-digest)Rules for formatting:
- Cap per-section rows at 5; collapse the rest as
+N more — see memory/issues/INDEX.md. - Omit HEALTHY list (count only). Omit any empty section.
- Always end with
Open issues: X · Resolved this run: Y. - If NO CRITICAL/DEGRADED/FLAPPING and no new/resolved issues → body is just
HEALTH: OK — N skills healthy.
7. Notify and log
- If the gate in step 5 said notify →
./notify "<report body>". Updatelast_notified_atin last-report.json to now. - If gate said skip → do not call
./notify. Log to memory/logs/${today}.md:### skill-health - view: health - SKILL_HEALTH_NOOP - state unchanged since <prev_run_at>, hash=<short>
On notify, log to memory/logs/${today}.md:
### skill-health
- view: health
- HEALTH: <OK|WARNING|DEGRADED|CRITICAL>
- filed: [ISS-NNN, ...]
- resolved: [ISS-NNN, ...]
- open: N
- systemic: <pattern or none>If all skills healthy, the body-only shortcut from step 6 still fires (once per 24h, per gate) so the operator gets confirmation the audit actually ran — but suppress if last-report.json shows a notify <24h ago with the same OK hash.
Health-view constraints
- Never file two open issues for the same
(skill, root_cause)pair — always check INDEX.md first. - Never edit a Resolved issue. If a previously-resolved issue re-fires, file a new ISS with a pointer (
related: ISS-NNN) in the body. - Do not notify on pure HEALTHY runs more than once per 24h.
- If in single-skill mode (selector was a skill slug), skip INDEX.md updates only if the single skill is HEALTHY — otherwise file/resolve as normal.
- Never touch
memory/issues/INDEX.mdResolved section except to move rows into it; never delete rows.
Analytics view
VIEW=analytics. Generate a fleet-level performance view of every Aeon skill that has run in the window. The point of this view is to answer four questions in one report: which skills run most, which fail most, which are silently skipping (new exit taxonomy from the autoresearch-evolution rewrites), and which scheduled skills haven't fired at all.
Why this exists
heartbeat runs daily and emits a per-skill ✓/✗. The health view (above) files issues for skills that breach degradation thresholds. Neither produces a ranked, fleet-wide view. The 80 autoresearch-evolution rewrites (aeon PRs #46–#136) introduced new exit taxonomies — SKIP_UNCHANGED, NEW_INFO, SKIP_QUIET — that classify quiet-but-correct runs separately from failures. Existing health checks treat any non-*_OK exit as worth attention; the analytics view makes the actual distribution visible so a skill running mostly SKIP_UNCHANGED reads as healthy-quiet, not silently broken.
Steps
1. Determine the window
WINDOW_HOURSwas set by the selector parse (default 168 = 7 days; a positive integer fromanalytics:N,metrics N, or a bare integer; capped at 720 = 30 days — anything longer slows thegh apipaginate).- Compute
WINDOW_LABEL(e.g."last 7d"or"last 72h").
2. Pull the run snapshot
With WINDOW_HOURS written in literally. A shell > redirect into a file is refused in this run, so the JSON reaches its file through a piped one-line node writer:
./scripts/skill-runs --json --hours <WINDOW_HOURS> 2>/dev/null | node -e 'require("fs").writeFileSync("output/.chains/skill-analytics-runs.json", require("fs").readFileSync(0))'If the script fails (auth, rate limit, network error) or the JSON is empty:
- Log
SKILL_ANALYTICS_NO_DATA — skill-runs returned empty (gh api / network error?)tomemory/logs/${today}.md(under the### skill-healthheading, see step 13) and stop with no notification. A silent fleet view is correct on data-fetch failure — fall back rather than guess.
The script's JSON shape (see scripts/skill-runs):
{
"period": {"since": "...", "until": "...", "hours": 168},
"summary": {"total": N, "succeeded": N, "failed": N, "cancelled": N, "in_progress": N},
"skills": [{"skill": "name", "total": N, "success": N, "failure": N, "cancelled": N, "in_progress": N, "last_run": "...", "last_conclusion": "..."}],
"anomalies": {"duplicates": [...], "failing": [...]}
}3. Cross-reference with cron schedule
Read aeon.yml and build SCHEDULED_SKILLS: dict {skill_name -> {enabled: bool, schedule: str}} for every entry under skills:. Treat schedule: "workflow_dispatch" and schedule: "reactive" as exempt from the "no runs in window" anomaly — those are dispatched on demand, not by cron.
For every skill in SCHEDULED_SKILLS where enabled: true AND schedule is a valid cron expression AND the skill is not present in the snapshot's skills array, mark silent_scheduled: true (zero runs in window despite an active schedule).
4. Cross-reference with cron-state.json
Load memory/cron-state.json if present (missing → empty dict, not failure). For each skill in the snapshot, attach:
consecutive_failures(0 if missing)last_status("unknown"if missing)
Used to compute the consecutive-failure anomaly without a second gh api round-trip.
5. Mine exit taxonomy from logs
For each daily log file memory/logs/YYYY-MM-DD.md whose date falls in the window, scan for these markers (one match per skill section):
_OK→ success (excluding_OK_SILENT)_OK_SILENT/_QUIET/SKIP_QUIET→ quiet-successSKIP_UNCHANGED→ skip-unchanged (autoresearch-evolution exit)NEW_INFO→ new-info (autoresearch-evolution exit)_SKIP*(other) → skip-other_ERROR/_FAILED→ error_PARTIAL→ partial- (no match) → uncategorized
Build EXIT_DIST[skill] = {ok: N, quiet: N, skip_unchanged: N, new_info: N, skip_other: N, error: N, partial: N, uncategorized: N}. The dominant bucket per skill is the one with the largest count; ties broken in the order listed above. If a skill has no log markers in the window, dominant bucket is "uncategorized".
This step is best-effort — the markers are regex-grepped from human-written logs, not parsed from a contract. A miss-rate of 10–20% is expected and acceptable; the GitHub Actions success/failure counts from step 2 remain the ground truth for pass/fail. The taxonomy distribution is a secondary signal.
6. Anomaly classification
For each skill in the snapshot OR silent_scheduled, assign at most one anomaly flag, first match wins:
| Flag | Trigger |
🔴 SILENT | silent_scheduled: true (enabled cron skill, zero runs in window) |
🔴 ALL_FAIL | total >= 2 AND failure == total |
🟠 CONSECUTIVE_FAILURES | consecutive_failures >= 3 (from cron-state) |
🟠 LOW_SUCCESS | total >= 3 AND success / total < 0.80 |
🟡 ALL_SKIP | total >= 3 AND EXIT_DIST.ok + EXIT_DIST.quiet + EXIT_DIST.new_info == 0 AND EXIT_DIST.skip_unchanged + EXIT_DIST.skip_other > 0 (every run skipped — possibly correct, possibly stuck) |
🟡 DUPLICATE_RUNS | total > 2 × expected_runs(schedule, window) (more runs than the cron should produce — manual reruns or scheduler glitch) |
expected_runs(schedule, window) is a coarse estimate — for a cron "0 H * * *" over 7 days, expect 7; for "0 H,H,H * * *", expect 21; for weekly "0 H * * D", expect 1. If the schedule string is unparseable, skip the duplicate check for that skill (do not flag false positives).
A skill with no flag is considered HEALTHY for analytics purposes.
7. Compute summary
total_runs: sum of every skill's total
distinct_skills: count of skills with total >= 1
overall_success_pct: snapshot.summary.succeeded / (succeeded + failed) × 100 (cancelled + in_progress excluded)
anomaly_count: count of skills with any flag in step 6
silent_scheduled_count: count of SILENT flags
exit_dominant: top 3 dominant exit buckets across the fleet, e.g. "ok (42), skip_unchanged (18), error (3)"8. Build the verdict line
Pick the strongest single claim, in priority:
- Any
🔴 SILENTexists →"${N} scheduled skill(s) didn't run this window — ${first_skill}" - Any
🔴 ALL_FAILexists →"${first_skill} failed every run (${N}/${N}) — investigate" - Any
🟠 CONSECUTIVE_FAILURESexists →"${first_skill} on ${N}-run failure streak" - Any
🟠 LOW_SUCCESSexists →"${first_skill} ${pct}% success over ${total} runs — degraded" - Any
🟡 ALL_SKIPexists →"${N} skill(s) only emitting skip-class exits this window — verify intent" - Otherwise →
"All ${distinct_skills} active skills healthy — ${overall_success_pct}% success across ${total_runs} runs"
9. Significance gate
Notify only if anomaly_count >= 1. Silent run = correct (no anomalies in fleet) = no notification. Following the autoresearch-evolution / fork-digest pattern: noisy skills break trust faster than missing pings.
If gate says skip, still write the article and JSON spec, and log SKILL_ANALYTICS_QUIET (no anomalies). The dashboard widget refreshes regardless; only the push notification is gated.
10. Write the article
Path: output/articles/skill-analytics-${today}.md. Overwrite if it exists (idempotent same-day reruns).
# Skill Analytics - ${today}
**Verdict:** ${verdict_line}
*Window: ${WINDOW_LABEL} · ${total_runs} runs across ${distinct_skills} skills · ${overall_success_pct}% success · ${anomaly_count} anomalies*
## Anomalies
| Flag | Skill | Detail | Action |
|------|-------|--------|--------|
| 🔴 SILENT | name | scheduled `<cron>` but zero runs in window | run **aeon-doctor** - zero-runs-despite-schedule is usually a static config bug (unquoted `schedule:`, dup key) it can't be seen in run data |
| 🔴 ALL_FAIL | name | N/N failed | investigate root cause |
| 🟠 CONSECUTIVE_FAILURES | name | N-run streak (last_error: "...") | see health view for filed issue |
| 🟠 LOW_SUCCESS | name | N% over M runs | review failures |
| 🟡 ALL_SKIP | name | M runs, all skip-class | confirm SKIP_UNCHANGED is the intent |
| 🟡 DUPLICATE_RUNS | name | M runs, expected ~K | check for manual reruns |
(If `anomaly_count == 0`: write `No anomalies - fleet healthy across ${distinct_skills} skills.`)
## Top runners (by run count)
| # | Skill | Runs | Success | Last status | Dominant exit |
|---|-------|------|---------|-------------|---------------|
| 1 | name | N | XX% | success | ok |
| 2 | name | N | XX% | success | skip_unchanged |
...
(Top 15 by total runs desc. If fewer than 15 active skills, list all.)
## Failure rate (sorted, ≥1 failure)
| Skill | Runs | Failures | Success rate | Last conclusion |
|-------|------|----------|--------------|-----------------|
(All skills with `failure >= 1`, sorted by `failure / total` desc. If none: "Zero failures across ${distinct_skills} skills this window.")
## Exit taxonomy distribution
| Bucket | Count | % | Top skills |
|--------|-------|---|------------|
| ok | N | XX% | a, b, c |
| skip_unchanged | N | XX% | d, e |
| new_info | N | XX% | f |
| quiet | N | XX% | g |
| error | N | XX% | h |
| partial | N | XX% | |
| uncategorized | N | XX% | |
(Sourced from `memory/logs/*.md` - best-effort regex grep, see Step 5. Cell-aligns to summary cells above where available.)
## Silent scheduled skills (enabled, zero runs)
${list of {skill, schedule} pairs OR "none - every enabled cron skill ran at least once."}
> **Root-cause pointer:** an enabled cron skill with *zero* runs is most often a **static config** fault the run data can't reveal - an unquoted `schedule:` the scheduler regex silently skips, a duplicate `aeon.yml` key, or an entry pointing at a missing `SKILL.md`. That class is `aeon-doctor`'s job; dispatch it (`./aeon skills run aeon-doctor`) to pinpoint the cause instead of guessing at the scheduler.
## Source status
- skill-runs JSON: ${ok|empty|fetch_error}
- Window: ${WINDOW_HOURS}h (${period.since} → ${period.until})
- aeon.yml: ${ok|missing}
- cron-state.json: ${ok|missing - first run for this fork?}
- Daily logs scanned: ${N_LOG_FILES}/${expected_log_files} for exit taxonomy
---
*The analytics view of `skill-health` (per-skill issue filing lives in the health view) and a companion to `heartbeat` (per-run pulse). Fleet-wide observability is the gap this view closes. Methodology: GitHub Actions run history is ground truth for pass/fail; daily-log markers are best-effort secondary signal for exit taxonomy.*11. Write the dashboard JSON spec
Path: apps/dashboard/outputs/skill-analytics.json. Use the catalog components (Card / Stack / Heading / Text / Badge / Table).
{
"version": "1",
"generated_at": "${ISO timestamp}",
"skill": "skill-analytics",
"title": "Skill Analytics — ${today}",
"spec": {
"type": "Stack",
"props": {"direction": "vertical", "gap": "md"},
"children": [
{"type": "Heading", "props": {"level": 2, "children": "Skill Analytics — ${today}"}},
{"type": "Text", "props": {"variant": "muted", "children": "${verdict_line}"}},
{"type": "Grid", "props": {"columns": 4, "gap": "sm"}, "children": [
{"type": "Card", "props": {"children": [
{"type": "Text", "props": {"variant": "muted", "children": "Total runs"}},
{"type": "Heading", "props": {"level": 3, "children": "${total_runs}"}}
]}},
{"type": "Card", "props": {"children": [
{"type": "Text", "props": {"variant": "muted", "children": "Active skills"}},
{"type": "Heading", "props": {"level": 3, "children": "${distinct_skills}"}}
]}},
{"type": "Card", "props": {"children": [
{"type": "Text", "props": {"variant": "muted", "children": "Success rate"}},
{"type": "Heading", "props": {"level": 3, "children": "${overall_success_pct}%"}}
]}},
{"type": "Card", "props": {"children": [
{"type": "Text", "props": {"variant": "muted", "children": "Anomalies"}},
{"type": "Heading", "props": {"level": 3, "children": "${anomaly_count}"}}
]}}
]},
{"type": "Heading", "props": {"level": 3, "children": "Top runners"}},
{"type": "Table", "props": {
"columns": [
{"key": "rank", "header": "#"},
{"key": "skill", "header": "Skill"},
{"key": "runs", "header": "Runs"},
{"key": "success", "header": "Success"},
{"key": "exit", "header": "Dominant exit"}
],
"rows": [
{"rank": "1", "skill": "name", "runs": "N", "success": "XX%", "exit": "ok"}
]
}}
]
}
}If anomaly_count >= 1, prepend an Alert block before the verdict:
{"type": "Alert", "props": {"variant": "destructive", "children": "${anomaly_count} anomaly flag(s) raised — see Anomalies section"}}If the file write fails (filesystem read-only, missing directory), log a warning but do not abort — the article is the canonical artifact, the JSON spec is a dashboard convenience.
12. Send notification (only if gate from step 9 passed)
Via ./notify:
*Skill Analytics - ${today}*
${verdict_line}
Window: ${WINDOW_LABEL} · ${total_runs} runs · ${distinct_skills} skills · ${overall_success_pct}% success
Anomalies: ${anomaly_count}
${If 🔴 flags (cap top 3):}
🔴 Critical:
- ${skill} - ${flag}: ${detail}
${If 🟠 flags (cap top 3):}
🟠 Degraded:
- ${skill} - ${flag}: ${detail}
${If 🟡 flags (top 3, only if no 🔴/🟠 already filled the slots):}
🟡 Watch:
- ${skill} - ${flag}: ${detail}
Top by runs: ${top_3_skills_by_run_count_with_counts}
Full: output/articles/skill-analytics-${today}.mdKeep the message body tight for signal. Drop the "Top by runs" line first if it runs long; flags are higher signal. (./notify auto-chunks, so length is about signal, not transport.)
13. Log to memory/logs/${today}.md
Log under the shared ### skill-health heading (the health loop parses this shape), with a view: analytics discriminator:
### skill-health
- view: analytics
- **Window**: ${WINDOW_LABEL} (${WINDOW_HOURS}h)
- **Total runs**: ${total_runs} across ${distinct_skills} skills
- **Overall success rate**: ${overall_success_pct}%
- **Anomalies**: ${anomaly_count} (🔴 ${red_count}, 🟠 ${orange_count}, 🟡 ${yellow_count})
- **Silent scheduled**: ${silent_scheduled_count} skills (${comma list capped at 5})
- **Top runner**: ${top_skill} (${top_runs} runs)
- **Exit dominant**: ${exit_dominant_summary}
- **Verdict**: ${verdict_line}
- **Article**: output/articles/skill-analytics-${today}.md
- **Dashboard**: apps/dashboard/outputs/skill-analytics.json
- **Notification sent**: ${yes|no - quiet (no anomalies)}
- **Status**: SKILL_ANALYTICS_OK | SKILL_ANALYTICS_QUIET | SKILL_ANALYTICS_NO_DATAAnalytics-view exit taxonomy
| Status | Meaning | Notify? |
SKILL_ANALYTICS_OK | snapshot fetched, ≥1 anomaly flagged | Yes |
SKILL_ANALYTICS_QUIET | snapshot fetched, zero anomalies | No (article + JSON written, log only) |
SKILL_ANALYTICS_NO_DATA | skill-runs returned empty / fetch failed | No (log only, no article overwrite) |
Analytics-view constraints
- Significance-gated. A clean fleet must produce zero notifications. Article and JSON spec still write so the dashboard reflects the latest state, but
./notifyis silent. - Never invent runs. If
skill-runsreturns empty, exitSKILL_ANALYTICS_NO_DATA— do not synthesise data from cron-state alone (cron-state's view is per-skill, not chronologically ordered, and would produce a misleading "top runners" table). - Best-effort exit-taxonomy parsing. Log markers are human-written; expect a 10–20% miss rate. Do not block the article on parse failures — drop the affected skill into
uncategorizedand continue. - Idempotent. Same-day reruns overwrite the article and JSON spec. The log entry is appended (one block per run, lets the operator see analytic drift across reruns).
- No issue filing in this view. The analytics view does not write to
memory/issues/— that contract belongs to the health view. Anomalies surface here as flags; persistence and resolution live in the health view's domain. - Respect workflow_dispatch / reactive. Skills with non-cron schedules cannot be SILENT — they fire only on demand. Excluding them from the silent-scheduled check prevents permanent false positives.
Network note (both views)
This skill fetches no URLs directly — all data is local or via gh / ./scripts/skill-runs (which uses gh api, so auth comes from GITHUB_TOKEN with no secret ever on the command line). No curl fallback needed.
- Health view: if
./scripts/skill-runsfails, logSKILL_HEALTH_PARTIAL — skill-runs unavailableand continue with cron-state only. - Analytics view: if
gh apiis rate-limited or the runner's network is degraded,./scripts/skill-runsexits non-zero; catch that and fall through toSKILL_ANALYTICS_NO_DATArather than emitting a partial fleet view that would mislead.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

