agentleFS
flopperjflopperj/dotfiles-core/.claude/skills/agent-stats/SKILL.md

agent-stats

Skill3 starsChanged 9 days agoMITView on GitHub

Aggregate metrics.jsonl from all projects into a system-level summary of agent pipeline health: first-pass success rate, classification distribution, swarm outcomes, and health flags. Use when the user types /agent-stats, asks 'how is the agent pipeline doing', 'show me agent metrics', 'is Cyrus first-passing?', or wants to evaluate the system rather than a single ticket.

The file

We split this skill into the rules it gives an agent

9 rules. About 0 are common, 5 are distinctive, and 4 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈5

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Not a per-ticket view (use /swarm-retro for run-level analysis).

    line 26
  2. Very specific:

    Not a real-time monitor — it reads the on-disk metrics.jsonl, which is updated by emitters at pipeline completion.

    line 27
  3. Distinctive:

    Not a writer — it never modifies metrics, never promotes lessons, never opens PRs.

    line 28
  4. Very specific:

    If the user names a project that does not match a directory under ~/.claude/tasks/, run anyway — the script will report "no events in …

    line 44
  5. Distinctive:

    After the raw output, add a short interpretation only when there is something the numbers don't say outright:

    line 60
  6. Distinctive:

    If the numbers speak for themselves, do not add interpretation.

    line 66
  7. Distinctive:

    Only if a flag is actionable, suggest exactly one next command:

    line 70
  8. Distinctive:

    Never suggest more than one.

    line 78
  9. Very specific:

    All five are read by scripts/agent-stats.sh.

    line 99

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Agent Stats — System-Level Pipeline Health

Render a summary of agent pipeline health from ~/.claude/tasks/<project>/metrics.jsonl files. The aggregation is read-only — no metrics are modified, no events are emitted.

This skill answers questions a staff/principal engineer asks about the system, not about a single run:

  • What is Cyrus's first-pass success rate?
  • How often does Optimus's plan need a CI-fix retry?
  • What does the classification distribution look like — am I escalating too often?
  • Are swarm runs converging or stalling?

When to Use

  • The user types /agent-stats.
  • The user asks "how is the agent pipeline doing?", "show me agent metrics", "is Cyrus first-passing?", "what's our first-pass rate?", or any system-level question that single-ticket views cannot answer.
  • A briefing or scheduled job wants to surface a weekly health snapshot.

What It Is Not

  • Not a per-ticket view (use /swarm-retro for run-level analysis).
  • Not a real-time monitor — it reads the on-disk metrics.jsonl, which is updated by emitters at pipeline completion.
  • Not a writer — it never modifies metrics, never promotes lessons, never opens PRs.

Workflow

Step 1: Resolve scope

Parse the user's invocation for filters:

User saysArgs
/agent-stats (no args)All projects, last 30 days
/agent-stats --project <name>One project, last 30 days
/agent-stats --days 7All projects, last 7 days
"agent stats for the last week"All projects, --days 7
"how is dotfiles doing"--project dotfiles, default 30 days

If the user names a project that does not match a directory under ~/.claude/tasks/, run anyway — the script will report "no events in window" cleanly.

Step 2: Run the aggregator

Invoke the script:

bash "$(readlink ~/.claude/skills/agent-stats)/../../../scripts/agent-stats.sh" [--project <name>] [--days <N>]

The script lives at scripts/agent-stats.sh in the dotfiles-core repo, which is not itself symlinked into ~/.claude/ — resolving through this skill's own symlink locates it regardless of where the repo is checked out.

The script's output is plain text designed to be copy-pasted as-is. Do not reformat it. Do not summarize it. Display the raw output.

Step 3: Add interpretation, only if signal warrants it

After the raw output, add a short interpretation only when there is something the numbers don't say outright:

  • If the script raised health flags: name the most likely cause based on what the user has been working on recently. Example: "First-pass rate dropped — your last 3 swarm runs were on the auth-rewrite branch, which is the most architecturally novel work this month."
  • If sample size is small: say it once, then stop. Don't editorialize on noisy data.
  • If first-pass rate is high (≥85%) and stable: a one-liner acknowledging it. Healthy systems deserve a sentence too.

If the numbers speak for themselves, do not add interpretation. The script's output is sufficient.

Step 4: Suggest one next step (optional)

Only if a flag is actionable, suggest exactly one next command:

FlagSuggested next step
First-pass rate <60% on Medium tickets"Type: Retro, analyze my last swarm" — find which classifications are misrouting
CI fix attempts > pipeline count"Type: Cyrus, audit the last 3 failed CI runs" — surface the recurring failure
No events in window"Type: Pickup PROJ-NNNN" — start a pipeline to populate metrics

Never suggest more than one. The user is reading a dashboard, not running a triage.

Failure Modes

ScenarioBehavior
jq not installedScript exits 0 with a one-line warning. Surface the warning verbatim.
~/.claude/tasks/ does not existScript prints onboarding text. Surface verbatim.
Metrics file exists but is emptyTreated as "no events in window". Surface verbatim.
Schema drift (events missing required fields)jq filters skip them silently. The dashboard shows fewer events than were on disk; this is acceptable for a dashboard, not for the schema validator (which is emit-metric.sh's job, not this skill's).

Schema Reference

This skill consumes events defined in ~/.claude/evals/schemas/metrics-event.schema.json. The five event types it reads:

  • pipeline_complete — emitted by cyrus-tdd-engineer and ticket-pickup. Contributes to first-pass and CI-fix metrics. Events with outcome: advisory (a correct no-code answer) are counted on their own line and excluded from the first-pass denominator.
  • ticket_classified — emitted by ticket-pickup after the classification gate. Contributes to classification distribution.
  • swarm_complete — emitted by ticket-swarm at run end. Contributes to swarm outcomes.
  • multi_repo_split — emitted by ticket-pickup on multi-repo detection. Contributes to the Multi-Repo Splits section: counts split by mode and gate_choice, plus the notice-vs-gate ratio — the detection false-positive denominator.
  • review_round_posted — emitted by review-respond once per posted or updated review-round summary. Contributes to the Review Rounds section: the total and the split by action (post vs update).

All five are read by scripts/agent-stats.sh. If a new event type beyond these is added to the schema, this skill will silently ignore it until the script is updated. That is intentional — the dashboard prefers stability over completeness.

Responsibility Boundaries

<!-- BEGIN RESPONSIBILITY BOUNDARIES --><!-- EXTRA_ROWS: agent-stats -->
AgentSole responsibilityNEVER does
Agent StatsAggregate metrics.jsonl files into a read-only summary; surface health flagsModify metrics, emit events, promote lessons, open PRs, run pipelines
AristotleStrategic analysis — assumptions, first principles, highest-leverage directionName file paths, produce code, plan execution, review PRs
OptimusExecution planning — file paths, step sequencing, risk assessment, architectureWrite code, make file edits, run commands, re-litigate upstream strategic decisions, review PRs
CyrusTDD implementation — write tests first, then code, hit 80%+ coverageRedesign architecture, question strategic direction, skip tests, review PRs
RangerStaff-level PR review — analyze diffs, score issues, report findingsWrite code, make edits, implement fixes, plan architecture, perform strategic analysis
ScoutPR review — analyze diffs, score issues, report findingsWrite code, make edits, implement fixes, plan architecture, perform strategic analysis
AuditorCode complexity analysis, reviewer routingWrite code, review code, post to GitHub, implement fixes
<!-- END RESPONSIBILITY BOUNDARIES -->

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.