agentleFS
sergeyklaysergeyklay/.agents/.agents/skills/audit-agent/SKILL.md

audit-agent

Skill6 starsChanged 50 days agoApache-2.0View on GitHub

Audits AI-agent runs and session snapshots for token usage, cache traffic, cost, duration, turns, tool calls, failures, retries, subagent attribution, duplicated reads, and artifact yield. Use when reporting or comparing a Codex CLI rollout, GitHub Copilot CLI, Gemini CLI, OpenCode, Claude Code, a runner export, or transcript logs. Do NOT use to monitor live CI, prove a check can fail, or sandbox a CLI.

1 other project uses a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

7 rules. About 0 are common, 5 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈5

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Keep the audit read-only. Never modify, vacuum, delete, migrate, or import into the runner's session store.

    line 17
  2. Distinctive:

    Treat transcripts and tool inputs as sensitive.

    line 18
  3. Distinctive:

    Do not load every adapter.

    line 37
  4. Very specific:

    Use the runner's recorded cost when present.

    line 55
  5. Very specific:

    Call time_updated - time_created a session span, not active runtime.

    line 57
  6. Distinctive:

    Use this order and omit sections the request does not need:

    line 67
  7. Distinctive:

    Every headline number names its source and formula in the same section.

    line 76

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Audit an Agent Run

Reconstruct measurements from persisted records, not from the run's own summary. A native counter is evidence only after its scope, update semantics, and run boundary are known.

Evidence contract

  • Keep the audit read-only. Never modify, vacuum, delete, migrate, or import into the runner's session store.
  • Treat transcripts and tool inputs as sensitive. Emit structural counts and identifiers by default; sanitize exports before they leave the machine.
  • Classify the result as exact, estimate, or snapshot. Exact requires a closed boundary and a successful second-path reconciliation. A live or resumable session is a snapshot through an explicit cutoff.
  • Preserve vendor categories. Report input, cache read, cache write, output, reasoning, and recorded cost separately; combine counters only when the schema proves they do not overlap.
  • Distinguish user turns, assistant messages, model steps, and tool calls. Never label one of them simply turns.

Script paths are relative to this skill directory, not the project working directory. If a command cannot resolve scripts/, prefix it with the skill directory reported by the platform.

Route by evidence source

Identify the runner and installed version before reading records, then load only the matching adapter:

  • OpenCode with opencode db: read references/opencode.md. This is the preferred and tested route.
  • Codex CLI local rollout JSONL or a saved codex exec --json stream: read references/codex.md.
  • GitHub Copilot CLI: read references/copilot-cli.md.
  • Claude Code JSONL: read references/claude-code.md.
  • Gemini CLI JSONL: read references/gemini.md.
  • Another runner, an export with unknown schema, or raw logs: read references/generic-logs.md.
  • Comparisons, duplicated reads, retries, or artifact yield: also read references/analysis.md.

Do not load every adapter. Their schemas are mutually exclusive, and carrying one vendor's reduction into another vendor's records is the primary failure this skill prevents.

Workflow

1. Pin the run boundary

Record the root run or session id, project or working directory, runner version, model/provider, agent configuration, start, end or cutoff, and terminal state (completed, aborted, or snapshot). Verify a discovered id against at least the directory and time window; nearest timestamp alone is not identity.

For a comparison, pin the task input and starting repository state before measuring either run. If they differ, report two case studies rather than a performance comparison.

2. Inventory the whole run

Enumerate the root and every descendant session before arithmetic. Rebuild parent-child links from explicit ids where available. Missing child evidence means attribution is incomplete, not that the child did not run. Deduplicate by the storage system's documented primary key or by (source, id) when scope is unknown.

3. Derive each metric from its update semantics

For every counter, determine whether each record is a delta, a cumulative snapshot, a repeated message total, or a session total. Sum deltas, take the terminal cumulative value, and count repeated totals once per proven key. There is no cross-vendor default reduction.

Use the runner's recorded cost when present. If cost must be derived from pricing, fetch current provider rates, name the model and rate date, and label the result derived.

Call time_updated - time_created a session span, not active runtime. Report active model or tool duration only when explicit start/end records support it.

4. Reconcile through a second path

Compare session totals with the sum of terminal step records, a native stats command, or another independently stored summary. State both values and the difference. A mismatch, missing descendants, unknown counter role, or absent terminal evidence downgrades the affected figure to estimate.

For every reported zero, run the extractor against a known positive record or fixture first. Without that control, say no hits observed, not none occurred.

5. Report without overclaiming

Use this order and omit sections the request does not need:

  1. Status: exact, estimate, or snapshot, with cutoff and one-sentence reason.
  2. Scope: root id, descendants included, runner/version, project, model/configuration, source queried.
  3. Usage: separate token categories and recorded or derived cost, total and per agent/session.
  4. Behavior: user turns, model steps, tool calls and errors; clearly label heuristic retry or strategy-change counts.
  5. Cross-checks: detail-versus-summary reconciliation and positive controls.
  6. Limitations: missing records, ambiguous fields, concurrent writers, or non-comparable inputs.

Every headline number names its source and formula in the same section. Do not convert missing usage data from artifact bytes, transcript length, or context-window size.

Completion gate

  • Run identity, boundary, state, and cutoff are explicit.
  • Root and descendant sessions are included exactly once.
  • Counter roles and update semantics were verified on this runner version.
  • Detail totals reconcile through a second path, or the figure is labeled estimate.
  • Token categories remain separate unless non-overlap is proven.
  • Turn and duration labels state exactly what was counted.
  • Every reported zero passed a positive control.
  • Comparison inputs and starting state match, or no comparative verdict is made.
  • The report contains no prompt, tool payload, credential, or unnecessary transcript content.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.