agentleFS
aliraza2025aliraza2025/receipts/skills/metric-receipts/SKILL.md

metric-receipts

Skill3 starsChanged 5 days agoMITView on GitHub

Pairs every product metric with the decision it informs and states what the number does not prove. Use when defining success metrics, writing dashboards or metric sections in PRDs, reporting results, preparing a metrics slide, or when the user asks "what should we measure" or "is this metric good".

The file

We split this skill into the rules it gives an agent

4 rules. About 0 are common, 3 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈3

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Set thresholds before looking: If the user has results already, label any threshold chosen after seeing them as POST-HOC.

    line 19
  2. Distinctive:

    Never upgrade a metric's layer in prose.

    line 56
  3. Distinctive:

    Every percentage gets its denominator and date in the same sentence.

    line 57
  4. Distinctive:

    If no metric reaches layer 3, say so plainly in the summary.

    line 58

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Metric receipts

A metric earns its place when it changes a decision. This skill makes each metric carry three things: what it measures, which decision it informs, and what it cannot tell you.

Workflow

  1. Start from decisions, not data. List the decisions the team will make in the next cycle (ship, roll back, fix ingestion, change pricing, hire). If a metric maps to no decision, mark it VANITY and move it to an appendix.
  2. Separate the layers. Most AI and workflow products need at least three different questions answered, and one number rarely answers more than one:
    • Did it run? (completion, latency, error rate)
    • Did it have what it needed? (coverage, evidence, data quality)
    • Was it right or useful? (accuracy against an independent judgment, task success, retention)
  3. Write a receipt card for each metric (template below).
  4. Check for proxy inflation. Flag any place where a layer 1 or 2 number is used to claim a layer 3 result.
  5. Set thresholds before looking. If the user has results already, label any threshold chosen after seeing them as POST-HOC.

Receipt card template

### [Metric name]
- **Definition:** numerator / denominator, unit, time window, who is counted once
- **Layer:** ran | had what it needed | right or useful
- **Decision it informs:** [specific decision and owner]
- **Threshold:** [value], chosen [before | after] seeing data
- **Does not prove:** [the tempting conclusion this number cannot support]
- **Source of truth:** [table, event, log, or report]

Example

From an AI video-credibility product, March 2026:

### Job completion rate
- Definition: completed jobs / submitted links, one batch of 780 YouTube links
- Layer: ran
- Decision it informs: whether the queue and workers are reliable enough to scale the batch size
- Threshold: none set for this batch
- Result: 780 of 780 completed (2 after retry)
- Does not prove: that results were accurate or useful. In the same batch, 759 of 780 were flagged low evidence.

### Average evidence tokens per video
- Definition: mean extracted evidence tokens per video, 19-link Instagram benchmark
- Layer: had what it needed
- Decision it informs: whether the ingestion redesign was worth keeping
- Result: 1.0 before, 130.9 after the redesign
- Does not prove: that credibility judgments became more accurate. That needs labeled evaluation.

Rules

  • Never upgrade a metric's layer in prose. "More evidence recovered" must not become "more accurate".
  • Every percentage gets its denominator and date in the same sentence.
  • If no metric reaches layer 3, say so plainly in the summary.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.