agentleFS
hollandkevinthollandkevint/data-product-operator/skills/metrics-definition/SKILL.md

metrics-definition

Skill5 starsChanged 2 months agoMITView on GitHub

Precise metric definitions for data products. Outcome metric trees, naming conventions, grain specification, and the "what does this number mean?" problem. Use when defining KPIs, writing metric specifications, resolving conflicting metric definitions, building a metrics catalog, or when someone asks "how should we measure success?" or "why don't these numbers match?"

4 other projects use a version of this.

Adopted by

Most recently taken up by

The file

Show its rules and text

We split this skill into the rules it gives an agent

13 rules. About 0 are common, 12 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈12

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Every metric must specify:

    line 19
  2. Distinctive:

    Never define a metric without specifying grain and time window.

    line 31
  3. Distinctive:

    Every metric at a lower level should causally influence the level above.

    line 44
  4. Distinctive:

    Use consistent patterns so metrics are discoverable:

    line 59
  5. Very specific:

    AVOID abbreviations unless they are universally understood in your domain.

    line 68
  6. Distinctive:

    If two reports show different values for the "same" metric:

    line 72
  7. Distinctive:

    Check the grain (are they aggregating at different levels?)

    line 73
  8. Distinctive:

    Check the time window (calendar month vs trailing 30 days?)

    line 74
  9. Distinctive:

    Check the filters (are they including/excluding the same populations?)

    line 75
  10. Distinctive:

    Check the source (are they querying different tables?)

    line 76
  11. Distinctive:

    When charts represent your metrics:

    line 82
  12. Distinctive:

    Use colorblind-safe palettes.

    line 84
  13. Distinctive:

    Never use dual Y-axes. They let you imply any correlation by adjusting the scale.

    line 86

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

The Core Problem

"47 dashboards and no answers." The failure mode is building metrics without agreeing on what they mean. "Revenue" means three different things to three different teams. "Active users" has no agreed-upon time window. Fix the definitions before building the dashboards.

Metric Definition Template

Every metric must specify:

  • Name: Human-readable, follows naming convention (see below)
  • Business definition: One sentence a non-technical person understands
  • SQL logic: Exact calculation including numerator, denominator, and filters
  • Grain: What level is this calculated at? (daily, weekly, per-user, per-account)
  • Time window: What period does this cover? (trailing 7 days, calendar month, since signup)
  • Dimensional attributes: What can you slice this by? (region, product line, customer segment)
  • Owner: Who maintains this definition?
  • Update cadence: How often does this refresh?
  • Known edge cases: What situations produce unexpected results?

Never define a metric without specifying grain and time window. "Monthly active users" means nothing until you define what "active" means and whether "monthly" is calendar month or trailing 30 days.

Outcome Metric Trees

Connect metrics from business outcomes down to leading indicators:

Business outcome: Reduce hospital readmissions 10%
  Product outcome: Clinical decisions made 3x faster
    Feature outcome: Risk scores updated in real-time
      Leading indicator: Query latency under 1 second

Every metric at a lower level should causally influence the level above. If you can't draw the causal link, the metric doesn't belong in the tree.

Trust Metrics

Alongside performance metrics, track trust:

  • Data accuracy rate: Percentage of values matching gold standard (target: 99.9%)
  • Query response time: P95 latency for consumer queries (target: <3 seconds)
  • Data freshness: Time between source update and availability (target: within SLA)
  • Support response time: How fast you resolve data questions (target: <24 hours)
  • Incident count: Zero algorithmic bias incidents, zero data breaches

Trust metrics matter because a single bad number in a board deck costs more credibility than a delayed feature.

Naming Conventions

Use consistent patterns so metrics are discoverable:

  • count_<thing> for counts (count_active_patients, count_claims_submitted)
  • rate_<thing> for ratios (rate_readmission_30day, rate_claim_denial)
  • avg_<thing> for averages (avg_length_of_stay_days, avg_time_to_resolution)
  • pct_<thing> for percentages (pct_records_complete, pct_goals_met)
  • sum_<thing> for totals (sum_charges, sum_paid_amount)

Include the unit in the name when ambiguous: avg_length_of_stay_days not avg_los.

AVOID abbreviations unless they are universally understood in your domain. mrr is fine for SaaS. los is fine for healthcare. ttfv is not fine for anyone.

When Numbers Don't Match

If two reports show different values for the "same" metric:

  1. Check the grain (are they aggregating at different levels?)
  2. Check the time window (calendar month vs trailing 30 days?)
  3. Check the filters (are they including/excluding the same populations?)
  4. Check the source (are they querying different tables?)

The fix is always a single source of truth with an explicit definition, not reconciling two conflicting reports.

Metric Visualization

When charts represent your metrics:

  • Use colorblind-safe palettes. 8% of men have color vision deficiency. Your VP of Sales might be one of them.
  • Label data directly on the chart. Legends force the reader to look away from the data.
  • Never use dual Y-axes. They let you imply any correlation by adjusting the scale.

For presentation structure, chart selection, and narrative framing, see data-storytelling.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.