agentleFS
hidenobunagaihidenobunagai/astack/skills/principle-respect-the-grain/SKILL.md

principle-respect-the-grain

Skill1 starsChanged 14 days agoMITView on GitHub

Apply before you join, filter, group, or aggregate a table, and before you trust a total. Name the grain (what one row means) of every table, and check key uniqueness and row counts before and after each step so fan-out and silent drops do not change the number.

The file

We split this skill into the rules it gives an agent

3 rules. About 0 are common, 3 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈3

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Every table has a grain: what one row represents, such as one order, one customer per day, or one page view per session.

    line 8
  2. Distinctive:

    Write the grain next to the load, for example "one row per order_id", and assert it with a uniqueness check on the key.

    line 12
  3. Distinctive:

    After each filter or aggregation, compare the count with the count the definition predicts.

    line 16

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Respect the Grain

Every table has a grain: what one row represents, such as one order, one customer per day, or one page view per session. Name the grain before you use the table. Most wrong numbers in analysis come from ignoring it.

A join that fans out duplicates rows and inflates sums. An inner join that silently drops unmatched keys shrinks the base. A group-by at the wrong level averages averages. Each of these still returns a clean-looking table.

  • Write the grain next to the load, for example "one row per order_id", and assert it with a uniqueness check on the key.
  • Before and after each join, log row counts and key uniqueness on the side that must be unique. A left join from orders to a customer table with duplicate customer_id rows multiplies every order. Pass validate="m:1" (or "1:1") to join on the lookup side. The default is validate="m:m", which checks nothing. The streaming engine does not run the check, so for streaming runs add an explicit assertion on key uniqueness.
  • Count null keys on each side before every join. A Polars join never matches null keys by default (nulls_equal=False, renamed from join_nulls in 1.24). An inner join drops them without a trace, and a left join keeps them unmatched. Write the count with pl.col(key).is_null().sum(), and state the handling: kept unmatched, or excluded by design.
  • Make the join type explicit. Use how="anti" to list the keys that did not match before you decide to drop them.
  • After each filter or aggregation, compare the count with the count the definition predicts. A filter that removes 40 percent of rows needs a reason you can state.
  • Aggregate at the grain the question asks for. Sum the lowest-level rows, then roll up. Do not average per-segment averages unless the weights are equal.
  • Compare the total against an independent figure at the same grain (see principle-explain-the-number).

You skipped this when a join changes the row count with no stated reason, or when a total is reported without row counts before and after.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.