agentleFS
Sign inSign up
Registry

Text rule trends

skill · published by agentleFS · version 1 · approved by frdchang@gmail.com on 2026-10-12

Measure how the rules in versioned text files change over time: extract, cluster into themes, and compare periods.

Mount
SKILL.md

Text rule trends

Turns per-file git histories into numbers about what those files say, and how that changed.

It extracts line-level items (by default "rules": imperative or normative lines) from every version of each file and follows each item's life. It reads each file at an age you choose (or at the end of every period), embeds the items with a sentence model (bge-small by default), clusters them with your own k, maps clusters to themes from your JSON, bins by month, quarter or year, labels each theme rise, fall or steady with your threshold, bootstraps a before/after comparison around any date, and counts one file kind's share of new files against another. The extractor can be swapped for a regex or your own plugin function. It reads histories from github-file-history and writes the JSON that tufte-charts and watercolor-data-art draw.

Example output (analyze.py on the made-up smoke test below):

== themes, share of a file's items 2025Q2 -> 2026Q2
  rise    Git: main branch, commits, secrets          25.0% ->  37.5%
  rise    Docs and changelog                          12.5% ->  20.8%
  fall    Tests                                       37.5% ->  29.2%
  fall    Code style                                  25.0% ->  12.5%

Use it when:

  • you collected many versions of one kind of file (with github-file-history) and want to know which kinds of lines appear, grow or disappear: rules in CLAUDE.md or .cursor/rules, sections in READMEs, dependencies in package.json, steps in CI files
  • you want input for tufte-charts or watercolor-data-art

Requirements

Python 3.8+ and pip install -r requirements.txt (numpy, scikit-learn, fastembed). fastembed downloads the embedding model the first time (bge-small is about 70 MB). Use --cache-dir to choose where. sentence-transformers is an optional second backend.

Inputs

  • One folder per sample, holding vers/ as written by github-file-history: one <key>.tsv.gz or <key>.tsv per file, one row per commit (sha, author date, Claude co-author 0/1, base64 body). A manifest.tsv beside it, if present, maps keys to paths.
  • A themes file (your JSON) that names clusters and groups them into themes, bottom to top: {"centroids_id": "sha256:...", "clusters": {"0": "name", ...}, "themes": [{"name": "...", "clusters": [0, 3]}, ...]}. Without one, every cluster is its own theme. centroids_id is the "id" printed in the centroids.json the names were written for. Cluster numbers mean nothing outside that one set of centroids, so analyze.py refuses a themes file whose centroids_id differs from the labels' (--force-themes overrides), and refuses samples labelled against different centroids.
  • Optionally a config file with the analysis settings (see examples/*/config.json).

Steps

Run the scripts by their path, and write outputs to a folder of your own, never inside the skill folder. Below, SKILL is this skill's folder and P, R are your sample folders.

# 1. items and their lives, per sample                   -> P/rules.json
python3 $SKILL/scripts/extract.py $P                       # the default "rules" extractor
python3 $SKILL/scripts/extract.py $P --extractor regex --regex '^#{1,3}\s+(.+)'   # Markdown headings
python3 $SKILL/scripts/extract.py $P --extractor plugin --plugin my_items.py:items

# 2. embeddings, per sample                              -> P/emb.npy (a local working file)
python3 $SKILL/scripts/embed.py $P [--model BAAI/bge-small-en-v1.5] [--cache-dir DIR]

# 3. clusters: fit on one sample, assign the others to its centroids
python3 $SKILL/scripts/cluster.py fit $P --k 40 > clusters.txt    # -> P/labels.json, P/centroids.json
python3 $SKILL/scripts/cluster.py assign $R --centroids $P/centroids.json

# 4. read clusters.txt, name the clusters and group them into themes.json,
#    with "centroids_id" set to the "id" in P/centroids.json

# 3b. or reuse someone's named clusters: assign EVERY sample to their centroids.json
python3 $SKILL/scripts/cluster.py assign $P --centroids their/centroids.json

# 5. trends                                              -> OUT/*.json
python3 $SKILL/scripts/analyze.py --sample a=$P --sample b=$R --themes themes.json \
    --config config.json --out OUT

Extractors (extract.py)

--extractorwhat is an itemdefault words
rulesa list item, or a line starting with a capital letter, whose text (after an optional bold label) starts with an imperative or normative word: always, never, use, run, avoid, must, should, ... Fenced code, headings and table rows are skipped4 to 45
linesevery non-blank line outside fenced code, list markers removed1 to 45
regexlines matching --regex; the item is group 1 if there is one, else the whole match1 to 45
pluginwhatever your function returns: --plugin FILE.py:FUNC, where FUNC(body) returns a list of strings1 to 45

Other settings: --min-words, --max-words, --match (default 0.6: an item keeps its identity through a rewording when its words overlap that much, by token Jaccard, with an unmatched item of the previous version), --kind-map (a JSON {"regex": "label"} matched against each file's path, to group file names into kinds; the default kind is the file name), and --order.

Version order. Versions are replayed oldest first. --order datetime (default) sorts by the parsed author date, so mixed UTC offsets come out in the right order. --order string sorts by the date text, which some older runs used; pass it to reproduce one of those.

Analysis settings (analyze.py, or keys in --config)

settingdefaultmeaning
modecohortcohort: bin files by creation period, read each at creation + age. snapshot: in every period, read every live file as it stood on the period's last day
age30days after creation
createdfirst-itema file's creation date: its earliest item, or first-version (its first commit)
periodquarterquarter, month or year
periodsall with datathe bins to report
from, tofirst, last binthe two bins compared for rise and fall
threshold0.15relative change in a theme's share that counts as rise or fall. A theme that starts at 0 and grows is rise. If from or to is an empty bin, its shares are null and every theme is too few
min_files20if a compared bin has fewer files, every theme is too few; a window side with fewer is flagged too few
before, afternonewindows A:B and C:D by creation date, [A, B) and [C, D)
splitnoneone date instead: before = every file created earlier, after = created on it or later
kindsall, most files firstkinds reported separately, in this order (the bootstrap draws follow it)
sharenoneA:B: kind A's share of new files among A and B, per period
boot, seed3000, 3bootstrap resamples and numpy default_rng seed
todaythe current datefiles younger than age on this day are left out, and share.json leaves out files first committed after it. Set it to make a run repeatable

Windows compare files by when they were created, each read at creation + age, in both modes. In snapshot mode the bins are calendar periods, but the windows still work this way.

Outputs

fileschema
<sample>/rules.json{nfiles, nhistory, life: [{kind, born, died, text, norm, repo}], files: {key: {kind, first, deleted}}, co_month, settings}. nhistory counts history files found, nfiles those with a version. repo is the history file's name (one file in one repo), kept under that name for compatibility
<sample>/emb.npyfloat32 [items, dim], same order as life. A local working file
<sample>/labels.json{k, labels, centroids_id}
<sample>/centroids.json{k, dim, centroids: [[...]]}, unit length, plain text. Values are the shortest decimals that read back as the same float32, so assigning with the JSON gives the same labels as a .npy. cluster.py convert turns one into the other
OUT/pooled.json{Q, nfiles, tot, med, pf, files_with_rules, overlap}: per period, files, mean and median items per file, mean items per file in each cluster. An empty bin is null, never 0
OUT/themes.json{q, nfiles, tot, med, from_q, to_q, threshold, groups: [{name, kind, v, share_from, share_to}], rest, files_with_items, items, samples: {name: {history_files, files, files_with_items, items}}}. kind is rise, fall, steady or too few. share_from and share_to are fractions (0.181 = 18.1%). Empty bins are null. The input to both chart skills
OUT/compare.jsonper sample and kind: periods (n, mean, median, pf per period, whatever the unit) and window (means, medians, difference, bootstrap 95% CI, and too_few)
OUT/share.jsonper sample, with share: {Q, files, history_files, <kind>: [new files per period], share_of, share_vs, share}. share is in percent (87.2 = 87.2%). Counts every file by its first commit, with or without items, over periods if set (else every period), leaving out files first committed after today

Examples

Smoke test: made-up CONTRIBUTING.md rules (examples/made-up/)

vers/ holds twelve made-up projects, each starting a CONTRIBUTING.md and editing it a few times. Later projects write fewer style rules and more git rules, so the trend has a known shape. centroids.json, themes.json and config.json beside it name one k=4 fit. Under a minute after the model download:

mkdir -p /tmp/trt && cp -r $SKILL/examples/made-up/vers /tmp/trt/
python3 $SKILL/scripts/extract.py /tmp/trt              # files 12  with items 12  items 96
python3 $SKILL/scripts/embed.py /tmp/trt                # embedded 96 items
python3 $SKILL/scripts/cluster.py assign /tmp/trt --centroids $SKILL/examples/made-up/centroids.json
python3 $SKILL/scripts/analyze.py --config $SKILL/examples/made-up/config.json \
    --sample made-up=/tmp/trt --themes $SKILL/examples/made-up/themes.json --out /tmp/trt/out

Expected: "Git: main branch, commits, secrets" rises (25.0% → 37.5% of a file's rules, 2025Q2 → 2026Q2), "Docs and changelog" rises (12.5% → 20.8%), "Tests" falls (37.5% → 29.2%) and "Code style" falls (25.0% → 12.5%). The split at 2025-10-01 compares 5 files with 7. The config uses min_files: 2 because the bins are tiny; real data wants the default 20.

README section headings (examples/readme-sections/)

The README histories of the five repos in github-file-history/examples/readme-package, read as Markdown headings, in snapshot mode by year:

python3 $SKILL/scripts/extract.py $D --extractor regex --regex '^#{1,3}\s+(.+)' \
    --kind-map '{"(?i)^readme": "README"}'          # files 12  with items 7  items 151
python3 $SKILL/scripts/embed.py $D
python3 $SKILL/scripts/cluster.py fit $D --k 6      # then name the clusters: themes.json
python3 $SKILL/scripts/analyze.py --config $SKILL/examples/readme-sections/config.json \
    --sample five=$D --themes $SKILL/examples/readme-sections/themes.json --out OUT

On the data collected 2026-10-11, headings per README went from 13.0 (2016) to 15.8 (2025), and "API reference entries" rose from 35.9% to 45.6% of them. themes.json names that one k=6 fit; a new collection or fit needs new names. package_deps.py is a plugin example: each dependency in a package.json becomes an item (93 items from the four package.json histories).

Worked example. These skills were first written for the agentleFS article "CLAUDE.md best practices" (https://agentlefs.com/blog/claude-md-best-practices), which studied 5,786 CLAUDE.md and AGENTS.md files. Its data, settings and step-by-step recipe live outside the skills, in the article-claude-md-best-practices folder (RECIPE.md and data/).

Limits and caveats

  • The extractors are heuristics. The rules extractor counts imperative lines in English and misses rules written as prose paragraphs or tables. Check a sample of rules.json by eye.
  • Clusters are a model's grouping; themes are a person's. Cluster numbers depend on the embeddings, the sklearn version and the data. A fit on new data gives new clusters, even with the same k and seed: in one case, a fit on a fresh collection of the same kind of files matched the older centroids at 0 of 40 positions. So to reuse named themes on new data, assign every sample to the centroids they were named for. cluster.py fit --match-to OLD.json renumbers a new fit to line up with old centroids (Hungarian matching on cosine) and reports how close they are, but its clusters are still new ones, and analyze.py will refuse the old themes for them. Labels from the fit itself and labels assigned to its centroids differ for a few items (about 1%).
  • Small numbers. A bin or window side with fewer than about 20 files is noise. analyze.py marks those too few. Look at nfiles before you read a bin.
  • Dates show timing, not cause. A before/after split on a date cannot separate that date from anything else that happened then.
  • Sample bias carries over from collection. Report samples separately before you pool them.
  • Licences. rules.json quotes the files. Only publish it, or quotes from it, for repos whose licence allows that.

Credits

  • bge-small-en-v1.5 by BAAI (Beijing Academy of Artificial Intelligence), the default embedding model, https://huggingface.co/BAAI/bge-small-en-v1.5 . MIT licence per its model card (checked 2026-10-11). fastembed downloads the quantized ONNX port Qdrant/bge-small-en-v1.5-onnx-Q, also MIT per its model card. The model is downloaded, not bundled; centroids.json files hold k-means centres computed from its vectors.
  • fastembed by Qdrant (Apache-2.0, per the package metadata of 0.9.0): the embedding runtime.
  • scikit-learn (BSD-3-Clause): KMeans for cluster.py fit. numpy (BSD-3-Clause) for the vectors and the bootstrap; SciPy (BSD-3-Clause) for the optional Hungarian matching in cluster.py fit --match-to. All are installed by you, not bundled.