agentleFS
atsushi-greenatsushi-green/ds-ai-coding-skills/.claude/skills/statistical-ml-review/SKILL.md

statistical-ml-review

Skill86 starsChanged 53 days agoMITView on GitHub

Use this when reviewing statistical analysis, experiments, model evaluation, or ML work done by someone else (a notebook, PR, or report) — checking that the required context is stated and that every method applied comes with the figures and values required by the matching *-diagnostics skill. When running an analysis yourself, apply the matching *-diagnostics skill directly.

The file

We split this skill into the rules it gives an agent

6 rules. About 0 are common, 3 are distinctive, and 3 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈3

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Use this skill when reviewing statistical analysis, experiments, model evaluation, or ML work.

    line 8
  2. Very specific:

    When you run an analysis yourself, the matching *-diagnostics skill applies directly.

    line 9
  3. Very specific:

    Check that the report contains the 診断サマリー table (| 診断項目 | 実測値 | 合格基準 | 判定 | 次アクション |) and that every 要対処 has a next action.

    line 58
  4. Distinctive:

    Avoid overclaiming — state what the data supports, not what you hope it shows.

    line 68
  5. Distinctive:

    Document the experimental design (A/B test, pre-post, observational, etc.).

    line 69
  6. Distinctive:

    Compare against a simple baseline: before reporting model performance.

    line 76

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Skill: Statistical and ML Review

Use this skill when reviewing statistical analysis, experiments, model evaluation, or ML work. When you run an analysis yourself, the matching *-diagnostics skill applies directly; this skill only checks that its required outputs (図・値・判定表) are present and that the context below is stated.

Required Context for Any Analysis

The analysis must clearly state:

  • Target variable: What are we predicting or measuring?
  • Unit of analysis: What does one row represent?
  • Time period: What date range does the data cover?
  • Sample definition: What population is included?
  • Exclusions: What was filtered out, and why?
  • Assumptions: What statistical or business assumptions apply?
  • Leakage risks: Could future information leak into training data?
  • Evaluation metric: How will success be measured?
  • Baseline: What is the simplest comparison point?

Method → Diagnostics Skill

The *-diagnostics skills are two-tier: a family router (SKILL.md) plus one references/<method>.md per method. For each method used in the code, open the router, then the reference named below, and check its 「必ず出す図」「必ず出す値」 against the outputs. Trigger vocabulary (library / class / function names) is listed in each router's description and routing table.

Method usedRouter skillReference to check
Linear regression, Lasso / Ridge, quantile regressionstatistical-inference-diagnosticsreferences/ols.md
Logistic / Poisson / negative binomial, GLMsamereferences/glm.md
Mixed effects, panel fixed effectssamereferences/mixed-effects.md
Hypothesis tests (t / χ² / ANOVA / nonparametric)samereferences/hypothesis-test.md
Kaplan-Meier, Cox, AFT, competing riskssamereferences/survival.md
MCMC (cmdstanpy)samereferences/bayesian-mcmc.md
Supervised ML evaluation (splits, CV, tuning, leakage)predictive-modeling-diagnosticsreferences/ml-evaluation.md
Decision tree, random forest, GBDTsamereferences/tree-model.md
SHAP, permutation importance, PDP / ICEsamereferences/model-interpretation.md
ARIMA / state space / Prophet / VAR, change points, forecastingsamereferences/time-series.md
A/B testcausal-inference-diagnosticsreferences/ab-test.md
Propensity score, IPW, DMLsamereferences/causal-observational.md
DiD, IV, RDD, synthetic controlsamereferences/causal-quasi-experimental.md
Missing data, outliers, EDA preprocessingunsupervised-eda-diagnosticsreferences/missing-data.md
k-means, hierarchical, GMM, DBSCANsamereferences/clustering.md
PCA, factor analysis, UMAP / t-SNEsamereferences/dimensionality-reduction.md
Anomaly detectionsamereferences/anomaly-detection.md
Monte Carlo, discrete-event simulationsimulation-optimization-diagnosticsreferences/simulation.md
LP / MIP, metaheuristicssamereferences/optimization.md

Review Procedure

  1. List every method applied as an analysis in its own right (search the code for the library calls named in each router's description and routing table). Do not count helper usages that the routers exclude — e.g. a PCA projection inside a clustering figure, a propensity-score LogisticRegression, smf.ols used to estimate an event study, random numbers for permutation or bootstrap baselines. Within one reference, check only the rows for the method actually used; a missing figure for a method that was not used is not a gap.
  2. For each method, compare the outputs with the reference's 「必ず出す図」「必ず出す値」 and 「落とし穴」, and the router's 「出力と報告」 format.
  3. Check that the report contains the 診断サマリー table (| 診断項目 | 実測値 | 合格基準 | 判定 | 次アクション |) and that every 要対処 has a next action.
  4. Report gaps as a table, in Japanese:

    手法欠けている図・値結論への影響推奨アクション

Experiments and Causal Inference

  • Distinguish correlation from causation explicitly.
  • Mention confidence intervals or uncertainty when reporting results.
  • Avoid overclaiming — state what the data supports, not what you hope it shows.
  • Document the experimental design (A/B test, pre-post, observational, etc.).

Machine Learning

  • Separate train, validation, and test sets clearly.
  • Avoid leakage — no future data in training, no target leakage in features.
  • Document preprocessing — encoding, scaling, imputation, feature engineering.
  • Compare against a simple baseline before reporting model performance.
  • Consider cross-validation — especially for small datasets where a single split may be unreliable.
  • Report limitations — data quality, sample bias, generalizability.
  • Version datasets and model artifacts when practical.

Review Checklist

  • Is the target variable well-defined?
  • Is the unit of analysis clear?
  • Are train/test splits time-aware if data is temporal?
  • Is there a baseline comparison?
  • Are evaluation metrics appropriate for the problem?
  • Are limitations documented?
  • Is the analysis reproducible?
  • For every method applied, are the figures and values required by its *-diagnostics skill present, with the 判定表?

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.