build-model
Skill4 starsChanged 7 days agoMITView on GitHub
Scoped machine-learning model development — framing, leakage-safe dataset construction, baseline, training, business-terms evaluation, model card, scoring pipeline, and monitoring plan. Use for churn/propensity, forecasting, segmentation, or any predictive modelling.
4 other projects use a version of this.
Adopted by
Most recently taken up by
- maxencefaldor/brickytheir own version, first seen Nov 8, 2026
- nikhilw/structured-agentic-workflowtheir own version, first seen Nov 6, 2026
- analyticsengineering/ae-toolkittheir own version, first seen Oct 12, 2026
- daloopa/investingtheir own version, first seen Oct 8, 2026
Other versions4 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 4
- daloopa/investing★ 488same name
- maxencefaldor/bricky★ 20same name
- nikhilw/structured-agentic-workflow★ 7same name
- analyticsengineering/ae-toolkit★ 3same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository21 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- analyzeSkill
- build-dashboardSkill
- build-pipelineSkill
- causal-analysisSkill
- connect-dataSkill
- define-kpisSkill
- evalSkill
- experimentSkill
- forecastSkill
- health-checkSkill
- investigate-metricSkill
- make-deliverableSkill
- powerbiSkill
- privacy-reviewSkill
- research-domainSkill
- scorecardSkill
- setup-teamSkill
- tableauSkill
- triageSkill
- upgradeSkill
The file
Files
Build Model — ML Development
Owner: data-scientist, with analytics-engineer/data-engineer for data plumbing. Args describe the goal, e.g. /build-model churn prediction for monthly subscribers.
Procedure
- Frame — and screen. Define: prediction target (unit, horizon, exact label definition), how predictions will be used (who acts, at what threshold, scored how often), success criterion in business terms ("flag 60% of churners in the top decile we can call"). A model earns its keep only once it drives an action — a modest model wired into a decision beats a brilliant one nobody uses, and beating the current guess is usually enough. Then the screening question: would a heuristic or segmentation answer this well enough? If yes, deliver that instead and stop — say why. If the goal is to change behaviour with a treatment (offer, outreach), consider uplift modelling — rank persuadables by incremental effect, not raw propensity — which needs a randomised control group (design via
/experiment). - Privacy screen. If the model uses data about people or its output will be attached to a person, run
/privacy-reviewbefore building the dataset. It sets the risk tier (minimal / limited / high), which features are off the table, and whether a human must be able to override the output. A feature that proxies for a protected characteristic (postcode for ethnicity, first name for gender) is a design decision to make now, not a finding to explain later. - Confirm feasibility. Enough labelled history? (rule of thumb: hundreds of positive examples minimum). Features available at prediction time? Environment ready ({{ML_ENVIRONMENT}})? If data is missing, route ingestion to data-engineer before modelling.
- Build the modelling dataset — leakage-safe. Point-in-time correct features only; document label window vs feature window with a timeline diagram in the README. Train/validation/test split: time-based for temporal problems, grouped by entity where entities repeat. Persist the dataset-building code; never hand-edit data.
- Baseline first. Majority class / last value / seasonal naive / simple regression — whichever fits. All later models are judged against this.
- Train. Start interpretable-and-boring (logistic regression, gradient-boosted trees; ETS/ARIMA-class for forecasts); reach for ensembles (random forests, boosting, stacking) only when the accuracy gain is worth the interpretability cost. Wrap all preprocessing (impute/scale/encode/select/resample) in a pipeline fit on the training fold only — no fitting on the full data before splitting. Tune modestly (grid/randomized/successive-halving); use nested CV when tuning and reporting on the same data. For class imbalance (churn, fraud), use class weights or train-fold resampling and threshold-tune to the business cost — never optimise raw accuracy. Seeds set, versions pinned in
models/<name>/requirements.txt. - Evaluate in business terms. Primary: the success criterion from step 1 (e.g. precision/recall at the actionable threshold, MAPE at the planning horizon). Secondary: calibration if probabilities are consumed, per-segment performance (does it fail a key segment?), stability across time slices, and honest uncertainty (forecast intervals). Always shown alongside the baseline.
- Document.
models/<name>/MODEL_CARD.md: purpose and intended use (the decision it supports, and uses it is not validated for), owner, data and date ranges, features, algorithm, performance vs baseline, segment caveats, known limitations, retraining cadence and trigger. Where the model scores or decides about people, the card also carries: the risk tier from the privacy review; a fairness evaluation — error rates and predictive value by the groups that matter, with the criterion you optimised and the trade-off stated (analytics.mdPart 1: you cannot equalise both when base rates differ); the human oversight arrangement — who can override a prediction and how a person affected can contest it; and the personal data used with its classification. - Ship the scoring path. A reproducible script: fresh data in → predictions out to a warehouse table or file the team can consume (
models/<name>/score.pyor equivalent). Schedule per the use case via {{ORCHESTRATOR or "documented manual run"}}. - Monitoring plan. Input drift checks, fresh-label performance tracking, per-group performance where the fairness evaluation applies, and the retraining trigger — handed to
performance-monitoras queries/thresholds. - Review, then confirm before automation.
analysis-reviewerchecks the evaluation before results are presented — leakage and an optimistic split are the defects it is looking for. If model outputs will drive automated customer-affecting decisions, get explicit user sign-off on thresholds and failure modes before wiring anything up; ahighrisk tier also needs the privacy contact named inknowledge/governance-policy.md.
For a forecast of a metric over time, use /forecast — it has the backtesting and interval-calibration procedure this skill does not.
Created by Colin Beck — https://www.linkedin.com/in/beckcolin/
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

