model-explainability
Skill2 starsChanged 12 days agoMITView on GitHub
Model Explainability. Explain machine learning models and export reusable knowledge - explainable AI (XAI), feature importance (permutation, impurity caveats), SHAP values, partial dependence, ALE, ICE, LIME, surrogate trees and rules, counterfactual explanations, feature dictionary, insights.json, model card, data sheet, metric glossary, slice and fairness analysis. Use for interpretability, why did the model predict this, what drives the model, model documentation, model cards or responsible AI reviews.
In this repository16 other files it gives its agents.
- ai-prediction-explainerSkill
- data-mining-machine-learningSkill
- data-miningSkill
- data-preparationSkill
- data-profilingSkill
- data-storytellingSkill
- feature-engineeringSkill
- hyperparameter-tuningSkill
- ml-quality-guardSkill
- ml-web-applicationSkill
- model-deploymentSkill
- model-evaluationSkill
- model-trainingSkill
- notebook-auditSkill
- problem-framingSkill
- statistical-analysisSkill
The file
We split this skill into the rules it gives an agent
1 rule. About 0 are common, 1 is distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 39
Add a metric glossary cell that defines every metric used in plain language.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Model Explainability
Explain the model on held-out data, state what the explanation cannot say, and export the knowledge so people and systems can reuse it.
Global
- Permutation importance on the test or validation set, scored with the headline metric, repeated (10 or more) with intervals.
- Model-native importance only with its caveat: impurity importance favors high-cardinality and continuous features.
- SHAP summary and dependence plots (TreeExplainer for tree ensembles, LinearExplainer for linear models, model-agnostic otherwise).
mlkit.explain.shapley_valuescomputes interventional Shapley values on raw predictors for any pipeline. - Partial dependence for independent features; ALE when features are correlated.
- Surrogate decision tree or extracted rules when a rule-based representation is requested; report its fidelity to the model.
Local
SHAP waterfall or bar for one case, ICE curves, LIME as a cross-check, counterfactuals (mlkit.explain.counterfactuals: smallest change inside the training range that flips the decision).
Caveats in every output
- Importance is not causation; SHAP explains the model, not the world.
- Correlated features split or distort attribution; use grouped importance or ALE.
- Explanations on training data flatter the model; use held-out data.
- Report stability: importance intervals across repeats or resamples.
Knowledge artifacts (artifacts/)
| File | Contents |
feature_dictionary.csv | name, meaning, source, transform, type, unit, rationale, availability, importance with interval, direction |
insights.json | finding, evidence metric with interval, segment, confidence, recommended action |
model_card.md | intended use, data, metrics by slice, limits, fairness checks, monitoring plan |
data_sheet.md | collection, bias risks, known quality issues |
rules.md | surrogate or extracted rules with fidelity, when requested |
Add a metric glossary cell that defines every metric used in plain language.
Fairness
For people-facing decisions: recall, precision and calibration by group with intervals; name the fairness definition chosen and why; record gaps and mitigations in the model card.
Method choice table and code: references/explainability.md.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
Reviews can't be read right now.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.