agentleFS
Sign inStart free
BULDEEBULDEE/ai-craftsman-superpowers/skills/mlops/SKILL.md

mlops

Skill35 starsChanged 53 days agoApache-2.0View on GitHub

Audit ML projects for production readiness. Use when reviewing ML pipelines, deployments, or infrastructure against MLOps best practices.

1 other project uses a version of this.

Adopted by

Most recently taken up by

  • Other versions1 project uses a version of this.

    Word for word · 0

    None found.

    Edited · 0

    None found.

    Wrote their own version · 1

    1. igbnetwork/skills-igb★ 1same name

    How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.

  • In this repository23 other files it gives its agents.

The file

Create a free account to mount
Files
SKILL.md

/craftsman:mlops - ML Production Readiness Audit

Outcome Contract

  • Outcome: a production-readiness verdict on the ML system, with gaps named and prioritized.
  • Done when: each MLOps dimension is assessed against what exists in the repository, not assumed, and every gap has a concrete remediation.
  • Evidence: the files and pipelines inspected, and the assessment per dimension.

Audit ML projects against production best practices.

MLOps Maturity Levels

Level 0: Manual           - Notebooks, manual deployment
Level 1: ML Pipeline      - Automated training, manual deploy
Level 2: CI/CD for ML     - Automated training + deployment
Level 3: Full MLOps       - Automated retraining, monitoring, feedback

Six Pillars of MLOps

1. Data Management

## Data Checklist

- [ ] Data versioning (DVC, Delta Lake)
- [ ] Data validation (Great Expectations)
- [ ] Feature store (Feast, Tecton)
- [ ] Data lineage tracking
- [ ] Privacy compliance (PII handling)

2. Model Development

## Development Checklist

- [ ] Experiment tracking (MLflow, W&B)
- [ ] Reproducible training (seeds, configs)
- [ ] Model registry
- [ ] Baseline comparisons
- [ ] Hyperparameter tuning automation

3. Model Deployment

## Deployment Checklist

- [ ] Model serving (TorchServe, TF Serving, FastAPI)
- [ ] Containerization (Docker)
- [ ] Orchestration (K8s, ECS)
- [ ] A/B testing capability
- [ ] Canary deployments
- [ ] Rollback mechanism

4. Monitoring

## Monitoring Checklist

- [ ] Model performance metrics
- [ ] Data drift detection
- [ ] Concept drift detection
- [ ] Prediction latency
- [ ] Resource utilization
- [ ] Alerting thresholds

5. CI/CD for ML

## CI/CD Checklist

- [ ] Automated testing (unit, integration)
- [ ] Model validation gates
- [ ] Automated retraining triggers
- [ ] Shadow deployments
- [ ] Feature flag integration

6. Governance

## Governance Checklist

- [ ] Model documentation
- [ ] Bias/fairness audits
- [ ] Explainability (SHAP, LIME)
- [ ] Audit trail
- [ ] Access control

Audit Template

# MLOps Audit: [Project Name]

## Summary
- **Maturity Level:** [0-3]
- **Critical Issues:** [Count]
- **Recommendations:** [Count]

## Pillar Assessment

### 1. Data Management [Score: X/10]

| Aspect | Status | Notes |
|--------|--------|-------|
| Versioning | ✅/⚠️/❌ | |
| Validation | ✅/⚠️/❌ | |
| Feature Store | ✅/⚠️/❌ | |

**Issues:**
- [Issue 1]
- [Issue 2]

**Recommendations:**
- [Recommendation 1]

### 2. Model Development [Score: X/10]
[Same structure]

### 3. Deployment [Score: X/10]
[Same structure]

### 4. Monitoring [Score: X/10]
[Same structure]

### 5. CI/CD [Score: X/10]
[Same structure]

### 6. Governance [Score: X/10]
[Same structure]

## Risk Assessment

| Risk | Probability | Impact | Mitigation |
|------|-------------|--------|------------|
| Model drift undetected | High | High | Implement monitoring |
| Data pipeline failure | Medium | High | Add validation |

## Roadmap

### Quick Wins (1-2 weeks)
- [ ] [Action 1]
- [ ] [Action 2]

### Medium Term (1-3 months)
- [ ] [Action 1]
- [ ] [Action 2]

### Long Term (3-6 months)
- [ ] [Action 1]
- [ ] [Action 2]

Common Anti-Patterns

Anti-PatternProblemSolution
Notebook-to-prodNo reproducibilityPipeline extraction
No versioningCan't reproduceDVC + MLflow
Train/serve skewDifferent preprocessingFeature store
No monitoringSilent failuresDrift detection
Manual deploymentSlow, error-proneCI/CD pipeline

Tool Recommendations

CategoryTools
Experiment TrackingMLflow, Weights & Biases, Neptune
Data VersioningDVC, Delta Lake, LakeFS
Feature StoreFeast, Tecton, Hopsworks
Model ServingBentoML, Ray Serve, Seldon
MonitoringEvidently, WhyLabs, Arize
OrchestrationKubeflow, Airflow, Prefect

Process

  1. Identify current maturity level
  2. Audit each pillar
  3. Identify gaps and risks
  4. Prioritize improvements
  5. Create actionable roadmap

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free