ai-lead-ops
Skill9 starsChanged 6 months agoMITView on GitHub
Guides AI ops leadership—LLM SRE, model/prompt releases, eval/incidents, cost/capacity, vendors, and cross-functional cadence. Use for AI platform ops, LLM SLAs, incidents, rollout governance, unit economics, red-team/eval gates, and team rituals—not memory (ai-memory-developer), context code (ai-context-engineer), security programs (cybersecurity), token roadmaps (ai-token-improvement-plan-engineer), solution architecture (applied-ai-architect-commercial-enterprise), skills portfolio (ai-skill-manager), or vertical AI product eng management (engineering-manager-vertical-ai-products). Prompt/eval team management and golden-set release policy: engineering-manager-agent-prompts-evals. Safeguard inference platform: ml-infrastructure-engineer-safeguards. Safeguard model research: ml-research-engineer-safeguards.
We split this skill into the rules it gives an agent
1 rule. About 0 are common, 1 is distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 112
Run structured bake-offs before annual renewals
≈ “Common” is an estimate: at least 1 in 5 of the 308,232 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
AI Lead Ops
When to Use
- Standing up AI platform operations and production service reliability
- Defining SLAs/SLOs for LLM-powered features
- Running AI incident reviews and post-mortems
- Governing model, prompt, and index rollouts with tiered gates
- Tracking AI unit economics (cost per session, tokens per feature)
- Coordinating red-team and evaluation gates before releases
- Building team rituals and cadence across engineering, research, risk, and product
- Managing AI vendor relationships, contracts, and bake-offs
When NOT to Use
- Implementing memory stores or context packing code →
ai-memory-developer/ai-context-engineer - Building RAG pipelines or agent tools →
ai-engineer - Designing corporate AI policy or regulatory mapping →
ai-risk-governance - General network penetration testing or enterprise security programs →
cybersecurity - Structured token/cost improvement roadmaps with backlog →
ai-token-improvement-plan-engineer - Commercial/enterprise AI solution architecture →
applied-ai-architect-commercial-enterprise - Vertical AI product engineering managers and squad roadmaps →
engineering-manager-vertical-ai-products
Related skills
| Need | Skill |
| Build RAG, agents, eval harnesses | ai-engineer |
| Memory and context implementation | ai-memory-developer, ai-context-engineer |
| Risk tiering and policies | ai-risk-governance |
| Adversarial testing execution | ai-redteam |
| CI/CD and platform incidents | devops |
| Pipeline security | devsecops |
| Token optimization roadmap and initiative backlog | ai-token-improvement-plan-engineer |
| Commercial/enterprise AI architecture | applied-ai-architect-commercial-enterprise |
| Skills portfolio governance | ai-skill-manager |
| Safeguard inference platform | ml-infrastructure-engineer-safeguards |
| Safety classifier research | ml-research-engineer-safeguards |
Core Workflows
1. Operating model and cadence
| Ritual | Frequency | Outcomes |
| AI ops standup | Daily | Blockers, incidents, deploys |
| Model/prompt change review | Per release | Approvers, eval delta |
| Cost review | Weekly | Spend vs budget, top features |
| Risk & safety sync | Bi-weekly | Incidents, policy gaps |
| Quarterly capacity | Quarterly | Model roadmap, vendor contracts |
Define RACI: who owns model, prompt, index, eval suite, on-call.
See references/operating_model.md for roles and escalation.
2. Release governance
Production promotion checklist:
- Eval regression passed on golden + safety set
- Red-team sign-off for tier-2+ use cases
- Model card / change log updated
- Canary with error and cost monitors
- Rollback procedure tested (previous prompt + model version pinned)
- Comms plan for customer-visible behavior change
See references/release_governance.md for tiered gates and canary metrics.
3. SLOs, incidents, and observability
Example SLIs:
| SLI | Notes |
| Availability | Successful completion / total requests |
| Latency | p95 end-to-end |
| Quality proxy | Thumbs-down rate, escalation rate |
| Safety | Policy violation rate post-deploy |
| Cost | USD per successful session |
AI incident types: toxic output, PII leak in logs, retrieval cross-tenant leak, runaway agent loop, vendor outage.
See references/incidents_slos.md for severity matrix and post-incident template.
4. Cost and capacity
- Track tokens by model, feature, tenant
- Set budgets and alerts at 80/100/110%
- Optimize via routing, caching, context engineering (partner with
ai-context-engineer) - Forecast from usage growth + model price changes
See references/cost_capacity.md for unit economics worksheet.
5. Vendor and eval program
- Maintain scorecard: quality, latency, safety, price, data terms
- Run structured bake-offs before annual renewals
- Own central eval harness ownership and dataset hygiene
See references/vendor_eval_program.md for RFP topics and eval program maturity.
When to load references
- Team cadence and RACI →
references/operating_model.md - Releases and canaries →
references/release_governance.md - SLOs and incidents →
references/incidents_slos.md - Cost and capacity →
references/cost_capacity.md - Vendors and eval ops →
references/vendor_eval_program.md
Other versions1 project uses a version of this.
Word for word · 1
- daemon-blockint-tech/Agentic-Enteprises-Skill★ 8100% same
Edited · 0
None found.
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository197 other files it gives its agents.
- ab-testing-engineerSkill
- actuarial-analystSkill
- actuarial-consultingSkill
- actuarySkill
- advanced-long-term-actuarial-mathematicsSkill
- advanced-persistent-threatSkill
- advanced-short-term-actuarial-mathematicsSkill
- agentic-ai-developerSkill
- ai-adversarial-robustness-engineerSkill
- ai-context-engineerSkill
- ai-engineerSkill
- ai-memory-developerSkill
- ai-redteamSkill
- ai-researcherSkill
- ai-risk-governanceSkill
- ai-skill-managerSkill
- ai-token-improvement-plan-engineerSkill
- aml-cftSkill
- aml-complianceSkill
- analytics-data-engineering-manager-productSkill
- analytics-data-engineerSkill
- anti-false-positive-decision-makingSkill
- applied-ai-architect-commercial-enterpriseSkill
- appointed-chief-actuarySkill
- asset-liability-managementSkill
- associate-actuarySkill
- assumption-settingSkill
- auditorSkill
- bcm-disaster-recovery-specialistSkill
- bi-analystSkill
- build-validatorSkill
- business-analystSkill
- business-consultantSkill
- business-model-researcherSkill
- certified-information-systems-security-professionalSkill
- chief-information-security-officerSkill
- ci-cd-engineerSkill
- cicd-engineerSkill
- cisco-certified-network-professionalSkill
- classified-cyber-security-senior-managerSkill
- classified-software-devsecops-engineerSkill
- cloud-architectSkill
- cloud-compliance-specialistSkill
- cloud-economistSkill
- cloud-engineerSkill
- cloud-security-engineerSkill
- cloud-system-administratorSkill
- cluster-deployment-engineerSkill
- code-securitySkill
- commercial-counselSkill
- communication-leadSkill
- community-executive-escalations-program-managerSkill
- compliance-engineerSkill
- compliance-specialistSkill
- compute-accounting-managerSkill
- control-software-developerSkill
- corporate-counselSkill
- country-managerSkill
- cross-department-translationSkill
- cryptographer-specialistSkill
- The first 60 shown.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
Reviews can't be read right now.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.