agentleFS
bouobbouob/sysadmin-skills/skills/incident/SKILL.md

incident

Skill2 starsChanged 6 months agoMITView on GitHub

IT incident response workflow. This skill should be used when the user asks to "log an incident", "handle an incident", "create incident report", "document an outage", "incident response", "service is down", "major incident", or describes a service disruption requiring structured response. Also use when the user mentions SRE concepts like error budget impact, blameless review, or AIOps-detected incidents. Guides through the full ITIL v5 incident lifecycle: detect, classify, investigate, resolve, communicate, and close.

21 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use runbook, perf and 10 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • AnthropicSkill

    runbook

    Create or update an operational runbook for a recurring task or procedure. Use when documenting a task that on-call or ops needs to run repeatably, turning tribal knowledge into exact step-by-step commands, adding troubleshooting and rollback steps to an existing procedure, or writing escalation paths for when things go wrong.

    Also in 4 projects that use this

  • MicrosoftSkill

    perf

    Speed and memory performance rules for Rust crates, the Node addon, webui-framework, and webui-delivery.

    Also in 3 projects that use this

  • drewdrewthisSkill

    sweep

    Search for similar patterns, anti-patterns, or issues elsewhere in the codebase after a fix or feature. Use after implementing a fix to catch the same bug in other places, or proactively to audit for a known pattern.

    Also in 3 projects that use this

  • OpenAISkill

    observability

    Vercel Observability expert guidance — Drains (logs, traces, speed insights, web analytics), Web Analytics, Speed Insights, runtime logs, custom events, OpenTelemetry integration, and monitoring dashboards. Use when instrumenting, debugging, or optimizing application performance and user experience on Vercel.

    Also in 5 projects that use this

  • mijunriSkill

    database

    数据库脚本与操作规范。脚本放在 db_scripts 目录,遵循查询优化、事务、安全等实践。

    Also in 3 projects that use this

  • tuistSkill

    migrate

    Migrates existing Xcode projects to Tuist generated workspaces with build and run validation, external dependency mapping, and migration checklists. Use when adopting Tuist for an existing app or converting a hand-edited Xcode project to generated projects.

    Also in 4 projects that use this

The file

Show its rules and text

We split this skill into the rules it gives an agent

15 rules. 0 are common, 12 are distinctive, and 3 are very specific to this project.

  • 0

    Common

    Most of the 19 edited versions we read keep these.

  • 12

    Distinctive

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  • could be automatic: a setting could enforce it
  1. Very specific:

    Default output directory: ./incidents/.

    line 37
  2. Distinctive:

    All incident documents are written in English.

    line 39
  3. Very specific:

    If SEV-1 or P1/P2: Major Incident procedures apply (see itil skill reference references/incident-management.md).

    line 78
  4. Distinctive:

    Create the incident report file with the initial entry.

    line 82
  5. Very specific:

    If a status page update is also needed, coordinate with the /statuspage skill.

    line 130
  6. Distinctive:

    Update the Status field (Investigating → Identified → Monitoring → Resolved)

    line 136
  7. Distinctive:

    Add to Investigation section with findings

    line 138
  8. Distinctive:

    When the user indicates the issue is resolved:

    line 142
  9. Distinctive:

    Update the incident report:

    line 151
  10. Distinctive:

    SRE Postmortem (for SEV-1 and SEV-2): If this was a SEV-1 or SEV-2 incident, a traditional PIR may not be sufficient.

    line 166
  11. Distinctive:

    Error budget: If the organization tracks SLOs, note the estimated error budget impact in the follow-up section.

    line 169
  12. Distinctive:

    Always use the user-facing service name in communications, not internal system names

    line 175
  13. Distinctive:

    Never include IP addresses, hostnames, or credentials in the incident report

    line 176
  14. Distinctive:

    If the incident was caused by a recent change, link the change request ID in the report

    line 179
  15. Distinctive:

    Ask the user whether this incident needs to follow their organization's ticketing system (ServiceNow, Jira Service Management, etc.) in …

    line 180

“Common” is measured: kept by at least 6 in 10 of the 19 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Incident Response Workflow

Structured incident response following ITIL 4 Incident Management. Produces a complete incident report as a local markdown file.

Related Skills (auto-loaded)

  • itil — priority classification (P1-P4), incident management process, escalation matrix
  • comms — stakeholder notification templates, severity-based tone guidance
  • done-ops — verification gate before closing the incident

Output

File: {OUTPUT_DIR}/incident-{YYYY-MM-DD}-{slug}.md

  • slug: lowercase kebab-case summary, e.g. email-outbound-failure, vpn-india-outage
  • Incident ID (INC-{YYYYMMDD}-{NNN}): {NNN} is a daily sequence number starting at 001. If the organization uses a ticketing system (ServiceNow, Jira SM), use that system's ID instead.

Default output directory: ./incidents/. If a docs/incidents/ directory exists in the project, use that instead. Ask the user if neither exists. A user-specified path takes precedence over all defaults.

All incident documents are written in English. Before writing to file, preview the content for the user in their preferred language.


Workflow

Step 1: Gather Information

Collect the following from the user (use AskUserQuestion for missing items):

FieldRequiredExample
What is happening?Yes"Email service is not sending outbound messages"
When did it start?Yes"Noticed at 09:15 today" / "Alerts fired at 14:30"
Who/what is affected?Yes"All users in HQ" / "Customer-facing API"
Severity indicatorsYesIs there a workaround? How many users? Revenue impact?
What has been tried?No"Restarted the mail service, no change"

Step 2: Classify (SEV then P)

First — assign Severity (objective impact scope):

LevelScopeDefinition
SEV-1All users / all regionsCore service completely down, no workaround
SEV-2Many users or major functionSignificant degradation, partial workaround
SEV-3Subset of users, non-criticalLimited impact, workaround available
SEV-4Single user or negligibleCosmetic, proactive, informational

Then — assign Priority (handling order, Impact × Urgency):

High UrgencyMedium UrgencyLow Urgency
High ImpactP1 CriticalP2 HighP3 Medium
Medium ImpactP2 HighP3 MediumP4 Low
Low ImpactP3 MediumP4 LowP4 Low

Present both SEV and P to the user for confirmation. MUST use the exact format: SEV-1, SEV-2, P1, P2 (not "Critical" or "High" alone).

If SEV-1 or P1/P2: Major Incident procedures apply (see itil skill reference references/incident-management.md).

Step 3: Document Timeline

Create the incident report file with the initial entry. MUST follow this exact structure — do not rearrange or rename sections:

# Incident Report: {TITLE}

| Field | Value |
|---|---|
| Incident ID | INC-{YYYYMMDD}-{NNN} |
| Severity | SEV-{1/2/3/4} |
| Priority | P{1/2/3/4} |
| Status | Investigating |
| Reported by | {NAME/SYSTEM} |
| Start time | {YYYY-MM-DD HH:MM TZ} |
| Affected services | {LIST} |
| Affected users | {SCOPE} |

## Impact Summary

{One paragraph describing what users are experiencing.}

## Timeline

| Time | Event |
|---|---|
| {HH:MM} | {Event description} |

## Investigation

{Findings so far, diagnostics performed, hypotheses.}

## Resolution

{To be completed upon resolution.}

## Follow-up Actions

- [ ] {Action items identified during the incident}

Step 4: Draft Communications

Using the comms skill, draft the appropriate notifications:

  • Initial notification — based on priority level and audience
  • Present the draft to the user for review before including in the report

For P1/P2: remind the user about the update cadence (P1: every 15-30 min, P2: every 30-60 min).

If a status page update is also needed, coordinate with the /statuspage skill. The incident report contains full technical detail; the status page update contains only impact-focused, user-safe information.

Step 5: Track Progress

As the user provides updates:

  • Append to the Timeline section
  • Update the Status field (Investigating → Identified → Monitoring → Resolved)
  • Draft follow-up communications as needed
  • Add to Investigation section with findings

Step 6: Resolution

When the user indicates the issue is resolved:

  1. Load done-ops verification gate
  2. Walk through the incident closure checklist:
    • Service restored and verified?
    • User/reporter confirmation obtained?
    • Monitoring stable (no recurring alerts)?
    • Documentation complete?
    • Problem record needed? (recurring incident or unknown root cause)
  3. Update the incident report:
    • Set Status to Resolved
    • Record resolution time
    • Fill in the Resolution section
    • Calculate total duration
  4. Draft the resolution notification using comms templates

Step 7: Close and Follow-up

Complete the incident report:

  • Record all follow-up actions with owners and due dates
  • Note if a Post-Incident Review is needed (mandatory for P1/P2, recommended for P3)
  • Recommend creating a Problem record if root cause is unknown
  • Present the final report to the user for review

SRE Postmortem (for SEV-1 and SEV-2): If this was a SEV-1 or SEV-2 incident, a traditional PIR may not be sufficient. Prompt the user:

"This was a {SEV-1/SEV-2} incident. Consider running a blameless postmortem using /postmortem — it produces a more systematic analysis with contributing factors, what went well, and SMART action items. Use /problem if root cause is unknown and the incident is likely to recur."

Error budget: If the organization tracks SLOs, note the estimated error budget impact in the follow-up section.


Gotchas

  • Always use the user-facing service name in communications, not internal system names
  • Never include IP addresses, hostnames, or credentials in the incident report
  • For P1 incidents, do not wait for complete information before drafting the initial notification — speed matters
  • Resolution time = time from detection to confirmed restoration, not when the ticket is closed
  • If the incident was caused by a recent change, link the change request ID in the report
  • Ask the user whether this incident needs to follow their organization's ticketing system (ServiceNow, Jira Service Management, etc.) in addition to the local markdown report

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.