Runbook Creation

Create structured runbooks with step-by-step procedures, troubleshooting, rollback, and escalation paths. Designed so any qualified operator can execute reliably.

Workflow

Step 1: Define Scenario

Trigger (alert, schedule, request, incident), scope, prerequisites (access, permissions, tools), expected duration, frequency, and SLA/SLO impact.

Step 2: Write Procedure

For each step: number and title, exact command or action in code blocks, expected output, failure indicators, decision points (if-then branches), and wait conditions.

Step 3: Add Troubleshooting

For common problems: symptom, likely cause, resolution steps, and when to stop troubleshooting and escalate.

Step 4: Define Rollback

Trigger conditions (specific error thresholds), ordered reversal steps, verification checks, and data considerations.

Step 5: Add Metadata

Owner, last tested date, escalation contacts by tier, related runbooks, and change history.

Output Format

## Runbook: [Task Name]
**ID**: RB-[NNN] | **Owner**: [Name] | **Last Tested**: [YYYY-MM-DD]
**Trigger**: [What initiates this] | **Duration**: [Est.] | **Frequency**: [How often]

### Prerequisites
- [ ] Access to [system] with [role]
- [ ] [Tool] installed

### Procedure

**Step 1: [Action]**

[exact command]

- Expected: [What you should see]
- If fails: [What to do]

**Step 2: [Action]**

[exact command]

- Decision: If [condition], go to Step [X].

### Verification
- [ ] [Confirmation check 1]
- [ ] [Confirmation check 2]

### Troubleshooting
| Symptom | Likely Cause | Resolution |
|---------|-------------|------------|
| [What you see] | [Why] | [Fix] |

### Rollback
**Trigger**: [Conditions] | **Duration**: [Est.]
1. [Rollback step]
2. Verify: [Confirmation]

### Escalation
| Tier | Contact | When to Escalate |
|------|---------|------------------|
| 1 | [Name/Channel] | [Condition] |

Examples

User: "Create a runbook for database failover to the replica" Action: Document trigger conditions, pre-failover checks (replica lag, connections), promotion commands, connection string updates, verification queries, and rollback to original primary.

User: "On-call runbook for high memory alerts on production" Action: Document alert identification, initial diagnostics (process, memory profile, recent deploys), remediation options (restart, scale, clear cache), root cause steps, and escalation criteria.

Best Practices

• Write commands that can be copy-pasted with all flags and parameters.
• Include expected output for every step so operators know they are on track.
• Test by having someone unfamiliar follow the runbook. If they get stuck, add detail.
• Include timing expectations ("takes approximately 5 minutes") to distinguish "running" from "broken."
• Add decision points explicitly with if/then branches.
• Version the runbook and record when it was last tested.
• Store where on-call can access during an incident (not in a system that might be down).