Operations

Use this skill for platform-neutral server and service operations. It defines the workflow and record formats; environment-specific skills and tools own Linux, SSH, containers, Kubernetes, IaC, cloud, and observability commands.

Destructive service operations

When an operation can stop or replace the process performing the operation, schedule it through an external supervisor or transient unit rather than running it synchronously in the service cgroup. Persist a unique operation id and an explicit planned → scheduled phase before dispatch. Recovery must use the idempotency record to verify the target, not replay the destructive command. Never substitute nohup, &, or sleep for a durable lifecycle boundary.

Core rules

1. Identify the exact environment, hosts, services, source of truth, and operator request.
2. Determine the authorization level before touching an external system:
   • L0: local inventory and documentation only.
   • L1: read-only checks within an explicitly confirmed scope.
   • L2: reversible changes, requiring per-action approval unless covered by a scoped, expiring, revocable runbook authorization.
   • L3: disruptive, privileged, network, IAM, DNS, data, secret, bulk, or irreversible work, requiring confirmation of the exact action.
3. Prefer read-only discovery and preflight checks. Do not infer repair authority from diagnosis or urgency.
4. Before a change, state targets, expected effect, risk, commands or API actions, success criteria, rollback trigger, and rollback steps.
5. Use a canary and bounded batches for multi-resource work. Verify each batch before proceeding.
6. Record actual actions, sanitized evidence, results, and deviations.
7. Verify service health and intended state after the action. Roll back only when authorized or when a confirmed runbook explicitly permits it.

Stop and ask when target identity, current state, authority, impact, rollback, or verification is unclear.

Inventory layout

Create only the files needed for the current environment:

work/
  inventory/
    sources.md
    hosts/<host-id>/info.yaml
    hosts/<host-id>/operations.md
    services/<service-id>/info.yaml
    services/<service-id>/operations.md
  runbooks/
  inspections/
  operations/YYYY/YYYY-MM/
  incidents/

sources.md records each authoritative source, owner, refresh method, and last confirmation. Prefer external sources of truth. AgentHome inventory is a cache unless the operator explicitly designates it authoritative.

Use stable IDs and schema_version: 1. A host info.yaml should contain:

schema_version: 1
id: prod-web-01
display_name: Production Web 01
environment: production
status: active
owners: [platform]
criticality: high
location: {provider: example-cloud, region: us-east-1}
roles: [web]
access:
  ssh_config_alias: prod-web-01
  bastion_alias: prod-bastion
  credential_ref: secret-manager://ops/prod-web
platform: {os: ubuntu, architecture: amd64}
services: [customer-web]
monitoring: {dashboards: [], alerts: []}
source:
  kind: cmdb
  ref: host/prod-web-01
  last_synced_at: 2026-09-06T00:00:00Z
last_confirmed_at: 2026-09-06T00:00:00Z

A service info.yaml should contain:

schema_version: 1
id: customer-web
display_name: Customer Web
environment: production
status: active
owners: [web-team]
criticality: high
service_type: systemd
runs_on: {hosts: [prod-web-01, prod-web-02]}
dependencies: [customer-api]
repository: github:example/customer-web
deployment_source: gitops:environments/prod/customer-web
runbook: work/runbooks/customer-web.md
monitoring: {dashboards: [], alerts: []}
backup: {policy_ref: backup/customer-web}
maintenance:
  timezone: America/New_York
  windows: []
source:
  kind: gitops
  ref: services/customer-web
  last_synced_at: 2026-09-06T00:00:00Z
last_confirmed_at: 2026-09-06T00:00:00Z

Never store a credential value. credential_ref may contain only a controlled secret-manager reference or local connection alias. Keep permission policies separate from resource facts so an inventory edit cannot expand authority.

When renaming or retiring a resource, preserve its stable ID through a redirect or tombstone so historical links remain valid.

Operation records

Create one authoritative record per logical operation:

work/operations/YYYY/YYYY-MM/OP-<UTC timestamp>-<slug>.md

Record:

• UTC time and operator-local time with IANA timezone
• WorkItem, incident, change, or request reference
• actor, target environment, host IDs, and service IDs
• request and authorization summary, including standing authorization scope
• purpose, risk level, preflight results, plan, and rollback plan
• actual commands or API actions, with secrets and sensitive output redacted
• exit status, changed resources, verification evidence, and final result
• rollback status, residual risk, follow-up owner, and due condition

Do not silently rewrite an error. Append a dated correction. Put large raw output in a controlled external log or attachment and retain only a summary and reference.

Append a concise link to each affected resource's operations.md:

## 2026-09

- `2026-09-06T10:30:00Z` · change · success ·
  [OP-20260906T103000Z-restart-customer-web](../../../operations/2026/2026-09/OP-20260906T103000Z-restart-customer-web.md)
  — Approved rolling restart; health checks passed.

The resource timeline is an index, not a duplicate operation log.

Incidents

Use work/incidents/INC-<UTC timestamp>-<slug>.md for a material anomaly. Record detection, impact, affected resources, evidence, timeline, communications, mitigations, current state, owner, and next update. Keep facts separate from hypotheses. Incident creation and notification do not authorize remediation.

Scheduled inspections

Scheduled inspection is off by default. Before creating a timer or recurring job, confirm:

• scope and exclusions
• IANA timezone and frequency or wall-clock time
• allowed checks, concurrency, and timeouts
• reporting and anomaly-notification behavior
• maintenance and silence windows
• report-only versus explicitly named remediation runbooks
• review date, end date, and pause/revocation conditions

Persist the confirmed policy in work/inspections/policy.yaml. Reconfirm any change to scope, schedule, notification, or remediation authority. Give every run its own tracked lifecycle and dated inspection record. The default for an anomaly is to report and open an incident, not to repair it.