agentleFS
Sign inStart free
dmzoneilldmzoneill/skills/debug-prod/SKILL.md

debug-prod

Skill2 starsChanged 19 days agoApache-2.0View on GitHub

Systematic production investigation - pods, logs, metrics, alerts, deployments, Kibana, Slack context. Suggests likely causes from patterns. Use when user says "debug prod", "investigate production issue", or escalated from investigate_alert.

1 other project uses a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

5 rules. About 0 are common, 4 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈4

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Use --kubeconfig=~/.kube/config.p for prod.

    line 19
  2. Distinctive:

    If no namespace: ask user (main vs billing)

    line 42
  3. Distinctive:

    If alert_name: search app-interface alert YAML for definition

    line 67
  4. Distinctive:

    Check app-interface CICD config for deployed SHA

    line 90
  5. Distinctive:

    Check namespace config from app-interface

    line 91

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Debug Production

Deep investigation of production issues in Automation Analytics. Gathers pod status, logs, metrics, alerts, recent deployments, and suggests likely causes.

Kubeconfig Rules

NEVER copy kubeconfig files.

FileEnvironment
~/.kube/config.sStage
~/.kube/config.pProduction

Use --kubeconfig=~/.kube/config.p for prod.

Inputs

InputTypeDefaultPurpose
namespacestring-main or billing (ask if missing)
alert_namestring-Prometheus alert if triggered by alert
pod_filterstring-Filter pods (e.g., fastapi, processor)
time_rangestring1hLog search window (15m, 1h, 6h, 24h)
issue_keystring-Jira key to attach session context

Persona

Load incident persona (prometheus, alertmanager, kibana, k8s, grafana).

Workflow

1. Bootstrap

  • persona_load("incident")
  • check_known_issues("kubectl_get_pods")
  • check_known_issues("kibana_search_logs")
  • If no namespace: ask user (main vs billing)

2. Load Patterns

  • memory_read("learned/patterns") → error_patterns

3. Pod Status

  • kubectl_get_pods(namespace, environment="production")
  • kubectl_top_pods(namespace, environment="production")
  • Parse: CrashLoopBackOff, OOMKilled, Error, Pending, ImagePullBackOff, high restarts

4. Events

  • kubectl_get_events(namespace, environment="production")
  • Filter: warning, error, failed, killed, backoff, unhealthy

5. Pod Logs

  • For unhealthy pods (up to 3): kubectl_describe_pod, kubectl_logs(pod, namespace, tail=500, since=time_range)
  • Extract error/exception/traceback lines

6. Alerts

  • alertmanager_alerts(environment="production")
  • If alert_name: search app-interface alert YAML for definition

7. Kibana Logs

  • kibana_search_logs(query="error OR exception OR traceback", namespace, environment="production", limit=20)
  • kibana_get_errors(namespace, environment="production")
  • kibana_trace_request — if request_id in summary

8. Slack Context

  • persona_load("slack") → slack_find_channel, slack_list_messages (team-automation-analytics)
  • persona_load("incident") — restore

9. Metrics

  • prometheus_query — 5xx error rate
  • prometheus_query_range — error rate trend, memory trend
  • prometheus_pod_health, prometheus_rules
  • grafana_dashboard_list, grafana_dashboard_get, prometheus_grafana_link

10. Deployments

  • kubectl_get_deployments, kubectl_get(resource="replicasets")
  • Check app-interface CICD config for deployed SHA
  • Check namespace config from app-interface

11. Knowledge & Code Search

  • code_search(query=error_from_logs, project="automation-analytics-backend")
  • knowledge_query(project="automation-analytics-backend", persona="devops", section="patterns.deployment")
  • knowledge_query(..., section="gotchas")

12. Match Patterns

Match pod issues against known_patterns. Add generic suggestions: OOMKilled, CrashLoopBackOff, ImagePullBackOff.

13. Attach to Jira

  • If issue_key: jira_attach_session(issue_key, include_transcript=true)

14. Log & Update

  • memory_session_log("Debugged production", ...)
  • Update state/environments

15. Failure Recovery

  • k8s "unauthorized" → kube_login(cluster="prod")
  • "no route to host" → vpn_connect()
  • Prometheus "connection refused" → VPN/auth
  • Kibana 401 → check credentials
  • learn_tool_fix(...) for patterns

Output

Report: pod status, resource usage, events, error logs, Kibana errors, metrics, deployments, pattern matches, Grafana/Kibana links, suggested Prometheus queries.

Chains To

  • hotfix, rollout_restart, scale_deployment
  • silence_alert, create_jira_issue, learn_pattern

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

Reviews can't be read right now.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free