agentleFS
opendatahub-ioopendatahub-io/docs-skills/skills/docs-workflow-pipeline-diagnostics/SKILL.md

docs-workflow-pipeline-diagnostics

Skill4 starsChanged 4 months agoApache-2.0View on GitHub

Diagnose a docs-orchestrator pipeline run for failures, bottlenecks, and context-pressure risks. Runs the diagnostics script, drills into failures, and writes a diagnostic report with actionable recommendations.

The file

We split this skill into the rules it gives an agent

9 rules. About 0 are common, 4 are distinctive, and 5 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈4

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 5

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    If a direct progress file path is known, use --progress-file instead:

    line 59
  2. Distinctive:

    Check the exit code. If the script failed, write an error to the report and exit.

    line 72
  3. Very specific:

    Read $DIAGNOSTICS_FILE. The script produces structured JSON with these sections:

    line 76
  4. Very specific:

    If --ci-log was provided, also analyze the session log:

    line 102
  5. Distinctive:

    Check for: error patterns, session aborts, context compaction markers, stop hook blocks.

    line 110
  6. Distinctive:

    Two checks the script cannot compute — add them yourself: only when the inputs are available, appending to the health list you carry into …

    line 130
  7. Very specific:

    Write $REPORT_FILE using this template:

    line 143
  8. Distinctive:

    Verify the file exists and contains the required fields:

    line 198
  9. Very specific:

    If the file is missing or malformed, the script did not emit it — re-run step 2 with --emit-sidecar.

    line 205

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Pipeline Diagnostics Step

Step skill for the docs-orchestrator pipeline. Follows the step skill contract: parse args → run diagnostics script → analyze results → write output.

Arguments

  • $1 — JIRA ticket ID (required)
  • --base-path <path> — Base output path (e.g., .agent_workspace/proj-123)
  • --ci-log <path> — Path to a CI session log (optional). Enables CI-specific analysis

Input

Reads the progress file and step-result sidecars from all upstream steps:

<base-path>/workflow/*.json
<base-path>/*/step-result.json

Output

<base-path>/pipeline-diagnostics/report.md
<base-path>/pipeline-diagnostics/diagnostics.json
<base-path>/pipeline-diagnostics/step-result.json

Execution

1. Parse arguments

Extract the ticket ID, --base-path, and optional --ci-log from the args string.

Set the paths:

OUTPUT_DIR="${BASE_PATH}/pipeline-diagnostics"
REPORT_FILE="${OUTPUT_DIR}/report.md"
DIAGNOSTICS_FILE="${OUTPUT_DIR}/diagnostics.json"
mkdir -p "$OUTPUT_DIR"

2. Run the diagnostics script

python3 ${CLAUDE_SKILL_DIR}/scripts/pipeline_diagnostics.py <TICKET> \
  --workspace "$(dirname "${BASE_PATH}")" \
  --format json \
  --emit-sidecar "${OUTPUT_DIR}/step-result.json" > "$DIAGNOSTICS_FILE"

If a direct progress file path is known, use --progress-file instead:

python3 ${CLAUDE_SKILL_DIR}/scripts/pipeline_diagnostics.py \
  --progress-file <path-to-progress-json> \
  --format json \
  --emit-sidecar "${OUTPUT_DIR}/step-result.json" > "$DIAGNOSTICS_FILE"

--emit-sidecar writes the schema-conformant step-result.json directly from the computed analysis — every field is derived, and completed_at is a real wall-clock timestamp. This is the authoritative sidecar; do not hand-author it later (see step 8).

Check the exit code. If the script failed, write an error to the report and exit.

3. Read and analyze the diagnostics output

Read $DIAGNOSTICS_FILE. The script produces structured JSON with these sections:

  1. summary — ticket, workflow type, status, total duration
  2. context_pressure — risk level, score, contributing factors
  3. failures — failed steps, missing outputs, missing sidecars, quality issues
  4. bottlenecks — steps that took disproportionately long
  5. orchestrator_health — self-introspection issues (computed by the script; see step 6)
  6. recommendations — actionable next steps

4. Drill into failures

For each failure in the diagnostics output, examine the root cause:

Failure typeWhere to look
step_failedRe-read the step's output folder for error messages. Check if an upstream dependency (inputs) was not met
missing_outputThe step was marked completed but its output folder is gone. Likely manual deletion or filesystem issue
missing_sidecarStep completed without writing step-result.json. May indicate context compaction lost the sidecar-write instruction
step_deferredUpstream when condition was never resolved. Check if source resolution failed
low_confidenceRead technical-review/review.md for specific issues
quality_gate_lowRead quality-gate/judge-results.md for the judge's rationale
empty_planRequirements may be too vague or code-analysis found nothing relevant
no_files_writtenCheck if the plan was empty or if the writer agent failed

5. CI log analysis (optional)

If --ci-log was provided, also analyze the session log:

rg -n 'ERROR:|FAILED|Traceback|exit code [1-9]' <ci-log-path>
tail -20 <ci-log-path>
rg -n 'compact|context.*limit|token.*limit' <ci-log-path>

Check for: error patterns, session aborts, context compaction markers, stop hook blocks.

6. Orchestrator self-introspection

Self-introspection covers orchestrator-level problems — not content quality, but the docs-orchestrator machinery itself. The script already computes the deterministic checks and returns them in the orchestrator_health array of $DIAGNOSTICS_FILE. Read that array; do NOT re-derive these checks by hand:

Check (check field)Severity
schema_drift — required progress fields missing/null, or no resolvable workflow typehigh / medium
missing_sidecar — step is completed but wrote no step-result.jsonhigh
null_result — step is completed but .steps[name].result is nullmedium
stuck_in_progress — step never left in_progresshigh
deferred_unresolved — a deferred step remained deferred at workflow endmedium
workarounds_applied — the progress file's workarounds array is non-emptymedium
active_marker_left — .agent_workspace/.active-workflow still exists after status=completedlow
timestamp_gap — > 10 min elapsed before a step completedlow

Two checks the script cannot compute — add them yourself only when the inputs are available, appending to the health list you carry into the report:

CheckHow to detectSeverity
Step order vs YAML mismatchCompare the progress step_order against the workflow YAML's step list. Missing or extra entries indicate manual edits or schema rotmedium
Hook errors during runIf --ci-log was provided, grep for Stop hook error: or hook.*error lineshigh

The combined list (script orchestrator_health + any of the two checks above) is what you tabulate in the report. The sidecar's orchestrator_issue_count reflects only the script-computed subset.

7. Write the diagnostic report

Write $REPORT_FILE using this template:

# Pipeline Diagnostic Report: <TICKET>

## Run summary
- **Status**: completed | failed | in_progress
- **Duration**: N minutes
- **Steps**: N/M completed
- **Context pressure**: LOW | MODERATE | HIGH | CRITICAL (score N)

## Failures
<!-- For each failure -->
### <step-name>: <failure type>
- **Root cause**: ...
- **Fix**: ...

## Workarounds
<!-- For each workaround in progress file -->
- **<step>**: <issue> — Orchestrator action: <action>

<!-- If no workarounds: -->
No script workarounds were applied.

## Bottlenecks
<!-- For each bottleneck -->
- **<step>**: N min (Nx average) — <mitigation>

## Context pressure
- **Level**: ... (score N)
- **Risk factors**: ...
- **Symptoms observed**: ...

## Orchestrator health

<!-- One row per entry in the combined health list (script orchestrator_health + step-6 additions) -->

| Step | Check | Severity | Detail |
|---|---|---|---|
| — | active_marker_left | low | `.active-workflow` still present after status=completed |

<!-- If no problems found: -->
No orchestrator issues detected.

## Recommendations
1. ...
2. ...

8. Verify the step-result.json sidecar

The --emit-sidecar flag in step 2 already wrote ${OUTPUT_DIR}/step-result.json from the computed analysis, with every field derived and a real wall-clock completed_at. Do not hand-author or overwrite it — a hand-written sidecar drifts from the schema.

Verify the file exists and contains the required fields:

jq -e '.schema_version and .pipeline_status and (.orchestrator_issue_count != null)' \
  "${OUTPUT_DIR}/step-result.json" >/dev/null || echo "ERROR: sidecar missing or malformed"

If the file is missing or malformed, the script did not emit it — re-run step 2 with --emit-sidecar. Do not substitute a stub.

The sidecar fields, all populated by the script:

FieldSource
pipeline_statussummary.status
context_pressure_level / context_pressure_scorecontext_pressure
failure_count / high_severity_failure_countfailures
bottleneck_countbottlenecks
orchestrator_issue_countorchestrator_health (script-computed subset)
workaround_countworkarounds
recommendation_countrecommendations
total_duration_minsummary.total_duration_min

Context pressure reference

See context-pressure.md for the full context pressure detection model, risk score calculation, and mitigation strategies.

LevelScoreMeaning
low0–2Normal operation. No intervention needed
moderate3–5Some context accumulation. Monitor for compaction behavior
high6–8Significant risk of compaction-related issues. Consider workflow splitting
critical9+Near-certain compaction will occur. Split the workflow or reduce step count

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.