docs-workflow-pipeline-diagnostics
Skill4 starsChanged 4 months agoApache-2.0View on GitHub
Diagnose a docs-orchestrator pipeline run for failures, bottlenecks, and context-pressure risks. Runs the diagnostics script, drills into failures, and writes a diagnostic report with actionable recommendations.
In this repository53 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- action-commentsSkill
- article-extractorSkill
- docs-convert-gdoc-mdSkill
- docs-orchestratorSkill
- docs-review-content-qualitySkill
- docs-review-modular-docsSkill
- docs-review-securitySkill
- docs-review-styleSkill
- docs-review-technicalSkill
- docs-ticket-readinessSkill
- docs-workflow-code-analysisSkill
- docs-workflow-create-jiraSkill
- docs-workflow-create-merge-requestSkill
- docs-workflow-jira-readySkill
- docs-workflow-planningSkill
- docs-workflow-pr-analysisSkill
- docs-workflow-quality-gateSkill
- docs-workflow-requirementsSkill
- docs-workflow-scope-req-auditSkill
- docs-workflow-security-reviewSkill
- docs-workflow-startSkill
- docs-workflow-style-reviewSkill
- docs-workflow-tech-reviewSkill
- docs-workflow-writingSkill
- git-pr-readerSkill
- ibm-sg-audience-and-mediumSkill
- ibm-sg-language-and-grammarSkill
- ibm-sg-legal-informationSkill
- ibm-sg-numbers-and-measurementSkill
- ibm-sg-punctuationSkill
- ibm-sg-referencesSkill
- ibm-sg-structure-and-formatSkill
- ibm-sg-technical-elementsSkill
- jira-readerSkill
- jira-writerSkill
- learn-codeSkill
- lint-with-valeSkill
- mdita-writeSkill
- query-codeSkill
- redhat-docs-tocSkill
- rh-ssg-accessibilitySkill
- rh-ssg-formattingSkill
- rh-ssg-grammar-and-languageSkill
- rh-ssg-gui-and-linksSkill
- rh-ssg-legal-and-supportSkill
- rh-ssg-release-notesSkill
- rh-ssg-structureSkill
- rh-ssg-technical-examplesSkill
- rn-known-issuesSkill
- understand-pull-requestSkill
- validate-plugin-prereqsSkill
The file
We split this skill into the rules it gives an agent
9 rules. About 0 are common, 4 are distinctive, and 5 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈4
Distinctive · estimate
This author's own choices. Worth a look.
5
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 59
If a direct progress file path is known, use --progress-file instead:
- Distinctive: line 72
Check the exit code. If the script failed, write an error to the report and exit.
- Very specific: line 76
Read $DIAGNOSTICS_FILE. The script produces structured JSON with these sections:
- Very specific: line 102
If --ci-log was provided, also analyze the session log:
- Distinctive: line 110
Check for: error patterns, session aborts, context compaction markers, stop hook blocks.
- Distinctive: line 130
Two checks the script cannot compute — add them yourself: only when the inputs are available, appending to the health list you carry into …
- Very specific: line 143
Write $REPORT_FILE using this template:
- Distinctive: line 198
Verify the file exists and contains the required fields:
- Very specific: line 205
If the file is missing or malformed, the script did not emit it — re-run step 2 with --emit-sidecar.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Pipeline Diagnostics Step
Step skill for the docs-orchestrator pipeline. Follows the step skill contract: parse args → run diagnostics script → analyze results → write output.
Arguments
$1— JIRA ticket ID (required)--base-path <path>— Base output path (e.g.,.agent_workspace/proj-123)--ci-log <path>— Path to a CI session log (optional). Enables CI-specific analysis
Input
Reads the progress file and step-result sidecars from all upstream steps:
<base-path>/workflow/*.json
<base-path>/*/step-result.jsonOutput
<base-path>/pipeline-diagnostics/report.md
<base-path>/pipeline-diagnostics/diagnostics.json
<base-path>/pipeline-diagnostics/step-result.jsonExecution
1. Parse arguments
Extract the ticket ID, --base-path, and optional --ci-log from the args string.
Set the paths:
OUTPUT_DIR="${BASE_PATH}/pipeline-diagnostics"
REPORT_FILE="${OUTPUT_DIR}/report.md"
DIAGNOSTICS_FILE="${OUTPUT_DIR}/diagnostics.json"
mkdir -p "$OUTPUT_DIR"2. Run the diagnostics script
python3 ${CLAUDE_SKILL_DIR}/scripts/pipeline_diagnostics.py <TICKET> \
--workspace "$(dirname "${BASE_PATH}")" \
--format json \
--emit-sidecar "${OUTPUT_DIR}/step-result.json" > "$DIAGNOSTICS_FILE"If a direct progress file path is known, use --progress-file instead:
python3 ${CLAUDE_SKILL_DIR}/scripts/pipeline_diagnostics.py \
--progress-file <path-to-progress-json> \
--format json \
--emit-sidecar "${OUTPUT_DIR}/step-result.json" > "$DIAGNOSTICS_FILE"--emit-sidecar writes the schema-conformant step-result.json directly from the computed
analysis — every field is derived, and completed_at is a real wall-clock timestamp. This is the
authoritative sidecar; do not hand-author it later (see step 8).
Check the exit code. If the script failed, write an error to the report and exit.
3. Read and analyze the diagnostics output
Read $DIAGNOSTICS_FILE. The script produces structured JSON with these sections:
- summary — ticket, workflow type, status, total duration
- context_pressure — risk level, score, contributing factors
- failures — failed steps, missing outputs, missing sidecars, quality issues
- bottlenecks — steps that took disproportionately long
- orchestrator_health — self-introspection issues (computed by the script; see step 6)
- recommendations — actionable next steps
4. Drill into failures
For each failure in the diagnostics output, examine the root cause:
| Failure type | Where to look |
step_failed | Re-read the step's output folder for error messages. Check if an upstream dependency (inputs) was not met |
missing_output | The step was marked completed but its output folder is gone. Likely manual deletion or filesystem issue |
missing_sidecar | Step completed without writing step-result.json. May indicate context compaction lost the sidecar-write instruction |
step_deferred | Upstream when condition was never resolved. Check if source resolution failed |
low_confidence | Read technical-review/review.md for specific issues |
quality_gate_low | Read quality-gate/judge-results.md for the judge's rationale |
empty_plan | Requirements may be too vague or code-analysis found nothing relevant |
no_files_written | Check if the plan was empty or if the writer agent failed |
5. CI log analysis (optional)
If --ci-log was provided, also analyze the session log:
rg -n 'ERROR:|FAILED|Traceback|exit code [1-9]' <ci-log-path>
tail -20 <ci-log-path>
rg -n 'compact|context.*limit|token.*limit' <ci-log-path>Check for: error patterns, session aborts, context compaction markers, stop hook blocks.
6. Orchestrator self-introspection
Self-introspection covers orchestrator-level problems — not content quality, but the
docs-orchestrator machinery itself. The script already computes the deterministic checks and
returns them in the orchestrator_health array of $DIAGNOSTICS_FILE. Read that array; do NOT
re-derive these checks by hand:
Check (check field) | Severity |
schema_drift — required progress fields missing/null, or no resolvable workflow type | high / medium |
missing_sidecar — step is completed but wrote no step-result.json | high |
null_result — step is completed but .steps[name].result is null | medium |
stuck_in_progress — step never left in_progress | high |
deferred_unresolved — a deferred step remained deferred at workflow end | medium |
workarounds_applied — the progress file's workarounds array is non-empty | medium |
active_marker_left — .agent_workspace/.active-workflow still exists after status=completed | low |
timestamp_gap — > 10 min elapsed before a step completed | low |
Two checks the script cannot compute — add them yourself only when the inputs are available, appending to the health list you carry into the report:
| Check | How to detect | Severity |
| Step order vs YAML mismatch | Compare the progress step_order against the workflow YAML's step list. Missing or extra entries indicate manual edits or schema rot | medium |
| Hook errors during run | If --ci-log was provided, grep for Stop hook error: or hook.*error lines | high |
The combined list (script orchestrator_health + any of the two checks above) is what you tabulate
in the report. The sidecar's orchestrator_issue_count reflects only the script-computed subset.
7. Write the diagnostic report
Write $REPORT_FILE using this template:
# Pipeline Diagnostic Report: <TICKET>
## Run summary
- **Status**: completed | failed | in_progress
- **Duration**: N minutes
- **Steps**: N/M completed
- **Context pressure**: LOW | MODERATE | HIGH | CRITICAL (score N)
## Failures
<!-- For each failure -->
### <step-name>: <failure type>
- **Root cause**: ...
- **Fix**: ...
## Workarounds
<!-- For each workaround in progress file -->
- **<step>**: <issue> — Orchestrator action: <action>
<!-- If no workarounds: -->
No script workarounds were applied.
## Bottlenecks
<!-- For each bottleneck -->
- **<step>**: N min (Nx average) — <mitigation>
## Context pressure
- **Level**: ... (score N)
- **Risk factors**: ...
- **Symptoms observed**: ...
## Orchestrator health
<!-- One row per entry in the combined health list (script orchestrator_health + step-6 additions) -->
| Step | Check | Severity | Detail |
|---|---|---|---|
| — | active_marker_left | low | `.active-workflow` still present after status=completed |
<!-- If no problems found: -->
No orchestrator issues detected.
## Recommendations
1. ...
2. ...8. Verify the step-result.json sidecar
The --emit-sidecar flag in step 2 already wrote ${OUTPUT_DIR}/step-result.json from the
computed analysis, with every field derived and a real wall-clock completed_at. Do not
hand-author or overwrite it — a hand-written sidecar drifts from the schema.
Verify the file exists and contains the required fields:
jq -e '.schema_version and .pipeline_status and (.orchestrator_issue_count != null)' \
"${OUTPUT_DIR}/step-result.json" >/dev/null || echo "ERROR: sidecar missing or malformed"If the file is missing or malformed, the script did not emit it — re-run step 2 with
--emit-sidecar. Do not substitute a stub.
The sidecar fields, all populated by the script:
| Field | Source |
pipeline_status | summary.status |
context_pressure_level / context_pressure_score | context_pressure |
failure_count / high_severity_failure_count | failures |
bottleneck_count | bottlenecks |
orchestrator_issue_count | orchestrator_health (script-computed subset) |
workaround_count | workarounds |
recommendation_count | recommendations |
total_duration_min | summary.total_duration_min |
Context pressure reference
See context-pressure.md for the full context pressure detection model, risk score calculation, and mitigation strategies.
| Level | Score | Meaning |
| low | 0–2 | Normal operation. No intervention needed |
| moderate | 3–5 | Some context accumulation. Monitor for compaction behavior |
| high | 6–8 | Significant risk of compaction-related issues. Consider workflow splitting |
| critical | 9+ | Near-certain compaction will occur. Split the workflow or reduce step count |
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

