CI triage

Red CI has exactly three honest outcomes: infrastructure noise (rerun), a broken workflow (fix YAML), or a real failure (fix code). Most wasted effort comes from guessing wrong — "rerun a broken workflow five times", or "debug a flaky registry timeout as if it were your code." This skill forces the classification step before any fixing.

Install the CLI:

pip install ci-autopsy

1. Pull the failure, not the whole log

Never read a 40 MB log in one scroll. Get only the failed jobs:

gh run view 123456789 --log-failed        # human view; big runs can be slow
autopsy run https://github.com/owner/repo/actions/runs/123456789   # classified summary

autopsy run fetches every failed job's log through the authenticated gh (no tokens of its own), classifies each, prints the verdict with evidence lines, extracted pytest summaries (failing tests, counts, first assertion) and the next action.

Use ci-autopsy 0.1.1+ and a current gh for live reads. All job pages are fetched, so large matrices are not truncated. For a historical failure, preserve /attempts/N in the supplied URL: the CLI reads that attempt's metadata and jobs. Without it, the metadata's latest attempt is selected and pinned for log retrieval. Check the printed attempt before drawing conclusions, and inspect the current attempt before rerunning a historical failure. run never starts a rerun itself.

If no failed-job logs are found, read the reported status/conclusion. An in-progress run is incomplete; a workflow-level failure can have no jobs. Neither proves a passing build. Retrieval errors exit 2; successful analysis exits 0 even when it describes failures. Do not use that exit code as a CI quality gate.

If the run failed at the workflow level (no job ever started — invalid YAML), there are no job logs. Read the annotations instead:

gh api repos/{owner}/{repo}/actions/runs/{id}/attempts/1 --jq '.workflow_annotations'

For saved logs or logs from other CI systems, autopsy classify file.log [file2.log ...] does the same classification offline. --json for machine-readable output.

2. Read the verdict

== Test (ubuntu-latest, 3.13) (failure)
   pytest    2 failed, 431 passed, 1 skipped
   failed    tests/test_pricing.py::test_discount_applied
   first     AssertionError: assert 90.0 == 85.0
   verdict   test-failure
   evidence  L3: FAILED tests/test_pricing.py::test_discount_applied - ...
   action    Deterministic-looking test failures. This is the code; reproduce locally, not more triage.

The categories, in the order that wins when several hit one log:

Priority is deliberate: a timed-out job ends with a "cancelled" line, so timeout outranks cancel — but a plain concurrency cancel still outranks test failures visible in its log, because those tests never finished.

3. Match the action to the verdict

• config — fix YAML. actionlint on the file catches most of this class before push.
• infra, dependency — rerun failed jobs only: gh run rerun <id> --failed. Never rerun the whole run when one matrix leg failed; the other legs already told you they're fine.
• When using a URL for another repository, include --repo owner/repo: gh run rerun <id> --repo owner/repo --failed. The repository is a flag, not a positional argument. A suggested rerun still needs to match the current failure and the user's requested scope.
• cancel — nothing is broken. Don't "fix" anything.
• timeout, oom — measure locally, then either fix the slowness/leak or raise the specific limit with a comment saying why.
• test-failure — reproduce locally before editing. If it doesn't reproduce locally, it was flaky-suspect all along: proceed to step 4.
• flaky-suspect — proceed to step 4. Do not "fix" code before confirming the flake.

4. Confirm a flake before fixing it

A flake fixits worth nothing unless you can show the flake. Minimal loop, pytest:

# order dependence: fixed seed first, then single test repeatedly
pytest tests/test_flaky.py -p no:randomly
for i in $(seq 1 20); do pytest tests/test_flaky.py -q --randomly-seed=$i | tail -1; done

# resource/timing: hammer it
pytest tests/test_flaky.py --count=20 -x    # needs pytest-repeat

If it fails under some seeds but passes alone, you have order dependence; if it fails standalone but passes in the full suite, you have isolation leakage (a fixture or global state another test "helpfully" sets). The full methodology (fixtures, sockets, time, seeds, quarantine with a TODO and an expiry date): references/flaky-tests.md.

For the narrow port/server-start class, check: does the test bind a fixed port instead of port 0 / an ephemeral one? Is there a sleep waiting for a server instead of polling readiness? Those two are most of the genre.

5. Teach autopsy your project's noise

Every project has its own recurring failures (a cancel-during-mergequeue, a known-flaky vendor health check). Instead of re-triaging them from scratch, keep a rule file in the repo:

autopsy rules > .github/autopsy-rules.toml   # start from the built-in registry
# add a [[category]] block:
#   name = "corp-proxy"
#   "corp-proxy" = ["our proxy is down again"]
autopsy classify log.txt --rules .github/autopsy-rules.toml            # extend
autopsy classify log.txt --rules .github/autopsy-rules.toml --replace # replace entirely

Rules are data (regexes in TOML), not code — commit them to the repo, review them in PRs. Keep the built-ins unless a category is provably wrong for your project.

6. Report

End every triage with the three-line summary a human needs:

verdict:  dependency (registry 429), not a code failure
evidence: L41 "HTTP 429 Too Many Requests for URL https://pypi.org/simple/..."
done:     rerun failed jobs (gh run rerun 123456789 --failed); caching added in #123 to prevent repeats

If the verdict was unknown or ambiguous, say so honestly — an honest "I don't know, read lines 200-260" beats a confidently wrong verdict.

Scope

This skill is for triaging failures in your own repositories and CI. Don't run it against systems you don't have authorization to inspect, and don't use triage tooling to hide failures you know are real (quarantining a test requires a linked issue and an expiry date, never a silent skip).