agentleFS
intelintel/torch-xpu-ops/.claude/skills/xpu-ci-health-check/SKILL.md

xpu-ci-health-check

Skill122 starsChanged 5 months agoApache-2.0View on GitHub

Check PyTorch ciflow/xpu (xpu.yml) on the main branch, collect the failing XPU test cases from the most recent completed run(s), analyze the ROOT CAUSE of each failure with AI, and produce a list with a prefilled "disable issue" link per case. USE WHEN the user asks to check XPU CI health, find failing XPU tests, or generate XPU disable-issue drafts.

The file

We split this skill into the rules it gives an agent

7 rules. About 0 are common, 3 are distinctive, and 4 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈3

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Run the bundled collection script from the PyTorch repository root.

    line 32
  2. Very specific:

    If the number of failed cases > 10, skip this step and let the root cause field be empty.

    line 83
  3. Very specific:

    Identify the real failure: Read error_excerpt and identify the failing frame and exception type (e.g., InductorError: NotImplementedError: …

    line 87
  4. Distinctive:

    Use available history (recent runs, torch-ci failure history, prior reports).

    line 101
  5. Very specific:

    IMPORTANT: Use issue_url exactly as produced by the script — do not truncate.

    line 114
  6. Distinctive:

    After the list, add a one-line health summary:

    line 119
  7. Distinctive:

    Zero failures: If the script returns no cases, report that main is green for the inspected run(s).

    line 127

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

XPU CI Health Check Skill

This skill automates XPU CI health checks with the following workflow:

  1. Collect failure evidence automatically by running the bundled script.
  2. Analyze the root cause of each case yourself (AI), using the traceback evidence plus the suspect commit/PR — never copy the raw exception as the conclusion.
  3. Render the result generate a list with one row per failing case.

Root cause analysis is YOUR job, not the script's. The script only gathers evidence (case id, traceback excerpt, commit sha, disable link). You must reason about why it failed and which change likely caused it.

Prerequisites

  • GitHub token: Optional. The script uses the gh CLI if --token is not provided; ensure gh auth login is configured.

Step 1 — Collect evidence (always run this first)

Run the bundled collection script from the PyTorch repository root. The script is located at <path/to/skills>/xpu-ci-health-check/scripts/collect_failures.py (included in this skill).

# Run from PyTorch repository root
cd /path/to/pytorch  # or your PyTorch clone location
python </path/to/skills>/xpu-ci-health-check/scripts/collect_failures.py --run-limit 1

Flags

  • --run-limit N — Inspect the last N completed main runs (default: 1).
  • --token — GitHub API token. If omitted, the script falls back to gh CLI auth.

The script prints a JSON evidence bundle to stdout:

{
  "runs": [{ "run_number": 9275, "head_sha": "5ef6fae…", "html_url": "…" }],
  "cases": [
    {
      "case_id": "test/inductor/test_cutlass_backend.py::TestCutlassBackend::test_xxx",
      "commit_sha": "5ef6fae…",
      "commit_short": "5ef6fae",
      "hud_url": "https://hud.pytorch.org/pytorch/pytorch/commit/5ef6fae…",
      "commit_url": "https://github.com/pytorch/pytorch/commit/5ef6fae…",
      "job_name": "linux-jammy-xpu-… / test (default, …)",
      "failure_line": "… FAILED …",
      "error_excerpt": "…traceback / exception text…",
      "issue_title": "DISABLED test_xxx (__main__.TestCutlassBackend)",
      "issue_body": "Platforms: xpu\n\nThis test was disabled because…\n\ncc …",
      "issue_labels": ["module: xpu", "triaged"],
      "issue_url": "https://github.com/pytorch/pytorch/issues/new?title=DISABLED%20…&body=…&labels=module%3A%20xpu,triaged"
    }
  ]
}

Disable issue template is frozen in the script. The issue_title, issue_body, issue_labels, and issue_url fields are produced by build_disable_issue() inside scripts/collect_failures.py, aligned with the reference issue template https://github.com/pytorch/pytorch/issues/185907 (Platforms line + "recent examples" section + cc mention + labels module: xpu, triaged). When creating an issue, use these exact fields verbatim — never rewrite the title, body, or labels.

Step 2 — Analyze the root cause (AI analysis required)

If the number of failed cases > 10, skip this step and let the root cause field be empty. For each case in cases, use subagent to do:

  1. Fetch latest origin main and git checkout to the commit_sha locally.
  2. Identify the real failure: Read error_excerpt and identify the failing frame and exception type (e.g., InductorError: NotImplementedError: ), not just the surface FAILED line.
  3. Root cause:
    • Determine regression status of the case:
      • Newly added or updated case → No
      • Existing old case (appeared before) → Yes
      • Insufficient evidence → Unknown (explain why)
    • Find the code change that cause the case failure and also the guilty commit/PR.
    • Analysis the root cause.
    • Write root cause in a strict 3-point structure (do not omit any point):
      • Introduced by which PR: Identify the most likely PR/commit that introduced the failure. If uncertain, state the top suspect and what evidence is missing.
      • Root cause of the failure: Explain the concrete failing mechanism on XPU using evidence from traceback/logs.
      • Is this fail only on XPU: try to analysis if cuda will fails, if no evidence is available, state that explicitly.
    • Use available history (recent runs, torch-ci failure history, prior reports).

Step 3 — Render output

Produce a list these details, one row per case:

List the details for each case as follows:

  • Commit: `<commit_short>` linked to HUD (e.g., [5ef6fae](https://hud.pytorch.org/...)).
  • Case: The case_id from the evidence bundle.
  • Is regression: Yes / No / Unknown (see Step 2.4; No = newly appeared).
  • Root cause: Both required points from Step 2 (introduced PR + why XPU fails mechanism).
  • Disable link: [Create disable issue](<issue_url>).
    • IMPORTANT: Use issue_url exactly as produced by the script — do not truncate.
    • The full URL carries prefilled title, body (Platforms + recent examples + cc), and labels (module: xpu, triaged).
    • A URL with only ?title=... is invalid and will not match the template.

After the list, add a one-line health summary:

  • Whether main is green or red for ciflow/xpu.
  • The run number(s) and count of failing cases inspected.

Important Notes

  • Zero failures: If the script returns no cases, report that main is green for the inspected run(s). Do not fabricate failures.
  • No auto-creation: Disable links are drafts only. A human must review and approve before creating issues.
  • Template reference: The frozen template follows https://github.com/pytorch/pytorch/issues/185907 exactly (Platforms + recent examples + cc + labels).

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.