xpu-ci-health-check
Skill122 starsChanged 5 months agoApache-2.0View on GitHub
Check PyTorch ciflow/xpu (xpu.yml) on the main branch, collect the failing XPU test cases from the most recent completed run(s), analyze the ROOT CAUSE of each failure with AI, and produce a list with a prefilled "disable issue" link per case. USE WHEN the user asks to check XPU CI health, find failing XPU tests, or generate XPU disable-issue drafts.
In this repository28 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- .github/copilot-instructions.mdCopilot instructions
- asm-source-mappingSkill
- at-dispatch-v2Skill
- auto-labelSkill
- extract-asm-onednnSkill
- extract-asm-syclkernel-aotSkill
- extract-asm-syclkernel-jitSkill
- extract-asm-tritonSkill
- extract-xpu-kernel-asmSkill
- fix-implementSkill
- fix-reproduceSkill
- fix-root-causeSkill
- fix-verifySkill
- issue-handlerSkill
- issue-triageSkill
- label-issueSkill
- oob-perf-analysisSkill
- pr-reviewSkill
- release-branchingSkill
- skill-writerSkill
- ut-checkSkill
- ut-issue-authoringSkill
- ut-refactor-reviewSkill
- xpu-alignmentSkill
- xpu-build-pytorchSkill
- xpu-ops-pr-creationSkill
- xpu-release-notesSkill
The file
We split this skill into the rules it gives an agent
7 rules. About 0 are common, 3 are distinctive, and 4 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
4
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 32
Run the bundled collection script from the PyTorch repository root.
- Very specific: line 83
If the number of failed cases > 10, skip this step and let the root cause field be empty.
- Very specific: line 87
Identify the real failure: Read error_excerpt and identify the failing frame and exception type (e.g., InductorError: NotImplementedError: …
- Distinctive: line 101
Use available history (recent runs, torch-ci failure history, prior reports).
- Very specific: line 114
IMPORTANT: Use issue_url exactly as produced by the script — do not truncate.
- Distinctive: line 119
After the list, add a one-line health summary:
- Distinctive: line 127
Zero failures: If the script returns no cases, report that main is green for the inspected run(s).
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
XPU CI Health Check Skill
This skill automates XPU CI health checks with the following workflow:
- Collect failure evidence automatically by running the bundled script.
- Analyze the root cause of each case yourself (AI), using the traceback evidence plus the suspect commit/PR — never copy the raw exception as the conclusion.
- Render the result generate a list with one row per failing case.
Root cause analysis is YOUR job, not the script's. The script only gathers evidence (case id, traceback excerpt, commit sha, disable link). You must reason about why it failed and which change likely caused it.
Prerequisites
- GitHub token: Optional. The script uses the
ghCLI if--tokenis not provided; ensuregh auth loginis configured.
Step 1 — Collect evidence (always run this first)
Run the bundled collection script from the PyTorch repository root. The script
is located at <path/to/skills>/xpu-ci-health-check/scripts/collect_failures.py
(included in this skill).
# Run from PyTorch repository root
cd /path/to/pytorch # or your PyTorch clone location
python </path/to/skills>/xpu-ci-health-check/scripts/collect_failures.py --run-limit 1Flags
--run-limit N— Inspect the last N completedmainruns (default:1).--token— GitHub API token. If omitted, the script falls back toghCLI auth.
The script prints a JSON evidence bundle to stdout:
{
"runs": [{ "run_number": 9275, "head_sha": "5ef6fae…", "html_url": "…" }],
"cases": [
{
"case_id": "test/inductor/test_cutlass_backend.py::TestCutlassBackend::test_xxx",
"commit_sha": "5ef6fae…",
"commit_short": "5ef6fae",
"hud_url": "https://hud.pytorch.org/pytorch/pytorch/commit/5ef6fae…",
"commit_url": "https://github.com/pytorch/pytorch/commit/5ef6fae…",
"job_name": "linux-jammy-xpu-… / test (default, …)",
"failure_line": "… FAILED …",
"error_excerpt": "…traceback / exception text…",
"issue_title": "DISABLED test_xxx (__main__.TestCutlassBackend)",
"issue_body": "Platforms: xpu\n\nThis test was disabled because…\n\ncc …",
"issue_labels": ["module: xpu", "triaged"],
"issue_url": "https://github.com/pytorch/pytorch/issues/new?title=DISABLED%20…&body=…&labels=module%3A%20xpu,triaged"
}
]
}Disable issue template is frozen in the script. The
issue_title,issue_body,issue_labels, andissue_urlfields are produced bybuild_disable_issue()insidescripts/collect_failures.py, aligned with the reference issue template https://github.com/pytorch/pytorch/issues/185907 (Platforms line + "recent examples" section + cc mention + labelsmodule: xpu,triaged). When creating an issue, use these exact fields verbatim — never rewrite the title, body, or labels.
Step 2 — Analyze the root cause (AI analysis required)
If the number of failed cases > 10, skip this step and let the root cause field be empty.
For each case in cases, use subagent to do:
- Fetch latest origin main and git checkout to the
commit_shalocally. - Identify the real failure: Read
error_excerptand identify the failing frame and exception type (e.g.,InductorError: NotImplementedError:), not just the surfaceFAILEDline. - Root cause:
- Determine regression status of the case:
- Newly added or updated case →
No - Existing old case (appeared before) →
Yes - Insufficient evidence →
Unknown(explain why)
- Newly added or updated case →
- Find the code change that cause the case failure and also the guilty commit/PR.
- Analysis the root cause.
- Write root cause in a strict 3-point structure (do not omit any point):
- Introduced by which PR: Identify the most likely PR/commit that introduced the failure. If uncertain, state the top suspect and what evidence is missing.
- Root cause of the failure: Explain the concrete failing mechanism on XPU using evidence from traceback/logs.
- Is this fail only on XPU: try to analysis if cuda will fails, if no evidence is available, state that explicitly.
- Use available history (recent runs, torch-ci failure history, prior reports).
- Determine regression status of the case:
Step 3 — Render output
Produce a list these details, one row per case:
List the details for each case as follows:
- Commit:
`<commit_short>`linked to HUD (e.g.,[5ef6fae](https://hud.pytorch.org/...)). - Case: The
case_idfrom the evidence bundle. - Is regression:
Yes/No/Unknown(see Step 2.4;No= newly appeared). - Root cause: Both required points from Step 2 (introduced PR + why XPU fails mechanism).
- Disable link:
[Create disable issue](<issue_url>).- IMPORTANT: Use
issue_urlexactly as produced by the script — do not truncate. - The full URL carries prefilled title, body (Platforms + recent examples + cc), and
labels (
module: xpu,triaged). - A URL with only
?title=...is invalid and will not match the template.
- IMPORTANT: Use
After the list, add a one-line health summary:
- Whether
mainis green or red forciflow/xpu. - The run number(s) and count of failing cases inspected.
Important Notes
- Zero failures: If the script returns no cases, report that
mainis green for the inspected run(s). Do not fabricate failures. - No auto-creation: Disable links are drafts only. A human must review and approve before creating issues.
- Template reference: The frozen template follows https://github.com/pytorch/pytorch/issues/185907 exactly (Platforms + recent examples + cc + labels).
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

