agentleFS
shanselmanshanselman/openclaw-windows-hub/.agents/skills/retrigger-flaky-ci/SKILL.md

retrigger-flaky-ci

Skill2.2k starsChanged 3 days agoMITView on GitHub

Triage failed CI on a pull request or the active branch, decide whether the failures come from its changes, and rerun only failures that are unrelated. Use when asked to retrigger, rerun, or kick CI on a PR with failing checks.

1 other project uses a version of this; 1 new project in the last 4 weeks, against 0 the 4 before.

Adopted by

Surging: 1 new project took it in the last 4 weeks, none the 4 weeks before.thin data

Week of Sep 20: 0 new projectsWeek of Sep 27: 0 new projectsWeek of Oct 4: 0 new projectsWeek of Oct 11: 0 new projectsWeek of Oct 18: 0 new projectsWeek of Oct 25: 0 new projectsWeek of Nov 1: 0 new projectsWeek of Nov 8: 1 new project
Sep 2026Oct 2026Nov 2026
New projects taking a version of this file each week, over the 8 weeks our latest run read: only 1 in all, so this is a hint, not a trend.

Most recently taken up by

The file

We split this skill into the rules it gives an agent

11 rules. About 0 are common, 10 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈10

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use this skill when a user asks to retrigger, rerun, or restart CI for a pull request, or for the PR backing the current branch.

    line 8
  2. Distinctive:

    Never rerun first and investigate later.

    line 13
  3. Very specific:

    If local HEAD differs from the PR head, or there are uncommitted changes, the CI run did not test them.

    line 35
  4. Distinctive:

    If checks are still queued or running, report that and stop.

    line 47
  5. Distinctive:

    Read the failed job logs:

    line 50
  6. Distinctive:

    Treat a failure as related if any of the following is true:

    line 65
  7. Distinctive:

    Treat a failure as unrelated only with positive evidence, such as:

    line 79
  8. Distinctive:

    If the evidence is mixed and a focused local run is possible, run it.

    line 102
  9. Distinctive:

    If the failure is unrelated but deterministic on main (it fails on every recent main run), a rerun will not help.

    line 105
  10. Distinctive:

    Do not rerun. Tell the user that the failures come from the PR's changes.

    line 111
  11. Distinctive:

    Do not rerun the same failure on the same head more than once.

    line 134

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Retrigger flaky CI

Use this skill when a user asks to retrigger, rerun, or restart CI for a pull request, or for the PR backing the current branch. The request authorizes a rerun of failed jobs on that PR only. It does not authorize pushing, editing the PR, changing labels, or rerunning other PRs.

Never rerun first and investigate later. Read the failure, compare it with the PR's changes, then either warn the user or rerun.

1. Resolve the PR and its changes

Resolve the PR by number, or from the current branch when no number is given:

gh pr view <number> --json number,title,headRefName,headRefOid,files,statusCheckRollup,url
gh pr view --json number,title,headRefName,headRefOid,files,statusCheckRollup,url

Record the PR head SHA and the changed file list. When the request refers to the active branch, also compare local work with the PR:

git rev-parse HEAD
git status -sb
git fetch origin main --quiet
git diff --stat origin/main...HEAD

If local HEAD differs from the PR head, or there are uncommitted changes, the CI run did not test them. Say so, and judge the failure against the PR head that CI actually ran.

Pull request runs of Build and Test check out refs/pull/<number>/merge, so they include the current main. A failure can come from main rather than from the PR.

2. Read the actual failures

List the failing checks from statusCheckRollup. CI Gate only aggregates the required jobs. Look at the jobs it reports as failed instead of CI Gate itself. If checks are still queued or running, report that and stop. A partial run is not grounds for a rerun.

Read the failed job logs:

gh run view <run-id> --log-failed
gh run view <run-id> --job <job-id> --log-failed

Find the specific failure in each failing job: [FAIL] and Failed <TestName> lines with their Error Message and stack trace, error CS or error MSB build errors, Test host process crashed, validator output, or the step that reported ##[error]. Record the test or step, the error, and the top stack frames that belong to the repository.

3. Decide whether the PR caused the failure

Treat a failure as related if any of the following is true:

  • It is a build, analyzer, or compile error in a changed file, or in code that calls an API the PR changed.
  • The failing test file is in the PR diff, or the test directly exercises a production owner, method, string resource, or contract the PR changed. Map the repository frames in the stack trace to the diff.
  • It is a deterministic assertion about behavior, copy, localization, architecture ledgers, docs, skills, proof pools, or CI classification that the PR touched.
  • It fails the same way on more than one attempt of the same head SHA.
  • It reproduces locally on the PR head with a focused run, for example dotnet test <project> --filter "FullyQualifiedName~<TestName>".

Treat a failure as unrelated only with positive evidence, such as:

  • The failing code path does not touch anything in the diff, and the diff adds no startup, background, or Gateway traffic that could reach it.
  • The same test or job fails on main or on unrelated PRs:

    gh run list --workflow "Build and Test" --status failure --limit 25 --json databaseId,headBranch,event,createdAt,displayTitle
    gh run view <other-run-id> --log-failed
  • Retained CI evidence or an existing issue shows the same failure on the recorded base revision, and comparison confirms the PR does not introduce it.
  • It has the shape of a timing or infrastructure failure: a UI Automation element briefly missing (NullReferenceException inside a wait lambda), a TimeoutException from a fixture wait, a wall-clock limit on a loaded runner, a crashed test host, a network or restore error, a runner shutdown, or a missing hosted tool.

A timeout counts as flaky only when the waited-for behavior is outside the diff. A timeout in a path the PR changed is a related failure until it is reproduced or ruled out locally.

If the evidence is mixed and a focused local run is possible, run it. If it remains ambiguous, do not rerun. Report the uncertainty and ask the user.

If the failure is unrelated but deterministic on main (it fails on every recent main run), a rerun will not help. Report it as a main breakage and do not rerun.

4a. Related: warn and do not rerun

Do not rerun. Tell the user that the failures come from the PR's changes. For each failure, give the job, the test or step, the error, and the changed file or owner that connects to it. Do not edit code unless the user asks for a fix.

4b. Unrelated: summarize, then rerun failed jobs

Show a short table before rerunning:

JobFailing test or stepErrorWhy it is unrelated

Then rerun only the failed jobs of each failed run. Rerunning a job also reruns the jobs that depend on it, including CI Gate:

gh run rerun <run-id> --failed
gh run view <run-id> --json status,attempt,url

Report the run URL and the new status. Do not wait for the rerun to finish unless the user asks. If gh run rerun is refused, for example because of missing permission or a fork run that needs approval, report the error as is.

Do not rerun the same failure on the same head more than once. A second identical failure on the same SHA reclassifies it as related, or as a real main breakage. Stop rerunning and report the evidence and blocker. When authorized to fix it, investigate the cause, implement the repair, and validate it. Do not create a checked-in failure backlog.

Report

  • PR number and title, head SHA, and whether local work matches it.
  • Each failing job with its test or step and error.
  • The related or unrelated decision for each failure, with its evidence (diff overlap, matching baseline failures, existing issue evidence, local reproduction).
  • The action taken: no rerun with a warning, or the rerun command and the new run status.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.