copilot-review-loop

When to invoke

When the user asks to loop a PR through automated review until it is clean — "run a review loop on #N", "get Copilot to review this and address everything", "keep requesting review until it's resolved". Also invoked from the two opt-in gates in feature-dev-workflow:developing-a-feature (sub-PRs, and the PR to main).

Not for a single one-off review pass — that is the review skill (local diff review) or a single gh pr edit --add-reviewer. This skill is the loop: request, wait, triage, address, re-request, until clean.

The mechanism (verified — do not improvise the commands)

Copilot review is requested only through gh pr edit. The REST requested_reviewers endpoint rejects Copilot (it accepts human logins and team slugs only), so gh api .../requested_reviewers -f "reviewers[]=copilot-..." returns 422. Use:

gh pr edit <pr> --add-reviewer "@copilot"

Requires gh ≥ 2.88.0 (gh --version). Not available on GitHub Enterprise Server.

Review effort level is a precondition

Copilot code review runs at one of two effort levels. Lite is the default: fast, targeted feedback on common issues. Balanced buys deeper analysis of complex logic, security-sensitive code, and cross-cutting changes from a higher-reasoning model, for more AI credits. A substantive change is worth balanced; a one-line correction is not.

No command sets it — do not improvise one. gh pr edit --add-reviewer "@copilot" takes no effort argument, the copilot_code_review ruleset rule carries only review_on_push and review_draft_pull_requests, and no REST or GraphQL field exposes the setting. It is a human choice in the GitHub UI, and where that choice lives depends on how the review is triggered:

Raise it before the first request, never after the review lands. The level applies to reviews requested after it is set, so discovering mid-loop that a substantive change drew a lite review costs a whole round to correct. Judge it at entry, alongside the take-stock checks below, and route it by the same context split as pushback:

• Interactive (standalone, or the PR-to-main gate): when the change is substantive, say so before requesting and ask the user to select balanced — in the Reviewers selector for a one-off, or in repository settings if the repo auto-reviews on push. Then request.
• Autonomous multi-PR fan-out (sub-PR gate): do not pause for it. The repository default governs; if a sub-PR is substantive enough that a lite review is a real gap, log it as a bubble-up concern in the state file's ## Bubble-up log and continue.

The review body is the only record of the level, and not every review carries it. A review can state its level in a labelled line of its body, **Review effort:** <level>, sometimes after an emoji and not always as the last line. Step 4 reads the body each round: when that line is present, report the level it states and name the line as your source. When it is absent, say that this review does not state its level, and report none. A missing line does not mean lite, a mention of effort elsewhere in the body is not the line, and how thorough the review looks is evidence of nothing.

The loop

digraph review_loop {
    "Un-triaged review on\ncurrent head?" [shape=diamond];
    "Repo auto-reviews on push?\n(review arrives without a request)" [shape=diamond];
    "Capture watermark, request Copilot" [shape=box];
    "Attached?" [shape=diamond];
    "Copilot unavailable:\ntriage existing human comments, exit" [shape=box];
    "Wait in BACKGROUND for new review" [shape=box];
    "Triage every finding: posted\n+ suppressed (receiving-code-review)" [shape=box];
    "Address: apply correct, push back on wrong" [shape=box];
    "New actionable findings\nOR unresolved threads?" [shape=diamond];
    "Cap reached?" [shape=diamond];
    "Done: clean" [shape=doublecircle];
    "Stop, report what remains" [shape=box];

    "Un-triaged review on\ncurrent head?" -> "Triage every finding: posted\n+ suppressed (receiving-code-review)" [label="yes"];
    "Un-triaged review on\ncurrent head?" -> "Repo auto-reviews on push?\n(review arrives without a request)" [label="no"];
    "Repo auto-reviews on push?\n(review arrives without a request)" -> "Wait in BACKGROUND for new review" [label="yes: wait for it"];
    "Repo auto-reviews on push?\n(review arrives without a request)" -> "Capture watermark, request Copilot" [label="no: request"];
    "Capture watermark, request Copilot" -> "Attached?";
    "Attached?" -> "Copilot unavailable:\ntriage existing human comments, exit" [label="no"];
    "Attached?" -> "Wait in BACKGROUND for new review" [label="yes"];
    "Wait in BACKGROUND for new review" -> "Triage every finding: posted\n+ suppressed (receiving-code-review)";
    "Triage every finding: posted\n+ suppressed (receiving-code-review)" -> "Address: apply correct, push back on wrong";
    "Address: apply correct, push back on wrong" -> "New actionable findings\nOR unresolved threads?";
    "New actionable findings\nOR unresolved threads?" -> "Cap reached?" [label="yes"];
    "Cap reached?" -> "Capture watermark, request Copilot" [label="no"];
    "Cap reached?" -> "Stop, report what remains" [label="yes"];
    "New actionable findings\nOR unresolved threads?" -> "Done: clean" [label="no"];
}

On entry, before the first request — take stock. Before you touch the reviewer, look at what is already on the PR:

• An un-triaged review on the current head is this round's review — triage it, do not request another. List Copilot reviews (gh api .../pulls/<pr>/reviews, author copilot-pull-request-reviewer[bot]) with their bodies, plus the open threads — the entry check is the one path where nothing hands you the review body, so a projection that drops body hides that round's suppressed findings. If a Copilot review already covers the current head commit and its findings are unaddressed, go straight to triage (Step 4). Firing --add-reviewer on top of an un-triaged review just duplicates the request and re-surfaces the same comments.
• If the repo auto-reviews on push, pushing is the trigger — never stack an explicit request on it. Check the PR timeline for "review requested due to automatic review settings", or a ruleset that runs Copilot review on push. When that is on, a push produces the review by itself, so an explicit --add-reviewer on top double-fires. This is the same rule Step 6 relies on for re-requests, applied to the first request too.

Fall through to Step 1's explicit request only when there is nothing current to triage and a push will not produce the review for you — the PR has never been reviewed and the repo does not auto-review, or the only existing review predates the latest push (its comments are on stale code).

One cycle, in order:

1. Capture the watermark, then request — only when the entry check says you need to. SINCE=$(date -u +%Y-%m-%dT%H:%M:%SZ) before gh pr edit <pr> --add-reviewer "@copilot". The watermark is what distinguishes the new review from a stale one left by an earlier round. Skip this step and go straight to triage (Step 4) when a current un-triaged review already exists, or when a push you are about to make will auto-trigger the review.
2. Confirm Copilot attached. gh pr view <pr> --json reviewRequests. If Copilot is not in the list, it is unavailable for this repo/plan (or gh is too old / this is GHES). Do not start waiting — there is no reviewer to produce a review. Instead do one triage pass over any human comments already on the PR (Step 4), state that Copilot review is unavailable, and exit. There is no loop without an automated reviewer to re-request.

3. Wait in the background. Launch the wait script with run_in_background: true so the session is not held hostage:

   ${CLAUDE_PLUGIN_ROOT}/skills/copilot-review-loop/templates/await-copilot-review.sh <pr> "$SINCE"

   It polls gh api .../pulls/<pr>/reviews for a Copilot review newer than $SINCE and exits 0 (printing the review) when one lands, or 124 on timeout. Do not write a foreground sleep/until loop — foreground sleep is blocked and it freezes the session.

   That one background shell is the whole wait. Its exit is the signal, and its output file holds the result. A subagent waits the same way as a top-level session:

   • End your turn after the launch. The harness re-invokes the session when the script exits.
   • On every wake, read the task's status and exit code from the harness, then its output file. Exit 0: go on to Step 4 with the review it printed. Exit 124: go to Step 6's timeout rule. Any other exit: the script failed before it could wait, so report the exit code and its output, and do not wait again. Still running: end your turn again. An empty output file alone does not show that the script still runs, because a failed script can exit without output.
   • If your harness gives no wake on a background exit, or you cannot tell that it does, run the same script in the foreground instead, with an interval and a timeout below your tool's command limit, for example await-copilot-review.sh <pr> "$SINCE" 30 540. It ends by itself, so nothing is left running.

   Never start a second command to learn when the wait script has ended. A watcher such as until ! pgrep -f "await-copilot-review.sh <pr>" never exits: pgrep -f matches whole command lines, and the watcher's own bash -c line holds the pattern, so it always finds itself. The same holds for ps | grep on that name.

   Before you report back or hand off, no background shell you started may still run. List your background tasks and stop any that is still live, such as a wait script that you left behind when you stopped early. A finished subagent with a live background shell stays alive, and the user sees it as still running.

4. Triage every finding the review carries — the comments it posted and the comments it suppressed — plus any human comments already on the PR. Copilot posts some findings as inline comments and withholds others into a Suppressed comments (N) block in the review body; a review can carry both, or carry suppressed findings alone with nothing posted at all. The body is also where a review states its effort level, when it states one (§Review effort level is a precondition). Read the body every round, not only the comment threads, or the loop triages a fraction of the review it was handed:

   gh api repos/{owner}/{repo}/pulls/<pr>/reviews --paginate \
     --jq '.[] | select((.user.login=="copilot-pull-request-reviewer[bot]") or (.user.login|test("[Cc]opilot"))) | {id, submitted_at, body}'

   Keeping id and submitted_at beside body is what proves the block you are triaging belongs to this round rather than a stale one; a bare .[].body dumps every review on the PR, human and stale alike, with nothing left to tell them apart. The author predicate is deliberately the same one templates/await-copilot-review.sh uses — exact bot login or a broad [Cc]opilot match — because a triage filter stricter than the wait filter would silently drop findings from a review the wait already accepted. Keep the two in step if either changes. REQUIRED SUB-SKILL: superpowers:receiving-code-review. Verify each against the codebase. No performative agreement, no blind implementation. Copilot is confidently wrong often enough that "apply all suggestions" is the wrong default.

5. Address, then always reply and resolve. Work one comment at a time, and before moving to the next, close the loop on it in this order: (a) for a comment that is correct, apply the fix, test it, and commit it; for a comment that is wrong or a judgment call, first handle the pushback per the calling context (below) to decide the resolution; (b) post a threaded reply stating the resolution — what you changed (name the fix or commit) for ones you applied, or why the comment does not apply for ones you reject; (c) resolve the thread (resolveReviewThread). The reply is mandatory and is never gated behind a confirmation prompt (see GitHub-mutation discipline) — every comment gets one, including rejected ones. Never resolve a thread without a reply, and never silently apply or silently ignore — a resolved-but-unexplained thread destroys the reasoning trail.

   REQUIRED SUB-SKILL: feature-dev-workflow:addressing-review-feedback before the first edit of the round, and again before the push in Step 6. It records the head before the first edit, finds the case of each finding, and runs its script on each comment that the round added, changed, or removed. Its table of verdicts goes in the same PR-level comment as the dispositions of the suppressed findings: one comment for the round. It adds no confirmation before a thread reply. For a finding whose fix adds, changes, or removes a comment in code, do (a) now, and do (b) and (c) after that skill applied its verdicts for the round: a verdict can change the fix, and the reply must say what the comment holds in the end. When a verdict changes the fix, commit that change too. Each other thread closes in the order above.

   A suppressed finding has no thread, so its disposition goes in one PR-level comment for the round. Reply-and-resolve needs a comment id and a suppressed finding has neither, but "nowhere to reply" is not "nothing to record". Post a single comment on the PR (gh pr comment <pr> --body-file <file>) naming each suppressed finding and what you did with it — applied (naming the fix or commit), or rejected (with the reason). One comment for the round, not one per finding. The obligation above is unchanged: no silent apply, no silent drop.

   OPTIONAL SUB-SKILL: simple-english:simple-english for the prose of each thread reply and the round's PR-level comment. If it is available, load it before you draft and write the sentences under it. Where its formatting rules (headings, bold, lists, or its register for chat replies) differ from the host skill or its template, the host skill and its template win. It sets sentence style only: each reply still states the resolution and nothing else, and it adds no confirmation or delay before the reply is posted. It also covers the body of each fix commit, whose subject line keeps the project's commit convention. If the skill is not available, do not stop or wait for it. If you have not already done so in this session, tell the user once that it can be installed with /plugin marketplace add AminBlg/SimpleEnglish and then /plugin install simple-english@simple-english. Then write short, plain, active sentences without it.

6. Re-request and loop. The fixes were committed per comment in Step 5 (commit convention from the project's CLAUDE.md). Capture a fresh watermark first, then push them. Now wait before re-requesting — launch the background wait (Step 3) against that watermark:

   • A repo that runs Copilot review on every push will produce the new review from the push alone. Waiting first lets that auto-review land and counts it (it is newer than the pre-push watermark). Re-requesting on top of it would double the review — so do not re-request until the wait has had a chance to catch a push-triggered review.
   • What a wait timeout means depends on whether the repo auto-reviews on push — and only one case authorizes a re-trigger. On a repo that auto-reviews on push, the push already queued the review, so a timeout means it is latent, not absent (auto-review can lag the push by hours). Re-triggering then stacks a second review of the same commit — do not re-trigger; stop and report that the push-triggered review is still pending and will land on its own schedule, and the user can resume the loop when it appears. Re-trigger explicitly with remove-then-re-add (gh pr edit <pr> --remove-reviewer "@copilot" then --add-reviewer "@copilot" — a plain re-add often no-ops once Copilot has already reviewed) only when the repo does not auto-review on push, so the push genuinely will not produce a review by itself; then launch the wait once more.

   When a new review lands (from either path), return to Step 4.

   If the second wait also times out, STOP — do not spin. Re-requesting is inherently unreliable (GitHub exposes no dependable programmatic re-review; Copilot does not reliably auto-review on push either). Report that automated re-review could not be triggered and point the user at the robust path: enable a repo ruleset that runs Copilot code review on every push, which makes the loop reliable, or click the re-request (🔄) icon in the PR's Reviewers menu. Do not loop further against a reviewer that isn't responding.

Termination

Stop and declare clean when all hold:

• a fresh Copilot review (newer than the latest watermark) returns no new actionable findings — posted or suppressed, AND
• every review thread is resolved or replied (a verified-and-declined comment with a stated reason counts as resolved — do not thrash re-litigating it), AND
• any artifact the loop's fixes drifted has been reconciled (§Reconcile what the loop's fixes changed) — or, in autonomous fan-out, logged as a bubble-up for the checkpoint to reconcile.

Copilot's own headline counts only what it posted. A review body that opens "generated no new comments" can still carry a Suppressed comments (N) block underneath, and a review can suppress every finding it made — so an empty comment list and an empty thread list are together no evidence the round was empty. Read past the summary line before treating a review as clean.

A loop that ends with a PR body — or tracking issue — contradicting its own diff is not clean, even with every thread resolved.

Safety cap: 3 request→address rounds. If the loop has not converged by the cap, stop and report what remains unresolved. Do not raise the cap to keep going — non-convergence (Copilot surfacing churny or contradictory nits round after round) is a signal to hand back to the user, not to loop harder. Only count a review as "clean" if it arrived after your last push; a review from before the push is stale and must not end the loop.

Stop early if a re-request can't be triggered. Separate from the convergence cap: if a round's re-request (Step 6) produces no new review before the wait times out, stop and report rather than retrying blindly. Re-review is not reliably automatable; the durable fix is the on-push Copilot ruleset. Also expect that when a re-review does fire, Copilot may repeat comments you already addressed or dismissed — re-verify against the current diff rather than assuming a repeated comment is a new problem.

Pushback handling depends on the calling context

The skill is told its context when invoked. This controls Step 5 pushback only.

• Standalone, or the PR-to-main gate (interactive): pause and surface the pushback to the user with technical reasoning, then act on their direction.
• Autonomous multi-PR fan-out (sub-PR gate): do not pause interactively — that breaks "autonomous fan-out has no per-sub-PR round-trips". Log the pushback as a bubble-up concern in the state file's ## Bubble-up log, apply the rest, and let feature-dev-workflow:reviewing-feature-progress surface it at the wave checkpoint.

Reconcile what the loop's fixes changed

Addressing review changes code, and a substantive fix can leave the PR body — or the tracking issue — describing something the diff no longer does. The loop does not own those artifacts and must not edit them silently, but it must not declare clean while they lie either. When a fix materially changed what the PR or its issue describes (the test: would a reader of the body be misled by what the diff now does?), reconcile before handing back, routed by the same context split as pushback above:

• Interactive (standalone or the PR-to-main gate): reconcile now. PR body → REQUIRED SUB-SKILL: feature-dev-workflow:opening-a-pull-request (reconcile the open PR body — body only, no comment). Tracking issue → REQUIRED SUB-SKILL: feature-dev-workflow:writing-github-issues Step 2D (decision comment + body, linking the commit). Each carries its own confirmation; you are not editing either artifact silently.
• Autonomous multi-PR fan-out (sub-PR gate): do not reconcile inline — that reintroduces the per-sub-PR round-trip the autonomous mode exists to avoid. Log the drift as a bubble-up concern in the state file's ## Bubble-up log, and let feature-dev-workflow:reviewing-feature-progress reconcile it at the wave checkpoint (it already routes drift through the owning skills).

GitHub-mutation discipline

Not every GitHub mutation in the loop is the same kind of action, and conflating them is what makes the reasoning trail disappear. Separate them:

• Replying in a thread and resolving it are how the loop records a decision you are already making — mandatory, and never confirmation-gated. The reply's body is dictated by the comment it answers (what you changed, or why the comment does not apply), so there is nothing separate to confirm. Do not pause to ask permission before leaving a reply, and do not skip or batch-defer it to avoid pestering. A thread addressed without a reply, or resolved silently, destroys the human-readable trace — which is the entire audit value of the loop. This holds in every context, interactive or autonomous.
• Requesting (or re-requesting) the reviewer and pushing commits are outward, harder-to-reverse mutations. These follow the project rule: in interactive contexts, a fresh confirmation against the specific thing about to land unless the user already granted that kind of action; in autonomous fan-out, the autonomous-mode discipline already established for sub-PRs (the user opted into the mechanical bundle up front).

What this skill does not do

• Does not merge the PR. It drives the PR to review-clean and hands back.
• Does not invent comments or run its own static analysis. Copilot and humans are the reviewers; this skill is the loop and the judgment layer over their output.
• Does not block on human reviewers. It sweeps in human comments already present, but only re-requests Copilot.

Red flags