Loop until verified

Overview

A loop step is done when an external check says so, not the agent. Bound it with a hard iteration cap plus a time or cost cap for long runs, and let it exit only on a check it cannot game. This skill covers what wraps a grading loop: caps, the anti-gaming exit, state and steering files, retry safety. See santa-method (Phase 2 to 4) for the loop itself, proofread (claim-verification-loop underneath) for repeated checks over a claim register.

When to use

• An agent loop runs several iterations with no stated stop.
• The agent doing the work would otherwise also judge when it is finished.
• The loop can fail or be interrupted and someone retries it.
• The loop runs unattended long enough to need mid-run redirection.

Do NOT use for a one-shot task, or a call a human reviews right after. This does not judge whether the goal itself is right, only whether the checks passed.

The pattern

1. Set caps before starting. max_iterations (default 5 to 10), plus a time or cost cap for long runs. No stop condition besides a pass.
2. Make the exit an external check. Named commands exiting zero, a checklist fully ticked, or a rubric graded by a separate grader decide, never the agent's own claim. Reject a check that always passes.
3. Every failing gate hands back the exact next fix, not "try again": name the item and the command to rerun.
4. Keep progress in a state file, re-read every iteration, one change per iteration, saved before the next.
5. Add an optional steering file. Read it once per iteration, splice it in, then clear it. Never block on it.
6. On retry: name the run, check the tree for leftover changes, require an explicit keep-or-discard decision. A cancelled retry is its own outcome, not an error. Skip only for a genuinely idempotent call (same result whether run once or twice, no stacking side effect).
7. After a pass, run held-back cases the loop never saw, graded the same way. A failure returns to step 3.
8. An iteration whose agent errored, timed out or returned nothing counts toward the iteration cap and is never read as a pass.
9. Stop condition: every gate passes, including held-back cases, or the cap is hit. At the cap, escalate with what fails. Never raise the cap mid-run.

Quick reference

Common mistakes

• No cap, or "iterate until it's good": never stops.
• The agent's own summary substituting for running the check.
• A gate that always exits zero: hides failure behind green.
• An "on_fail" message saying "fix it" instead of the exact command.
• A minimum-iteration floor with no ceiling: still runs forever.
• Raising the cap mid-run instead of escalating: unbounded by another name.

Source

• https://github.com/mikeyobrien/ralph-orchestrator, commit edc2b3268c9bd0c08a12c8193a7ace7ab2789261, MIT. File: ralph.yml (backpressure.gates, core.guardrails). Adapted: gate-plus-on_fail-plus-anti-placeholder (steps 2 to 3), generalized from its Rust fmt/clippy/test trio.
• https://github.com/Th0rgal/open-ralph-wiggum, commit 7559f267b227d74d82776048697615536c2a6b2a, MIT. Files: ralph.ts, completion.ts (tasksMarkdownAllComplete, checkTerminalPromise, containsPromiseTag). Adapted: parser-veto (step 2) and read-inject-clear steering (step 5), generalized beyond this repo's file names.
• https://github.com/alirezarezvani/claude-skills, commit 19392f7a08264ed00486a251f5b2098321771f94, MIT. File: agent-launcher/skills/grade-iterate/SKILL.md. Adapted: the held-back-case check after a pass (step 7).
• https://github.com/BloopAI/vibe-kanban, commit d5cbb5380fa0b32e98ef9b8d987f63decce4be3a, Apache-2.0. File: packages/web-core/src/shared/hooks/useRetryProcess.ts. Adapted: confirm-then-retry with keep-or-discard and a non-error cancel (step 6).
• The iteration-cap-plus-cost/time-cap pairing (steps 1, 9; the open-ralph-wiggum draft above used a floor with no ceiling) and the santa-method/verification-loop cross-references are this skill's own additions.