Simmer Plugin

Overview

Simmer is an iterative refinement skill. It takes any artifact — a single file, a workspace of multiple files, a pipeline, a configuration — and hones it repeatedly against user-defined criteria. Evaluation can be subjective (judge-based), objective (runnable script), or hybrid (both).

Skills Included

Subskills (invoked by orchestrator)

Flow

"Simmer this" / "Refine this" / "Optimize this pipeline"
    ↓
Setup (identify artifact + criteria + evaluation method)
    ↓
Iteration 0: Evaluate the seed
    ↓
Loop N times:
  Generate (address ASI) → Evaluate (run evaluator if present) → Judge (score + new ASI) → Reflect (record)
    ↓
Output best candidate + trajectory

Key Design Decisions

Context discipline: Generator doesn't see scores (avoids optimizing for numbers). Judge doesn't see intermediate scores (avoids anchoring). Reflect is the only subskill that sees the full trajectory.

Seed calibration: Judge receives the seed artifact + iteration-0 scores on every iteration as a fixed calibration reference. Can score above or below the seed. This compresses score variance across runs.

ASI (Actionable Side Information): Judge identifies the highest-leverage direction each round. For single-file targets, this is a single focused fix. For workspace targets, this is a single strategic direction that may involve coordinated changes across multiple files. Focused directional improvement compounds better than scattered edits.

Pluggable evaluation: Three modes — judge-only (default, subjective criteria), runnable (script/command output), or hybrid (both). No format contract on evaluator output — the judge interprets whatever the evaluator produces.

Regression handling: Reflect tracks best-so-far. If an iteration regresses, next generator receives the best candidate, not the regressed one. In workspace mode, this means git checkout to the best iteration's commit.

Judge board: For complex artifacts, the board constructs problem-specific judges that investigate before scoring — reading the evaluator script, ground truth, prior candidates, and config to understand the problem before proposing improvements. Judges are composed once per run, tailored to the specific problem, then reuse for all iterations with updated context. Tracks stable wins (WORKING/NOT WORKING) across iterations to prevent undoing proven improvements.

Artifact Modes

Evaluation Modes

Related Skills

Simmer is part of the test-kitchen family but independently installable:

• test-kitchen:omakase-off — parallel design exploration
• test-kitchen:cookoff — parallel implementation competition
• simmer — iterative refinement of any artifact or workspace

Directory Structure

simmer/
  .claude-plugin/
    plugin.json
  skills/
    simmer/
      SKILL.md                  # Orchestrator
    simmer-setup/
      SKILL.md                  # Identify artifact + elicit criteria + evaluation method
    simmer-generator/
      SKILL.md                  # Produce improved candidate
    simmer-judge/
      SKILL.md                  # Score + produce ASI (with optional evaluator output)
    simmer-judge-board/
      SKILL.md                  # Multi-judge panel with deliberation
    simmer-reflect/
      SKILL.md                  # Record trajectory + track best
  docs/
    specs/
      2026-03-16-simmer-v2-design.md
  tests/
    integration/
      simmer-scenario.md        # Integration test scenarios
  CLAUDE.md
  README.md

Artifact Output

Single-file mode:

{OUTPUT_DIR}/                    # Default: docs/simmer
  iteration-0-candidate.md      # Seed
  iteration-1-candidate.md      # Each improved candidate
  iteration-2-candidate.md
  iteration-3-candidate.md
  trajectory.md                 # Running score table
  result.md                     # Final best output

Workspace mode:

{WORKSPACE}/                    # The target directory (modified in place)
  [project files]               # Tracked via git commits per iteration

{OUTPUT_DIR}/
  trajectory.md                 # Running score table