agentleFS
terravicterravic/pde-agent-plugin/skills/run-protocol/SKILL.md

run-protocol

Skill0 starsChanged 13 days agoApache-2.0View on GitHub

Supervisor run protocol -- state machine, worker lifecycle, budget accounting, convergence detection, and error handling for orchestrating multi-epoch hypothesis exploration runs.

3 other projects use a version of this, 1 of them edited.

Adopted by

Most recently taken up by

The file

Show its rules and text

We split this skill into the rules it gives an agent

39 rules. About 0 are common, 27 are distinctive, and 12 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈27

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 12

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Budget exhaustion: If the remaining per-epoch budget reaches 0, skip TOURNAMENT and proceed to EVOLVE (or META if this is a late epoch).

    line 55
  2. Distinctive:

    Total match budget: Default 100 matches across all epochs.

    line 60
  3. Very specific:

    Budget exhaustion: When TOTAL_MATCHES_USED >= TOTAL_MATCH_BUDGET, finalize with reason: budget_exhausted regardless of stability.

    line 64
  4. Very specific:

    Max epochs: Default 5. When the epoch counter reaches the max, finalize with reason: budget_exhausted.

    line 66
  5. Distinctive:

    After META completes in each epoch, the supervisor checks whether to loop or finalize:

    line 75
  6. Distinctive:

    Read the current epoch's standings:

    line 89
  7. Very specific:

    Read the previous epoch's standings from ratings/epoch-<N-1>.json.

    line 95
  8. Distinctive:

    If both conditions are met for 2 consecutive epochs (i.e., the top-5 was also stable between epoch N-2 and epoch N-1), the stability …

    line 102
  9. Distinctive:

    Wall-clock limit exceeded: If configured and the run has exceeded the time limit.

    line 121
  10. Distinctive:

    Check the hard finalization criterion first.

    line 128
  11. Distinctive:

    Check stability criterion (soft limit -- requires 2 consecutive stable epochs).

    line 129
  12. Distinctive:

    If either criterion is met -> proceed to FINALIZE.

    line 131
  13. Distinctive:

    If neither is met -> increment the epoch counter and loop back to GENERATE.

    line 132
  14. Distinctive:

    When looping back to GENERATE after META, the supervisor injects the steering memo into worker task messages:

    line 139
  15. Very specific:

    Read the steering memo: <run-dir>/meta/epoch-<N>-steering.md.

    line 142
  16. Very specific:

    Before FINALIZE completes, the supervisor runs pkde hypex ingest <run-dir> and pkde hypex analyze <artifact> so the tournament enters PkDE …

    line 173
  17. Very specific:

    Resolve the artifact path: Check whether the task message contains an override line of the form Artifact path: <path>.

    line 201
  18. Very specific:

    Verify success by checking the output: Initialized run: ${ARTIFACT_PATH}/<run-id>

    line 231
  19. Very specific:

    Run directory exists with valid run.yaml.

    line 255
  20. Distinctive:

    Purpose: Start generation workers to explore each focus area and produce hypotheses.

    line 263
  21. Distinctive:

    If epoch > 0, a steering memo exists from the previous epoch.

    line 271
  22. Distinctive:

    All worker reports have been received (or timed-out workers are logged).

    line 320
  23. Distinctive:

    Purpose: Start reflection workers to review and score all hypotheses.

    line 326
  24. Distinctive:

    All hypothesis IDs are known.

    line 331
  25. Distinctive:

    Purpose: Run proximity analysis to identify clusters and near-duplicate hypotheses, then retire duplicates before the tournament.

    line 391
  26. Distinctive:

    Only active (non-merged) hypotheses proceed to TOURNAMENT.

    line 440
  27. Distinctive:

    Purpose: Run pairwise Elo-rated matches to produce competitive rankings.

    line 459
  28. Distinctive:

    If pairings is empty (no valid pairings possible), skip to EVOLVE.

    line 492
  29. Distinctive:

    Purpose: Run tournament matches where evolved variants compete against their parents and optionally against each other.

    line 693
  30. Distinctive:

    Example: If H-0055 evolved from H-0012 and H-0056 evolved from H-0017, write:

    line 715
  31. Very specific:

    If len(REMATCH_PAIRINGS) > REMAINING, truncate the pairings JSON file to budget before passing it to elo pair.

    line 741
  32. Very specific:

    Budget exhausted before rematches: If no budget remains, skip TOURNAMENT_REMATCH entirely.

    line 794
  33. Distinctive:

    Read current epoch standings: top-5 IDs and Elo values

    line 879
  34. Very specific:

    Read previous epoch standings (ratings/epoch-<N-1>.json): top-5 IDs and Elo values

    line 880
  35. Distinctive:

    If epoch >= 2, also check epoch N-1 vs N-2 stability

    line 882
  36. Distinctive:

    If converged -> proceed to FINALIZE

    line 890
  37. Distinctive:

    Write the convergence check result to the run metadata:

    line 898
  38. Distinctive:

    All epochs' data is available (hypotheses, reviews, matches, ratings, steering memos).

    line 921
  39. Very specific:

    All workers are started with agent start <name> --type <template> and report back via agent message.

    line 1015

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Run Protocol (v2)

This skill defines the supervisor's protocol for orchestrating a research exploration run. The PkDE port uses multi-epoch looping, evolution, meta-review steering, convergence detection, and a finalization phase. The initial generation-through-tournament sequence is followed by the v2 states and epoch loop; the retired single-epoch pilot is not a fallback mode.

State Machine (v2)

INIT -> GENERATE -> REVIEW -> DEDUP -> TOURNAMENT -> EVOLVE -> REVIEW_EVOLVED ->
  TOURNAMENT_REMATCH -> META -> [convergence check] -> loop to GENERATE or FINALIZE -> DONE

Each state has a clear entry condition, a set of actions, and an exit condition that advances to the next state.

State Summary

StatePurposeWorkers
INITSet up run directory, decompose goal, initialize budgetsNone
GENERATEExplore focus areas, produce hypotheses2-3 hypex-generation
REVIEWScore and evaluate all hypotheses1-2 hypex-reflection
DEDUPCluster hypotheses, merge duplicates1 hypex-proximity
TOURNAMENTPairwise Elo-rated matches1-3 hypex-tournament
EVOLVEApply evolutionary operators to top-K hypotheses1 hypex-evolution
REVIEW_EVOLVEDReview evolved variants (recurrent mode)1 hypex-reflection
TOURNAMENT_REMATCHEvolved variants compete; mandatory parent rematches1-3 hypex-tournament
METAProduce per-epoch steering memo1 hypex-meta-review
FINALIZETask meta-review agent for final report, then publish PkDE artifacts1 hypex-meta-review
DONESignal completionNone

Budget Accounting

v2 maintains two levels of match budget:

Per-Epoch Budget

  • Default: 20 matches per epoch. Configurable in run.yaml or via task message.
  • Tracking: The supervisor tracks EPOCH_MATCHES_USED within each epoch. This count is reset to 0 at the start of each epoch.
  • Enforcement: The --budget N flag on elo pair limits pairings. Pass the remaining per-epoch budget (or remaining total budget, whichever is lower).
  • Budget exhaustion: If the remaining per-epoch budget reaches 0, skip TOURNAMENT and proceed to EVOLVE (or META if this is a late epoch).

Cross-Epoch Budget

  • Total match budget: Default 100 matches across all epochs. Configurable.
  • Tracking: The supervisor maintains TOTAL_MATCHES_USED across all epochs. After each elo pair call (in both TOURNAMENT and TOURNAMENT_REMATCH), add the pairing count.
  • Budget exhaustion: When TOTAL_MATCHES_USED >= TOTAL_MATCH_BUDGET, finalize with reason: budget_exhausted regardless of stability.
  • Max epochs: Default 5. When the epoch counter reaches the max, finalize with reason: budget_exhausted.
  • Wall-clock limit: Optional. If the run exceeds the configured wall-clock limit, finalize with reason: budget_exhausted at the next check.

Convergence Criteria

After META completes in each epoch, the supervisor checks whether to loop or finalize:

Stability Criterion

Top-5 hypotheses (by Elo) unchanged across 2 consecutive epochs.

"Unchanged" means both:

  1. Same 5 hypothesis IDs in the top-5 (order may differ).
  2. All Elo deltas in the top-5 are below a configurable threshold (default: 30 points).

To check stability:

  1. Read the current epoch's standings:
    elo standings --run-dir <path> --format json
    Extract the top-5 hypothesis IDs and their Elo ratings.
  2. Read the previous epoch's standings from ratings/epoch-<N-1>.json. Extract the top-5 hypothesis IDs and their Elo ratings.
  3. Compare:
    • Are the same 5 IDs present in both top-5 sets?
    • For each ID in the top-5, is |elo_current - elo_previous| < 30?
  4. If both conditions are met for 2 consecutive epochs (i.e., the top-5 was also stable between epoch N-2 and epoch N-1), the stability criterion is satisfied.

    Note: This requires at least 3 epochs (0, 1, 2) to evaluate -- stability is first checkable after epoch 2.

    Note: Convergence stability checks use raw elo (without --composite), because composite moves whenever a new review lands and convergence asks "has the tournament stopped learning".

Hard Finalization Criterion

Any of the following triggers finalization without claiming convergence:

  • Max epochs reached: CURRENT_EPOCH + 1 >= MAX_EPOCHS (default 5). That is, if starting the next epoch would exceed the limit, finalize now.
  • Total match budget exhausted: TOTAL_MATCHES_USED >= TOTAL_MATCH_BUDGET (default 100).
  • Wall-clock limit exceeded: If configured and the run has exceeded the time limit.

Decision

After META completes:

  1. Check the hard finalization criterion first.
  2. Check stability criterion (soft limit -- requires 2 consecutive stable epochs).
  3. If either criterion is met -> proceed to FINALIZE.
  4. If neither is met -> increment the epoch counter and loop back to GENERATE.

Steering Memo Injection

When looping back to GENERATE after META, the supervisor injects the steering memo into worker task messages:

  1. Read the steering memo: <run-dir>/meta/epoch-<N>-steering.md.
  2. On the next epoch's GENERATE, include the memo content in each generation worker's task message. This tells generation agents:
    • What kinds of hypotheses to produce more of
    • What weaknesses to avoid
    • Which focus areas need more attention
  3. On the next epoch's REVIEW, include the memo content in each reflection worker's task message. This tells reviewers:
    • Any calibration adjustments
    • Patterns to watch for
    • Known systemic issues to flag

The memo is included as a section in the task message, prefixed with a clear header:

--- STEERING MEMO (from epoch N) ---
<memo content>
--- END STEERING MEMO ---

PkDE Subgraph Boundary

The Hypex supervisor is dispatched from a PkDE work order and remains the one accountable agent at the controller boundary. It must maintain meta/roster.ndjson, meta/progress.json, coordinated shared pacing, and meta/termination.json throughout the state machine. Worker messages carry the PkDE run ID and project-relative run directory.

Before FINALIZE completes, the supervisor runs pkde hypex ingest <run-dir> and pkde hypex analyze <artifact> so the tournament enters PkDE as a Layer 0 pkde.hypex.v1 artifact plus pkde.hypothesis-assessment.v1 analysis. Only those PkDE artifact paths, the final Hypex report path, and the termination summary cross back to the research-operations controller.


State: INIT

Purpose: Set up the run directory and decompose the research goal into focus areas.

Entry Condition

  • Supervisor receives a task message containing a research goal.

Actions

  1. Parse the research goal from the PkDE work order supplied by the dispatching controller. A direct launch may use a command like:

    agent start my-run --type hypex-supervisor "Explore mechanisms of age-related cognitive decline"

    The quoted string is the research goal.

  2. Resolve the artifact path. Check whether the task message contains an override line of the form Artifact path: <path>. If present, set ARTIFACT_PATH to the specified value. If absent, default to the shared volume location:

    ARTIFACT_PATH = executions

    This resolved value is used for the remainder of the run -- in every hypex, elo, and prox invocation, in every worker task message, and in every path reference to the run directory. The default points to the shared scratchpad volume so that all worker agents (which run in independent clones) can read and write the same datastore.

  3. Choose a run ID. Derive a short, descriptive run ID from the goal (e.g., cognitive-decline-01). Use lowercase letters, hyphens, and digits only. Keep it under 30 characters.
  4. Initialize the run directory:

    hypex init-run <run-id> --run-dir "${ARTIFACT_PATH}" --goal "<goal>"

    This creates the standard directory layout at ${ARTIFACT_PATH}/<run-id>/ with subdirectories: hypotheses/, reviews/, matches/, ratings/, proximity/, meta/, quarantine/, report/, and a run.yaml configuration file.

    Verify success by checking the output: Initialized run: ${ARTIFACT_PATH}/<run-id>

  5. Decompose the goal into 3-6 focus areas. Each focus area should be:
    • A specific sub-topic that can be independently explored
    • Narrow enough for a generation agent to cover in one session
    • Broad enough to yield 2-4 hypotheses
    • Balanced so that the set covers the research space without major gaps
  6. Record focus areas by writing them to ${ARTIFACT_PATH}/<run-id>/meta/focus-areas.md.
  7. Initialize budgets:
    • EPOCH_MATCH_BUDGET = 20 (per-epoch, configurable)
    • TOTAL_MATCH_BUDGET = 100 (cross-epoch, configurable)
    • MAX_EPOCHS = 5 (configurable)
    • TOTAL_MATCHES_USED = 0
    • CURRENT_EPOCH = 0

Error Handling

  • hypex init-run fails: Report error and abort the run. Transition to DONE.

Exit Condition

  • Run directory exists with valid run.yaml.
  • 3-6 focus areas are defined.
  • All budgets initialized.

State: GENERATE

Purpose: Start generation workers to explore each focus area and produce hypotheses.

Entry Condition

  • INIT completed successfully (epoch 0), OR
  • META completed and convergence check decided to loop (epoch > 0).
  • Focus areas are defined.
  • If epoch > 0, a steering memo exists from the previous epoch.

Actions

  1. Start 2-3 generation workers -- one per focus area batch. If you have more focus areas than workers, distribute focus areas across workers (e.g., 5 focus areas across 3 workers: 2+2+1).

    For each worker:

    agent start gen-<short-name> --type hypex-generation "<task message>"

    The task message must include:

    • The focus area text
    • The run ID (so the worker can use --run <run-id> with hypex)
    • The run directory path: ${ARTIFACT_PATH}/<run-id>
    • The current epoch number
    • How many hypotheses to produce (2-4)
    • If epoch > 0: The steering memo content (see Steering Memo Injection)
  2. Signal that you are waiting:

    agent-status blocked "Waiting for generation workers to complete"
  3. Wait for workers to complete. Workers report back via agent message with the hypothesis IDs they created and a brief summary.
  4. After all workers complete, verify hypotheses exist:

    hypex list --run <run-id> --run-dir "${ARTIFACT_PATH}" --type hypothesis
  5. Collect hypothesis IDs from worker reports.

Error Handling

  • Worker timeout or crash: Log failure, continue with hypotheses from other workers. Do not abort the entire run.
  • No hypotheses produced: Report failure to the dispatching agent with diagnostics. Transition to DONE.

Exit Condition

  • At least one hypothesis exists in hypotheses/.
  • All worker reports have been received (or timed-out workers are logged).

State: REVIEW

Purpose: Start reflection workers to review and score all hypotheses.

Entry Condition

  • GENERATE completed with at least one hypothesis.
  • All hypothesis IDs are known.

Actions

  1. Collect hypothesis IDs to review:

    hypex list --run <run-id> --run-dir "${ARTIFACT_PATH}" --type hypothesis
  2. Start 1-2 reflection workers depending on hypothesis count:

    • <= 6 hypotheses: 1 worker
    • 6 hypotheses: 2 workers, split the hypothesis list evenly

    For each worker:

    agent start review-<N> --type hypex-reflection "<task message>"

    The task message must include:

    • The list of hypothesis IDs to review
    • The run ID
    • The run directory path
    • The current epoch
    • If epoch > 0: The steering memo content (see Steering Memo Injection)
  3. Signal that you are waiting:

    agent-status blocked "Waiting for reflection workers to complete"
  4. Wait for workers to complete. Workers report back via agent message with review IDs, per-hypothesis summaries, and any quarantine events.
  5. After all workers complete, verify reviews exist:

    hypex list --run <run-id> --run-dir "${ARTIFACT_PATH}" --type review
  6. Check for quarantined hypotheses. Note these for tracking.

Error Handling

  • Worker timeout or crash: Log and continue. Some hypotheses will lack reviews. This is acceptable.
  • No reviews produced: Proceed to DEDUP. Reviews are not required for dedup or tournament (though they inform tier selection).

Exit Condition

  • Worker reports received (or timed-out workers are logged).
  • Proceed to DEDUP regardless of whether all reviews completed.

State: DEDUP (new in v1)

Purpose: Run proximity analysis to identify clusters and near-duplicate hypotheses, then retire duplicates before the tournament.

Entry Condition

  • REVIEW state completed.

Actions

  1. Start the proximity worker:

    agent start prox-worker --type hypex-proximity "<task message>"

    The task message must include the run directory path.

  2. Signal that you are waiting:

    agent-status blocked "Waiting for proximity worker to complete"
  3. Wait for the worker to complete. The proximity worker runs the full pipeline: prox embed, prox graph, prox clusters, prox dupes. It adjudicates borderline pairs (0.80-0.92 similarity) and reports:
    • Cluster labels and member hypotheses
    • Merge recommendations (verdict: MERGE, KEEP, or FLAG)
    • Statistics
  4. Read proximity results:

    cat ${ARTIFACT_PATH}/<run-id>/proximity/graph.json
    cat ${ARTIFACT_PATH}/<run-id>/proximity/clusters.json
  5. Process merge recommendations:
    • For each MERGE recommendation: retire the duplicate hypothesis (set its status to "merged"). By convention, keep the earlier-ID hypothesis.
    • For each FLAG recommendation: note for human review in the summary.
    • Record all decisions in meta/dedup-results.md.
  6. Update the active hypothesis list:

    hypex list --run <run-id> --run-dir "${ARTIFACT_PATH}" --type hypothesis

    Only active (non-merged) hypotheses proceed to TOURNAMENT.

Error Handling

  • Proximity worker crash: Log the failure. Skip DEDUP and proceed to TOURNAMENT without cluster information. elo pair works without a proximity graph -- it just cannot use cluster-guided pairing.
  • No duplicates found: Normal. Proceed to TOURNAMENT with all hypotheses.

Exit Condition

  • Proximity results read (or worker failure logged).
  • Merge recommendations processed (or skipped on failure).
  • Active hypothesis list updated.

State: TOURNAMENT (new in v1)

Purpose: Run pairwise Elo-rated matches to produce competitive rankings.

Entry Condition

  • DEDUP state completed (or skipped on failure).
  • At least 2 active hypotheses exist.
  • Match budget has remaining capacity (both per-epoch and total).

Actions

  1. Compute remaining budget:

    EPOCH_REMAINING = EPOCH_MATCH_BUDGET - EPOCH_MATCHES_USED
    TOTAL_REMAINING = TOTAL_MATCH_BUDGET - TOTAL_MATCHES_USED
    REMAINING = min(EPOCH_REMAINING, TOTAL_REMAINING)
  2. Compute pairings:

    elo pair --run-dir ${ARTIFACT_PATH}/<run-id> --epoch <N> --budget <REMAINING>

    The command outputs a JSON array of {"a": "H-XXXX", "b": "H-YYYY"} objects to stdout.

    Update budgets:

    EPOCH_MATCHES_USED += len(pairings)
    TOTAL_MATCHES_USED += len(pairings)

    If pairings is empty (no valid pairings possible), skip to EVOLVE.

  3. Split pairings across workers:

    • <= 5 pairings: 1 ranking worker
    • 6-12 pairings: 2 ranking workers
    • 12 pairings: 3 ranking workers

    Each worker gets a non-overlapping subset of pairings.

  4. Start ranking workers:

    agent start rank-<N> --type hypex-tournament "<task message>"

    The task message must include:

    • The pairing list (JSON array)
    • The run ID and run directory path
    • The epoch number
    • Instruction to NOT run elo recompute (the supervisor handles this)
  5. Signal that you are waiting:

    agent-status blocked "Waiting for ranking workers to complete"
  6. Wait for ALL ranking workers to complete. This is critical -- ratings must be recomputed from the complete match ledger.
  7. Recompute ratings:

    elo recompute --run-dir ${ARTIFACT_PATH}/<run-id> --epoch <N>
  8. View standings:

    elo standings --run-dir ${ARTIFACT_PATH}/<run-id> --format json

    Save these standings -- you need them for EVOLVE (top-K selection) and convergence checking.

Error Handling

  • Ranking worker crash: Log the failure. Proceed with whatever match records exist. Run elo recompute on the available matches.
  • No match records produced: Skip elo recompute. Fall back to review-score ranking.
  • elo recompute fails: Log the error. Fall back to review-score ranking.
  • Single active hypothesis: Skip TOURNAMENT entirely (no pairings possible). Proceed to EVOLVE.

Exit Condition

  • Ratings computed (or failure logged with fallback plan).
  • Standings available for EVOLVE and convergence checking.

State: EVOLVE (new in v2)

Purpose: Apply evolutionary operators to top-K hypotheses to produce improved variants.

Entry Condition

  • TOURNAMENT state completed with Elo standings available.
  • At least 1 hypothesis has been rated (top-K selection takes as many as available, up to K).

Actions

  1. Select top-K hypotheses (default K=5) from the composite Elo standings:

    elo standings --run-dir ${ARTIFACT_PATH}/<run-id> --composite --format json

    Select the top-K hypotheses (default K=5) by elo_composite. Take the first K entries from the JSON array. Record their hypothesis IDs.

    Operational Note (Mid-run upgrades): Old reviews are never backfilled. A hypothesis reviewed before Phase 3 keeps S_goal = 0.5 until reviewed again. Operational mitigation: do not enable --composite mid-run on a run whose reviews predate Phase 3.

  2. Gather inputs for the evolution agent:
    • Top-K hypothesis IDs
    • Reviews for each hypothesis (read from reviews/)
    • Match-loss rationales: read match files from matches/ where each top-K hypothesis was the loser. Extract the rationale field.
    • Cluster memberships from proximity/clusters.json (if available)
  3. Start the evolution worker:

    agent start evolve-epoch-<N> --type hypex-evolution "<task message>"

    The task message must include:

    • The list of top-K hypothesis IDs
    • The reviews for each hypothesis (summaries with key criticisms and scores)
    • Match-loss rationales for each hypothesis
    • The run ID and run directory path
    • The current epoch number
    • Cluster memberships (if available)
  4. Signal that you are waiting:

    agent-status blocked "Waiting for evolution worker to complete"
  5. Wait for the worker to complete. The evolution worker reports:
    • Variant hypothesis IDs created (e.g., H-0055, H-0056)
    • For each variant: parent ID(s), operator applied, summary
    • Any hypotheses not evolved and why
  6. Record evolution results: Save the variant-to-parent mapping for use in REVIEW_EVOLVED and TOURNAMENT_REMATCH:
    EVOLVED_VARIANTS = {
      "H-0055": {"parents": ["H-0012"], "operator": "ground"},
      "H-0056": {"parents": ["H-0017"], "operator": "simplify"},
      ...
    }

Error Handling

  • Evolution worker crash: Log the failure. Skip EVOLVE, REVIEW_EVOLVED, and TOURNAMENT_REMATCH. Proceed directly to META.
  • No variants produced: Same as crash -- skip to META. Note in meta that evolution produced no variants.

Exit Condition

  • Evolution worker reported completion with variant IDs.
  • Variant-to-parent mapping recorded.

State: REVIEW_EVOLVED (new in v2)

Purpose: Review evolved hypothesis variants using recurrent review mode, which compares each variant against its parent and checks whether prior review criticisms were addressed.

Entry Condition

  • EVOLVE completed with at least one variant.
  • Variant-to-parent mapping is known.

Actions

  1. Collect evolved variant IDs from the evolution worker's report.
  2. Start a reflection worker in recurrent mode:

    agent start review-evolved-<N> --type hypex-reflection "<task message>"

    The task message must include:

    • The list of evolved variant hypothesis IDs
    • The run ID and run directory path
    • The current epoch number
    • recurrent: true -- activates recurrent review mode
    • For each variant: the parent_id (e.g., parent_id: H-0012)
    • The parent hypothesis IDs so the reviewer can read them
  3. Signal that you are waiting:

    agent-status blocked "Waiting for recurrent review worker to complete"
  4. Wait for the worker to complete. The reviewer reports:
    • Review IDs for each evolved variant
    • Per-variant summary including recurrent notes (criticisms addressed, remaining, net quality delta)
    • Any quarantine events

Error Handling

  • Reflection worker crash: Log the failure. Evolved variants proceed to TOURNAMENT_REMATCH without reviews. They will still compete against parents, but without the benefit of reviewer feedback in the record.

Exit Condition

  • Reviews for evolved variants completed (or failure logged).

State: TOURNAMENT_REMATCH (new in v2)

Purpose: Run tournament matches where evolved variants compete against their parents and optionally against each other. Parent rematches are mandatory to validate that evolution produced genuine improvements.

Entry Condition

  • REVIEW_EVOLVED completed (or skipped on failure).
  • Evolved variant IDs and their parent IDs are known.
  • Match budget has remaining capacity (both per-epoch and total).

Actions

  1. Write parent-vs-child pairings to a JSON file. Build the mandatory rematch list and write it to <run-dir>/rematch-pairs.json:

    REMATCH_PAIRINGS = []
    for each variant in EVOLVED_VARIANTS:
      for each parent_id in variant.parents:
        REMATCH_PAIRINGS.append({"a": parent_id, "b": variant.id})

    Example: If H-0055 evolved from H-0012 and H-0056 evolved from H-0017, write:

    [
      {"a": "H-0012", "b": "H-0055"},
      {"a": "H-0017", "b": "H-0056"}
    ]
  2. Generate all pairings with elo pair --rematch. A single call handles both forced rematches and exploration pairings. Forced pairings are prepended and consume budget first; remaining budget goes to strategy-generated pairings. Deduplication is automatic.

    elo pair --run-dir <path> --epoch <N> --budget <REMAINING> --rematch <run-dir>/rematch-pairs.json

    The default --rematch-window 2 applies to strategy-generated pairings (preventing rematches within 2 epochs). Forced --rematch pairings bypass the window -- they are explicitly requested rematches.

  3. Enforce budget:

    REMAINING = min(EPOCH_MATCH_BUDGET - EPOCH_MATCHES_USED, TOTAL_MATCH_BUDGET - TOTAL_MATCHES_USED)

    If len(REMATCH_PAIRINGS) > REMAINING, truncate the pairings JSON file to budget before passing it to elo pair. Parent-vs-child pairings are prioritized because they are prepended by --rematch.

    Update budgets:

    EPOCH_MATCHES_USED += len(all_pairings)
    TOTAL_MATCHES_USED += len(all_pairings)
  4. Split pairings across workers (same rules as TOURNAMENT):
    • <= 5 pairings: 1 worker
    • 6-12 pairings: 2 workers
    • 12 pairings: 3 workers

  5. Start ranking workers:

    agent start rematch-<N> --type hypex-tournament "<task message>"

    The task message must include:

    • The pairing list (JSON array)
    • The run ID and run directory path
    • The epoch number
    • Instruction to NOT run elo recompute
  6. Signal that you are waiting:

    agent-status blocked "Waiting for rematch ranking workers to complete"
  7. Wait for ALL ranking workers to complete.
  8. Recompute ratings (includes all matches from all epochs):

    elo recompute --run-dir ${ARTIFACT_PATH}/<run-id> --epoch <N>
  9. View updated standings:

    elo standings --run-dir ${ARTIFACT_PATH}/<run-id> --format json

    Save these standings for convergence checking and META input.

Error Handling

  • Ranking worker crash: Log the failure. Proceed with available match records. Run elo recompute on available matches.
  • Budget exhausted before rematches: If no budget remains, skip TOURNAMENT_REMATCH entirely. Proceed to META.
  • No match records produced: Log failure. Proceed to META with pre-rematch standings.

Exit Condition

  • Ratings recomputed with rematch results (or failure logged).
  • Updated standings available.

State: META (new in v2)

Purpose: Task the meta-review agent to produce a steering memo that analyzes the current epoch's results and provides guidance for the next epoch.

Entry Condition

  • TOURNAMENT_REMATCH completed (or skipped).
  • Current epoch standings are available.

Actions

  1. Start the meta-review worker in steering-memo mode:

    agent start meta-epoch-<N> --type hypex-meta-review "<task message>"

    The task message must include:

    • Mode: "steering memo"
    • The run directory path
    • The current epoch number
    • A note that the steering memo will be used to guide the next epoch's generation and review workers
  2. Signal that you are waiting:

    agent-status blocked "Waiting for meta-review worker to complete"
  3. Wait for the worker to complete. The meta-review worker:
    • Reads reviews, match records, and standings for the current epoch
    • Reads prior steering memos (if any)
    • Identifies recurring patterns and systemic issues
    • Writes the steering memo to meta/epoch-<N>-steering.md
    • Reports a summary of key findings
  4. Read the steering memo:

    cat ${ARTIFACT_PATH}/<run-id>/meta/epoch-<N>-steering.md

    Save this content -- it will be injected into the next epoch's GENERATE and REVIEW task messages.

Error Handling

  • Meta-review worker crash: Log the failure. Proceed to convergence check without a steering memo. If looping, the next epoch's workers will not receive steering guidance (suboptimal but not fatal).

Exit Condition

  • Steering memo written to meta/epoch-<N>-steering.md (or failure logged).
  • Proceed to convergence check.

Convergence Check (after META)

This is a decision point, not a state with workers. The supervisor evaluates convergence criteria immediately after META completes.

Actions

  1. Check budget exhaustion (hard limits):
    • Is CURRENT_EPOCH + 1 >= MAX_EPOCHS? -> converged
    • Is TOTAL_MATCHES_USED >= TOTAL_MATCH_BUDGET? -> converged
    • Is the wall-clock limit exceeded (if configured)? -> converged
  2. Check top-5 stability (soft limit):
    • Read current epoch standings: top-5 IDs and Elo values
    • Read previous epoch standings (ratings/epoch-<N-1>.json): top-5 IDs and Elo values
    • If epoch >= 2, also check epoch N-1 vs N-2 stability
    • Stability requires: same 5 IDs in top-5, AND all Elo deltas < 30
    • Stable for 2 consecutive epoch transitions -> converged
    • Note: Convergence stability checks use raw elo (without --composite), because composite moves whenever a new review lands and convergence asks "has the tournament stopped learning".
  3. Decision:
    • If converged -> proceed to FINALIZE
    • If not converged:
      • Increment CURRENT_EPOCH
      • Reset EPOCH_MATCHES_USED = 0
      • Loop back to GENERATE with steering memo injection

Recording the Decision

Write the convergence check result to the run metadata:

cat <<'EOF' >> ${ARTIFACT_PATH}/<run-id>/meta/convergence-log.md
## Epoch <N> Convergence Check

- Budget: <TOTAL_MATCHES_USED>/<TOTAL_MATCH_BUDGET> matches used, epoch <N+1> of <MAX_EPOCHS>
- Top-5 stability: <stable/not stable> (delta: <max delta>)
- Decision: <LOOP to epoch N+1 / FINALIZE>
- Reason: <why>
EOF

State: FINALIZE (new in v2)

Purpose: Task the meta-review agent to produce the final research report, then publish the completed run into PkDE's Layer 0 artifact model.

Entry Condition

  • Convergence check decided to finalize.
  • All epochs' data is available (hypotheses, reviews, matches, ratings, steering memos).

Actions

  1. Start the meta-review worker in final-report mode:

    agent start meta-final --type hypex-meta-review "<task message>"

    The task message must include:

    • Mode: "final report"
    • The run directory path
    • The total number of epochs completed
    • A note to include Elo trajectories across all epochs, verified evidence for top-10, and an experimental roadmap
  2. Signal that you are waiting:

    agent-status blocked "Waiting for meta-review worker to produce final report"
  3. Wait for the worker to complete. The meta-review worker:
    • Reads all data across all epochs
    • Computes Elo trajectories
    • Independently verifies evidence for top-10 hypotheses
    • Identifies unresolved contradictions
    • Builds an experimental roadmap
    • Writes the final report to report/final.md
  4. Verify the report exists:

    ls ${ARTIFACT_PATH}/<run-id>/report/final.md
  5. Write meta/termination.json, then publish through PkDE:

    hypex validate --run <run-id> --run-dir "${ARTIFACT_PATH}"
    pkde hypex ingest "${ARTIFACT_PATH}/<run-id>" --json
    pkde hypex analyze <raw-hypex-artifact> --json
  6. Report the final report and PkDE artifact paths to the dispatching agent.

Error Handling

  • Meta-review worker crash: Log the failure. Write a minimal summary report yourself using Elo standings and hypothesis data (ranked top-10 with Elo ratings, review summaries, match counts). Still publish the run and report the limitation to the dispatching agent.

Exit Condition

  • Final report written to report/final.md (or fallback summary written).
  • PkDE artifacts verified and report delivered to the dispatching agent.

State: DONE

Purpose: Signal task completion.

Entry Condition

  • FINALIZE completed.

Actions

  1. Signal completion:

    agent-status task_completed "Hypothesis exploration run <run-id> complete"

Worker Lifecycle Summary

Worker TypeTemplateCountStarted InReports
Generationhypex-generation2-3GENERATEHypothesis IDs, summaries
Reflectionhypex-reflection1-2REVIEWReview IDs, scores, quarantine events
Proximityhypex-proximity1DEDUPClusters, merge recommendations
Rankinghypex-tournament1-3TOURNAMENTMatch IDs, match summaries
Evolutionhypex-evolution1EVOLVEVariant IDs, parent mappings, operators
Reflection (recurrent)hypex-reflection1REVIEW_EVOLVEDReview IDs, recurrent notes, quality deltas
Ranking (rematch)hypex-tournament1-3TOURNAMENT_REMATCHMatch IDs, match summaries
Meta-review (steering)hypex-meta-review1METASteering memo path, key findings
Meta-review (final)hypex-meta-review1FINALIZEFinal report path, summary

All workers are started with agent start <name> --type <template> and report back via agent message. The supervisor uses agent-status blocked while waiting for each group of workers.


Error Handling Summary

FailureActionAbort Run?
Worker timeout / crashLog failure, continue with remaining workersNo
No hypotheses produced by any workerReport failure with diagnosticsYes
Some hypotheses lack reviewsInclude as "unreviewed" in summaryNo
No reviews produced by any workerProceed to DEDUP/TOURNAMENT without reviewsNo
Proximity worker crashSkip DEDUP, proceed to TOURNAMENTNo
Ranking worker crashRecompute from available matchesNo
No match records producedFall back to review-score rankingNo
elo recompute failsFall back to review-score rankingNo
hypex init-run failsReport error and abortYes
Per-epoch budget exhaustedSkip TOURNAMENT, proceed to EVOLVENo
Total budget exhaustedRecord budget_exhausted, proceed to FINALIZENo
Only 1 active hypothesis after DEDUPSkip TOURNAMENTNo
Evolution worker crashSkip EVOLVE/REVIEW_EVOLVED/TOURNAMENT_REMATCH, go to METANo
No evolved variants producedSkip REVIEW_EVOLVED/TOURNAMENT_REMATCH, go to METANo
Recurrent review worker crashProceed to TOURNAMENT_REMATCH without evolved reviewsNo
Rematch budget exhaustedSkip TOURNAMENT_REMATCH, go to METANo
Meta-review worker crash (steering)Loop without steering memo (suboptimal)No
Meta-review worker crash (final)Write fallback summary, report to dispatching agentNo

Epoch Flow Summary

Epoch 0:
  INIT -> GENERATE -> REVIEW -> DEDUP -> TOURNAMENT ->
  EVOLVE -> REVIEW_EVOLVED -> TOURNAMENT_REMATCH ->
  META -> [check convergence] -> not converged -> loop

Epoch 1:
  GENERATE (with steering memo) -> REVIEW (with steering memo) ->
  DEDUP -> TOURNAMENT -> EVOLVE -> REVIEW_EVOLVED ->
  TOURNAMENT_REMATCH -> META -> [check convergence] -> not converged -> loop

Epoch 2:
  GENERATE (with steering memo) -> REVIEW (with steering memo) ->
  DEDUP -> TOURNAMENT -> EVOLVE -> REVIEW_EVOLVED ->
  TOURNAMENT_REMATCH -> META -> [check convergence] -> converged (top-5 stable) ->
  FINALIZE -> DONE

OR:

Epoch 4 (max epochs):
  ... -> META -> [check limits] -> budget_exhausted ->
  FINALIZE -> DONE

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.