Agent Improvement Loop: Traces → Feedback → Evals → Ranked Harness Changes → Codegen

This skill gives you the operational flywheel from OpenAI's Agents SDK cookbook for continuously improving a deployed agent: run the agent to produce traces, attach human and model feedback to specific traces, compile that feedback into reusable Promptfoo evals, score the current harness against them as a validation gate, run a HALO optimization pass to rank the next harness changes with evidence, and write a handoff file a codegen agent (Codex) can implement. Each stage keeps what the previous one learned: traces record the agent's actual behavior, feedback pins down which behaviors mattered, evals turn those expectations into checks you can re-run, and the ranked change set gives the codegen agent something concrete to build.

When to use

Use this loop when:

• An agent is running (or runnable) end-to-end and you have, or can generate, real execution traces.
• You are deciding which harness change to make next and want the decision ranked by trace evidence rather than intuition.
• Reviewers or users have given feedback on specific runs and you need to convert it into regression checks before changing anything.
• You are setting up a recurring improve-and-ship cadence for an agent already in production.

Do NOT use this loop when:

• There is no runnable agent yet — the loop needs traces as its raw material.
• You only need a one-off prompt tweak with no intent to keep the resulting evals.
• You cannot run live model calls: the loop is live-only by design; mocked or cached outputs demonstrate a scripted preview, not the actual loop.

Rules

1. Define the harness as one versioned unit before optimizing anything. The harness is everything wrapped around the model — its instructions, tool set, routing, output requirements, and validation checks. Optimize it as an integrated, versioned configuration (e.g. an AgentConfig with version, system_prompt, model_settings, tool_policy, eval_metadata), not as isolated prompt strings, so every recommendation and diff targets a known version.
2. Make the tool policy explicit and machine-checkable. Encode allowed_data_root, writable_output_root, evidence_preference (priority order for conflicting sources), unknown_handling (rules for missing evidence), and mutation_policy (allowed/forbidden file modifications). Give the agent runtime validators (in the cookbook: check_evidence_coverage.py and validate_output_contract.py) so contract violations are caught during the run, not during review.
3. Generate traces on a fixed, representative question set. The cookbook uses a 10-question bank with TRACE_LIMIT = 5 and DEFAULT_TRACE_INDICES = [0, 1, 2, 4, 6]; budget ~20 minutes for a full loop at that scale.
4. Record trace metadata that survives handoff. Each trace record carries trace_id (SHA256-derived 32-char hex), sdk_trace_id (the same hex with a trace_ prefix), trace_label (human-readable, e.g. "trace-01"), question, answer, output_dir, and mode. Export spans as OpenTelemetry-style JSONL (one line per SDK span, with attributes like inference.observation_kind, llm.model_name, token counts) via the HALO trace processor to a local path such as halo_traces/traces.jsonl — this file is what the optimizer reads.
5. Tie every piece of feedback to a specific trace. Human feedback records use feedback_id, trace_id, trace_label, source_type: "human_feedback", theme, summary, required_observations, and prohibited_claims. Model-generated insight records use insight_id, trace_id, source_type: "llm_insight", and observations of recurring behavior. Keep feedback grounded in trace evidence; generic expectations with no trace behind them do not convert into useful evals.
6. Compile feedback into structured evals, not prose. Each generated eval carries eval_id, title, scoring_method (deterministic | llm_judge | hybrid), expected_behavior, source_trace_id, rubric, deterministic_assertions (Promptfoo types contains / icontains / not-contains), plus suggested_pass_example and suggested_fail_example. Use deterministic assertions for literal facts (e.g. contains: "11 months", not-contains: "more than 12 months") and an LLM rubric for semantic requirements; hybrid combines both.
7. Gate with Promptfoo against pre-computed outputs — do not re-run the agent to score it. Use a file-based provider (file://trace_output_provider.py with a trace_outputs_path) that serves each trace's stored answer back to Promptfoo. Set llm-rubric assertions with threshold: 0.8 and a dedicated judge model. Read total / passed / failed, and per-eval score and reason — the judge's reason on failures is direct input for the next optimization pass.
8. Feed the whole loop — traces, feedback, eval results, current harness config — into HALO and take ranked output. HALO (context-labs/HALO, installed with pip install openai openai-agents halo-engine) returns a ranked list of recommended harness changes, the evidence linking each change to traced failures, and implementation guidance for codegen. Do not cherry-pick changes by gut feel when a ranked, evidence-linked list exists.
9. Write the handoff as a single artifact: codex_handoff.md. Include everything the codegen agent needs to make the next harness update: HALO's diagnosis, the ranked recommendations, the supporting evidence, and implementation guidance. This file is the loop's carry-forward memory — future iterations reference it instead of re-running earlier stages.
10. Start with a reviewed loop; automate only after the gate earns trust. Per the cookbook, begin where "the system proposes the change set and a developer approves the diff before merge." The same handoff can also feed a workflow that opens, merges, and deploys pull requests automatically — connect that deeper automation only as the eval gate becomes more trusted.
11. Split model assignments per stage to control cost. Configure via env vars: OPENAI_AGENT_MODEL (agent execution), OPENAI_ANALYSIS_MODEL (feedback/insights), and OPENAI_EVAL_GENERATION_MODEL, OPENAI_JUDGE_MODEL, OPENAI_HALO_MODEL (each defaults to the analysis model). Keep the agent model high-quality; use cheaper models for analysis stages. Pin Promptfoo (PROMPTFOO_VERSION = "0.121.9" in the cookbook); Node.js with npx on PATH and OPENAI_API_KEY are prerequisites.
12. Human-tighten generated evals before they steer repeated optimization. "Even when eval generation is automated, humans can still tighten weak evals before letting them steer repeated optimization" (OpenAI cookbook). A weak or overfit eval that enters the gate will bias every later HALO pass toward satisfying it.

Checklist

Per iteration of the loop:

• Harness config versioned as one unit (instructions, tool policy, output requirements, validation checks).
• Agent run on the fixed representative question set; one trace record per run; all required output artifacts written.
• Traces exported as OTel-style JSONL to the HALO trace path.
• Human feedback recorded per trace: theme, summary, required_observations, prohibited_claims.
• LLM insights generated per trace and stored with source_type: "llm_insight".
• Feedback compiled into Promptfoo evals with scoring_method, assertions, rubric, and source_trace_id.
• Weak/overfit evals tightened by a human before entering the gate.
• Promptfoo gate run against stored trace outputs (file provider, no agent re-execution); llm-rubric threshold 0.8.
• Failure reason strings from the judge collected.
• HALO pass run over traces + feedback + eval results + current harness config.
• codex_handoff.md written: diagnosis, ranked recommendations, evidence, implementation guidance, harness version.
• Codegen agent (Codex) applies the change set; developer reviews the diff before merge (until the gate is trusted enough to automate).
• Artifacts preserved so the next iteration builds on this one instead of re-running earlier stages.

Anti-patterns

• Eval overfitting. Generating evals so specific to one trace that they don't transfer. Include pass/fail examples and prefer requirements that state the expected behavior, not restatements of one answer.
• Letting weak evals steer optimization. Unreviewed auto-generated evals feeding the gate compound: HALO then optimizes the harness toward bad targets.
• Skipping the review stage. Jumping straight to auto-merged harness changes before the eval gate has earned enough trust to stand in for a developer's diff review.
• Mocking or caching loop stages. Every stage (trace generation, feedback, eval synthesis, gate scoring, HALO analysis) runs on fresh model outputs; a mocked loop validates nothing about the real system.
• Re-executing the agent to validate it. The gate scores stored trace outputs through a file provider; re-running conflates generation variance with harness quality and doubles cost.
• Ungrounded feedback. Feedback not tied to a trace_id with specific observed behavior produces generic evals that can't localize failures.
• Treating harness pieces separately. Editing the system prompt while ignoring tool policy, routing, output contracts, and validators breaks the versioned-unit assumption every recommendation depends on.

Source

• Build an Agent Improvement Loop with Traces, Evals, and Codex — https://developers.openai.com/cookbook/examples/agents_sdk/agent_improvement_loop — 2026-05-12

Distilled from the official document(s) above on 2026-08-12. If this skill and the source disagree, trust the source.