agent-framework-harness

Quick decision table

Agent Harness is runtime scaffolding, not hosting. Native approval provides interactive UX and session-backed standing-rule convenience; it is not non-bypassable policy. ResponsesHostServer adapts an existing Agent and does not create the Harness composition.

This skill excludes deployment, RBAC, identity grants, containers, registries, azure.yaml, rollout, service lifecycle, deterministic governance, audit ownership, sandbox ownership, Foundry Skills REST distribution, and general evaluation design.

Architecture and internal factory pipeline

create_harness_agent returns a normal Agent Framework Agent around any compatible chat client. Preserve this exact context-provider order:

1. history;
2. optional after-call CompactionProvider;
3. todo;
4. AgentModeProvider;
5. FileMemoryProvider;
6. optional FileAccessProvider;
7. optional SkillsProvider;
8. optional BackgroundAgentsProvider;
9. optional shell provider;
10. caller context providers.

Before-call compaction is not a context provider: it is attached to agent.compaction_strategy. When both token budgets create the shared strategy, the factory assembles the before-call strategy and after-call CompactionProvider phases together.

Preserve this tool order:

1. conditional web-search tool when the client implements SupportsWebSearchTool and web search is not disabled;
2. optional shell tool when shell_executor is supplied and the client implements SupportsShellTool;
3. caller-supplied tools.

Preserve this middleware order, outermost first:

1. optional AgentLoopMiddleware;
2. default ToolApprovalMiddleware;
3. always-on MessageInjectionMiddleware;
4. caller middleware.

Python 1.14.0 accepts either one caller middleware object or a sequence through the middleware parameter; both forms remain appended after the built-ins.

The default history provider is InMemoryHistoryProvider. Per-service-call history persistence and provider-owned state are session-backed, but neither is durable across process loss unless the host persists the full AgentSession. Pending tool approval stops autonomous progress and returns control to the caller. The loop is outermost, so each iteration is a complete run through history, providers, approval, and telemetry.

Default, opt-in, and experimental feature matrix

Compaction activation

Supplying both max_context_window_tokens and max_output_tokens creates one shared default ContextWindowCompactionStrategy. The before-call phase is assigned to agent.compaction_strategy; the after-call phase is installed through CompactionProvider. A custom before-only strategy can independently populate agent.compaction_strategy, and a custom after-only strategy can independently add CompactionProvider. disable_compaction=True overrides default and custom strategies. Supplying only one token budget does not create the shared default strategy.

Released Python 1.14.0 still invokes after-run providers on every AgentLoopMiddleware iteration. Upstream issue #7236 is fixed on main but not in 1.14.0. Keep explicit loop caps and revalidate after-run compaction when the next stable core release lands.

Canonical local Python recipe

MUST: Copy or adapt references/python/local_harness.py. Do not reproduce its functions inline.

Keep both token budgets so compaction is active, the explicit .agent-memory store, disabled web search, and the AgentSession passed to agent.run. Any compatible chat client can replace FoundryChatClient. Foundry constructs an async AIProjectClient, so use an async Azure credential and close it through an async context manager or an equivalent host-owned lifecycle.

Canonical Foundry Hosted recipe

MUST: Copy or adapt references/python/hosted_harness.py, then use foundry-hosted-agents for deployment, RBAC, containers, rollout, and lifecycle. Do not reproduce the reference functions inline.

Keep default_options={"store": False} because the hosting adapter owns Responses transcript history. The pinned ResponsesHostServer uses the Agent Server Responses 2.x provider-backed session, checkpoint, and function-approval storage model and still rejects a history provider that loads messages. Therefore the Harness must receive the reference's no-load/no-store InMemoryHistoryProvider (load_messages=False, store_inputs=False, and store_outputs=False). FoundryChatClient constructs an async AIProjectClient, so use an async Azure credential. The reference's internally created DefaultAzureCredential is process-lifetime and its handle is not exposed. A host requiring deterministic close must create and inject its own async credential, retain that handle, and close it.

The baseline also disables mode, file memory, and web search. Re-enable mode only when the protocol transports explicit plan approval and transitions. Re-enable file memory only after choosing durable storage, authenticated tenant partitioning, and path policy.

The maintainer-approved CI path executes this skill's SDK smoke directly in the runner, without a Copilot process. It injects an async AzureCliCredential for the workflow's subscription. Its private AZURE_CONFIG_DIR is created before azure/login@v2, which binds the tenant and subscription; the SDK uses that runner-owned login. Do not pass both tenant and subscription to the CLI credential's token request: Azure CLI rejects that combination. This is a CI credential handoff, not a new role or permission grant. The workflow also provisions the pinned interpreter under the approved workspace's .scratch/ directory. The fixture executes only test-fixture/live_smoke.py, which rejects a substituted interpreter or dependency cohort and writes the marker only after its live assertions. The virtual environment uses copied executables, not symlinks to a tool-cache path outside the approved workspace; path verification remains enabled. The receipt is a format/integrity check, not proof of execution provenance. The mandatory harness-native step's outcome is authoritative: that step performs the actual Azure call, and no agent can replace it by writing files. This proves SDK execution, not Copilot instruction-following behavior.

Do not infer service maturity from adapter semver or adapter maturity from service status.

Safe plan-to-execute pattern

MUST: Use references/python/plan_execute.py. Do not collapse it into one agent or reproduce its functions and classes inline.

Use separate plan and execute agents because the default mode_set tool is model-accessible and auto-approved. The planning agent must expose neither mode_set nor looping, so it cannot switch itself into execution. Share the history provider, todo provider, file-memory store, and full session to preserve state continuity.

The host callback is the plan-approval boundary: persist the session, display the plan, obtain explicit caller approval, and only then set execute mode. The execute agent alone receives an execute-scoped loop with an explicit positive cap. Never use loop_max_iterations=None.

Inspect the structured response before reading .text. A function_approval_request must return caller control as the full AgentResponse, as represented by ToolApprovalRequired; reducing it to .text discards the pending approval.

Session persistence and recovery

MUST: Copy or adapt references/python/session_recovery.py. Do not reproduce its helper bodies inline.

Persist and restore the full opaque AgentSession, not only transcript text. Restore it with the same agent, providers, middleware, and stores. Partition durable state by authenticated tenant and user; authorize the session identifier before loading; reject caller-supplied serialized state; and never mutate provider-owned keys directly.

Full AgentSession persistence restores serializable session, provider, and middleware state, including history, approval, mode, and todo state. It does not serialize provider-instance runtime or external stores. BackgroundAgentsProvider child sessions and running asyncio tasks are non-serializable process memory; after restart, running tasks are LOST. Python 1.14.0 resolves issue #7385 with await provider.release_session(session), whose default cancellation path is bounded to 30 seconds. Long-lived hosts must invoke that API when a session is evicted.

File-memory and other external stores must be made separately durable and partitioned by authenticated tenant and user. Assign one owner to transcript persistence: in the hosted recipe, keep store=False and the no-load/no-store Harness history provider so the adapter remains the owner.

Offline contract and pin validation

MUST: Run references/python/test_harness_contract.py with the packages and validation procedure in references/upstream-pin.md. Do not reproduce the smoke-test bodies inline.

The smoke validates construction, signature, defaults, provider and middleware order, compaction phases, hosted imports, and plan/execute composition without credentials, network access, or a model call. Import-only success is insufficient.

Harness vs Hosted Agents vs AGT

Harness Agent -> AGT middleware/policy -> ResponsesHostServer
              -> Foundry hosted runtime, identity, scale, and lifecycle

Failure modes and security callouts

Security boundaries — do not weaken these statements:

• Deny lists and a constrained working directory are not a sandbox.
• File access and file memory require tenant partitioning and host-owned path policy.
• Background agents multiply credentials, tools, cost, and cancellation obligations.
• Loop caps bound iterations and cost; they do not establish safety.
• Native approval is bypassable by host design and is not deterministic authorization.
• Deterministic governance belongs to foundry-agt.

Upstream pin and reference policy

• Record agent-framework 1.14.0 for release context, but the agent-framework meta-package MUST NOT be used in canonical hosted dependency sets; canonical hosted dependencies use component packages.
• Use compatible stable pins for agent-framework-core~=1.14.0, agent-framework-foundry~=1.11.0, and azure-identity~=1.25.3.
• Pin agent-framework-foundry-hosting==1.0.0b260813, its Agent Server beta dependencies, and optional agent-framework-tools==1.0.0b260730 exactly because they are prerelease surfaces.
• Preserve the immutable audited source evidence: tag python-1.14.0 at SHA ae7fa3389c8f70b3ed702b0e04b85a3ee62b1bd1.
• Track the Hosted Agents service lifecycle separately from the Python hosting adapter lifecycle.
• On every MINOR refresh, reverify web-search capability detection and empty approval-rule behavior.
• Before using experimental file access, background agents, looping, or shell, inspect the open issues recorded in references/upstream-pin.md.
• Validate signatures, resolved defaults, construction, and provider/middleware ordering; an import-only check is not sufficient.
• Keep non-trivial Python code single-sourced under references/python/; SKILL.md provides imperative links and must not duplicate canonical bodies.

Completion checklist

• One-minute routing distinguishes Harness, Hosted Agents, AGT, Skills REST, and eval ownership.
• Exact pinned defaults and provider/middleware order are verified.
• Compaction is described as conditional on both budgets or explicit custom phases.
• Web search is described as conditional on client capability.
• Every autonomous recipe uses an explicit positive loop cap.
• Native approval is identified as UX rather than authorization or governance.
• Deny lists and working-directory confinement are identified as non-sandbox controls.
• Experimental and prerelease features are visibly labeled.
• All canonical local, hosted, plan/execute, recovery, and offline files are linked without duplicated bodies.
• Hosting adapter and Hosted Agents service statuses remain separate.
• Offline smoke proves signature, defaults, construction, and ordering without a model call.
• Current-head Azure T3 evidence exists for the approved runner-native smoke; offline checks are not a substitute.
• No deployment, RBAC, lifecycle, Skills REST, AGT policy, or general eval ownership leaks into this skill.