Context Compression for Long-Horizon Agents (ACON)

This skill distills Microsoft Research's ACON (Agent Context Optimization) framework into an operating procedure for compressing agent context on long-horizon tasks. Applied correctly, this class of compression cut peak token usage by 26-54% across AppWorld, OfficeBench, and Multi-objective QA while largely preserving task success, and lifted smaller-LM agents by up to 46% by removing context distraction. Applied naively, it destroys performance: on AppWorld, a generic "summarize this" compressor scored 43.5% task accuracy where an optimized-guideline compressor scored 56.5% (no-compression baseline: 56.0%).

When to use

Use this skill when:

• An agent loop accumulates long histories of actions and observations across many steps (tool outputs, API responses, file dumps) and context growth is raising cost or degrading decisions.
• Tool/environment observations are verbose and noisy (raw API payloads, scraped pages, logs) — the highest-value compression target.
• You are running a smaller LM as the agent. Compression helps small models most: in ACON's experiments, compressed context improved smaller-LM agents by 32% (relative) on AppWorld and 46% on Multi-objective QA, because "excessively long contexts dilute relevant information, distracting the model with outdated or extraneous details" (ACON paper).
• You need to write or tune the prompt/guideline for a summarizer that sits inside an agent loop.

Do NOT use when:

• The task is single-step or short-horizon — context never approaches the trigger thresholds, and the compressor call is pure overhead.
• The task is information-dense and every fine-grained detail may be load-bearing. ACON's own results show a real accuracy tax on such tasks: OfficeBench dropped from 76.84% to 72.63% under aggressive (utility + compression-maximized) compression, and Multi-objective QA exact match fell from 0.366 to 0.336 even as peak tokens fell 54.5%. If accuracy is sacrosanct and budget allows, keep full context.

Rules

1. Compress two things separately: observations and history. ACON defines distinct compressors for (a) each environment observation as it arrives and (b) the accumulated interaction history. They have different guidelines and different triggers. Do not use one generic summarizer for both.
2. Trigger compression by threshold, never unconditionally. Apply observation compression only when the observation length exceeds a threshold T_obs; apply history compression only when history length exceeds T_hist. ACON's experiments used T_obs = 1024 tokens and T_hist = 4096 tokens, and ablations found these moderate values give the best trade-off — accuracy close to no-compression while still cutting peak tokens substantially. Smaller thresholds compress more often and degrade accuracy; larger thresholds preserve accuracy but cost more.
3. Compress with an LLM following an explicit natural-language guideline, not with heuristics. The guideline tells the compressor what to keep and what to drop. ACON's guideline-driven compressor beat FIFO (keep the most recent k interactions), embedding-based retrieval, LLMLingua, and naive summarization prompting on task success. The naive-prompt gap alone was 43.5% vs 56.5% on AppWorld — the guideline is where the performance lives.
4. Optimize the guideline from contrastive failures, not from intuition. The ACON loop: run tasks with full context and with compressed context; collect the pairs where the agent succeeds with full context but fails with compressed context; have a capable LLM (ACON used OpenAI o3 as the optimizer) analyze each failure to identify what information the compression destroyed; update the guideline from that feedback. Generate multiple candidate guidelines, evaluate each on the contrastive task set, and keep the best performer. This is optimization in natural-language space — no fine-tuning, works with proprietary agent models.
5. Run utility maximization first; treat compression maximization as an optional second pass. ACON's two stages: the utility step (ut) fixes information loss using success/failure pairs — it is the safer step. The compression step (co) then conditions on tasks that succeeded with compressed context and asks which information was actually used during execution, pushing toward shorter outputs. The co step can slightly hurt accuracy depending on the environment (it helped on AppWorld's noisy API outputs: 51.2% ut-only vs 56.5% ut+co; it cost accuracy on OfficeBench). If your task punishes lost detail, stop after ut.
6. Budget the optimization, not just the inference. Guideline optimization itself cost under $2 per benchmark with o3; the dominant cost is collecting trajectory rollouts (roughly $20 for the AppWorld training set). A one-time spend of tens of dollars is the price of a compressor that does not silently break the agent.
7. Distill the optimized compressor into a small model to kill runtime overhead. ACON distilled the gpt-4.1 compressor (with its optimized guideline) into Qwen3-14B, Qwen3-8B, and Phi-4 via LoRA fine-tuning with cross-entropy loss on teacher input-output pairs — (h_t, h't) for history, ((h{t-1}, o_t), o'_t) for observations — using only trajectories where the teacher agent succeeded with compressed context. Distilled compressors retained over 95% of the gpt-4.1 compressor's performance. Distill only after the guideline is optimized; you are freezing the guideline's behavior into weights.
8. Judge compression by end-task success, never by compression ratio or summary quality. The only valid metric is: does the agent still complete the task with the compressed context? ACON's entire optimization signal is task success/failure on paired trajectories. A fluent summary that drops one API response ID is a failed compression.
9. Expect asymmetric payoff by environment. Verbose, redundant environments (AppWorld-style noisy API outputs) compress nearly for free — ACON matched or beat the no-compression baseline there (56.0% → 56.5%) while cutting peak tokens over 25%. Detail-dense environments (OfficeBench ~38% token cut, Multi-objective QA 54.5% cut) trade a few accuracy points for the savings. Decide per-environment whether that trade is acceptable.
10. If the agent model is small, compress even when budget is not the concern. For smaller LMs, compression is a performance intervention, not just a cost one — mitigating context distraction improved smaller-LM agents up to 46% (Multi-objective QA). For large frontier agents, compression is primarily a cost/efficiency intervention with roughly neutral-to-slightly-negative accuracy impact.

Checklist

Before shipping a compressor into an agent loop:

• Observations and history are compressed by separate compressors with separate guidelines.
• Compression is threshold-gated: only observations > T_obs (start: 1024 tokens) and history > T_hist (start: 4096 tokens) are compressed.
• The compressor is an LLM following an explicit written guideline — not FIFO truncation, not embedding retrieval, not a bare "summarize" prompt.
• You collected contrastive pairs: tasks that succeed with full context but fail with compressed context.
• A strong optimizer model analyzed those failures and rewrote the guideline; multiple candidate guidelines were generated and scored on the contrastive set; the best was kept.
• Utility-maximization (ut) pass done first; compression-maximization (co) pass added only after verifying it does not cost accuracy in your environment.
• Evaluation metric is end-to-end task success with compression vs without — not compression ratio, not summary fluency.
• Peak-token reduction measured (expect 26-54% range depending on environment verbosity).
• If runtime compressor cost matters: distilled into a small model (LoRA on teacher pairs from teacher-success trajectories only) and verified ≥95% of teacher compressor performance retained.
• For detail-critical tasks: explicitly signed off on the measured accuracy delta (ACON saw ~4 points on OfficeBench) or reverted to ut-only / no compression.

Anti-patterns

• Compressing everything, every step. Unconditional compression adds compressor-call overhead and degrades accuracy. Gate by length thresholds; moderate thresholds (1024 obs / 4096 hist) beat aggressive ones in ablation.
• Naive summarization prompts. A generic instruction cost 13 accuracy points vs an optimized guideline on AppWorld (43.5% vs 56.5%). The guideline content is the product; invest in it.
• Heuristic truncation as a substitute. FIFO recency windows, embedding retrieval, and LLMLingua-style extractive compression all underperformed guideline-optimized LLM compression on task success in ACON's comparisons.
• Optimizing the guideline by intuition. Guessing what to preserve misses the actual failure causes. Only contrastive success/failure trajectory analysis reveals which dropped information broke the agent.
• Maximizing compression ratio. The co step exists, but the paper is explicit that it can hurt accuracy depending on the environment. Never chase token savings past the point where the contrastive set shows failures.
• Distilling before optimizing. Distillation freezes the teacher's behavior. Distill only the optimized compressor, and train only on trajectories where the teacher succeeded with compressed context.
• Declaring victory on token counts. Peak-token reduction without an end-task accuracy comparison is meaningless; ACON reports both, always, per benchmark.

Source

• ACON: Optimizing Context Compression for Long-horizon LLM Agents (Microsoft Research; Kang, Chen, Han, Inan, Wutschitz, Chen, Sim, Rajmohan; ICML 2026) — https://www.microsoft.com/en-us/research/publication/acon-optimizing-context-compression-for-long-horizon-llm-agents/ — 2026-06
• Full paper (arXiv 2510.00615) — https://arxiv.org/abs/2510.00615 — v3, 2026-06 (first posted 2025-10)

Distilled from the official document(s) above on 2026-08-12. If this skill and the source disagree, trust the source.