ArXiv

Self-Compacting Language Model Agents

2026-06-22 · 16:08 UTC ·Tianjian Li, Jingyu Zhang, William Jurayj... ·1 min read
Authors
Tianjian Li, Jingyu Zhang, William Jurayj...
Categories
cs.CL
arXiv
https://arxiv.org/abs/2606.23525v1
PDF
https://arxiv.org/pdf/2606.23525v1

Brief

SelfCompact is a runtime scaffold that lets language-model agents call a compaction tool under guidance from a simple rubric that specifies when to compact or suppress compaction. Evaluated on six benchmarks and seven models (ArXiv 2026-06-22), it matches or outperforms fixed-interval summarization while cutting per-question token costs 30–70% and improving accuracy up to 18.1 points on math.

Why it matters

SelfCompact pairs an inference-time compaction tool with a lightweight rubric that signals when to fire (e.g., sub-task resolved or trajectory converging) and when to suppress (e.g., mid-derivation or stuck), enabling adaptive compaction without fine-tuning or external supervision.

Key details

  • Across six benchmarks (competitive math and agentic search) and seven models, SelfCompact matches or exceeds fixed-interval summarization while reducing per-question token cost by 30–70%, yielding improvements up to 18.1 points on math and 5–9 points on agentic search.
  • The work exposes a meta-cognitive gap: unprompted models unevenly invoke compaction or fail to detect 'rotten' context, but a lightweight rubric closes this gap and elicits reliable compaction behavior without retraining.
Source evidence

Abstract

Long agent traces composed of chains of thought and tool calls accumulate stale content that anchor subsequent generations, and eventually outgrow the context window. Existing scaffolds mitigate it with fixed-interval compaction triggered at a token threshold. Such triggers pay no heed to trajectory structure, risking discard of partial results mid-derivation or mid-search. We propose SelfCompact, a scaffold that allows the model itself to decide when and how to compact. Specifically, it pairs two inference-time elements: (i) a compaction tool the model invokes to summarize the accumulated context, and (ii) a lightweight rubric specifying when to fire (a sub-task has resolved, or the trajectory is converging) and when to suppress (mid-derivation, or when stuck). Both are needed. The tool alone is unevenly used across open-weight models, often invoked at unhelpful moments or not at all; the rubric alone cannot act. Together, they elicit effective adaptive compaction without any fine-tuning or external supervision. We present empirical results on six benchmarks (competitive math and agentic search) and seven models. Our results show that SelfCompact matches or exceeds fixed-interval summarization at a fraction of the token cost, improving over a no-summarization baseline by up to 18.1 points on math and 5-9 points on agentic search at 30-70% lower per-question cost. Our results expose a meta-cognitive gap: although unprompted models cannot reliably tell when their own context is rotting, a lightweight rubric closes this gap, reframing when to compact as a capability that scaffolds can supply without training.

Comment: 25 pages, 3 figures