‼️The Bitter Lesson for context management: Giving LMs unrestricted control over their context…
Summary
Rulin Shao shows that Context Language Models (CLMs) outperform human-designed context management by treating context as an editable file that the model manipulates natively.
CLMs treat context as a file.
The live context is mirrored as an editable file, allowing CLMs to perform arbitrary manipulations like writing loops to prune irrelevant results or defining functions to compact data.
Treat context as a file
Unrestricted control beats human design.
By allowing models to manage their own context, CLMs outperform specialized baselines like OpenEvolve on math and achieve better accuracy on deep research tasks.
Giving LMs unrestricted control over their context beats human-designed SOTA
Online RL internalizes management skills.
Stepwise GRPO with a success-gated efficiency advantage improves Qwen3.5-9B on BrowseComp-Plus by 47.6% while reducing FLOP usage by 12%.
CLMs can also explore and internalize better context management strategies into weights through online RL
Arbitrary edits increase serving costs.
Frequent context edits challenge prefix-cache reuse; Shao introduces Prefix-Reuse FLOPs (PF) to measure this, though Suffix Cache Reuse helps mitigate the impact.
Arbitrary context edits challenge prefix-cache reuse in serving systems
ContextBench isolates management capabilities.
The benchmark evaluates selective verbatim retention, in-place surgical editing, and offloading/retrieval to reveal where existing baselines fail under context pressure.
ContextBench, a diagnostic benchmark that isolates context-management capabilities
Highlights
Alongside this, we present ContextBench, a diagnostic benchmark that isolates context-management capabilities in selective verbatim retention, in-place surgical editing, and offloading & retrieval.
We also develop Suffix Cache Reuse to further reduce PFLOPs without hurting performance.
- outperform Codex summary on 12hr EdgeBench and 24hr AgentWorld - beat specialized OpenEvolve on math problems - achieve the best performance at lower cost
It improves Qwen3.5-9B on BrowseComp-Plus by 47.6% while using 12% fewer FLOPs.
- Natively manage their own context - Treat context as a file - Learn policies in CLM weights, no harness
Read the original ↗Save to your Sponge
Summarized with Sponge. Quotes are the author’s own words.