NVIDIA paper: frequent context compaction costs almost nothing — Opus 4.5 even rises from 70.0 to 74.8 on SWE-bench
omarsar0 · x · 2026-11-04
A NVIDIA paper, "You don't need long context," argues agents don't need full transcript histories.
- On frontier open-weight models, later tokens cost far more to generate: growing context from 16K to 256K raises per-1M-output-token cost up to 7x (Qwen 3 Coder Next), 5.6x for Kimi K2.6 and 3.6x for GLM 5.2 — a gap hidden by flat retail token pricing.
- Frequent context compaction, where the agent carries a short state summary instead of the full transcript, loses little to no accuracy. Claude Opus 4.5 with one action per context scores 70.2 on SWE-bench Verified vs 70.0 with full history, while median peak context drops from 32K to 8K tokens.
- Counterintuitively, simple compaction at the end of a conversation beats full history, lifting Opus 4.5 from 70.0 to 74.8.
- The pattern holds on Terminal-Bench, FRAMES, BFCLv3 and multilingual tasks; the authors outline "semantic state space models" (SSSMs), questioning the need for long-context LLMs.
Related event: NVIDIA Paper Argues Agents Don't Need Long Context(2 posts)→
More from coding & agent
- Living Brain tracks 3 health scores to stop AI agents relying on stale memory — cneuralnetwork · 2026-11-04
- Open-source skill makes Claude Code ship polished, notarized macOS apps in one shot — vista8 · 2026-11-04
- Power user tallies 12B coding-agent tokens, finds 96% were cache reads — peakpirate007 · 2026-11-04
- Indian code-mixed calls have no single correct transcript: LLM, CRM and compliance each need a different one — BEYOND__7 · 2026-11-04
- Alexey Grigorev releases free kit of 28 prompts to ship an app from spec to production — Al_Grigor · 2026-11-04
- Hugging Face Hub MCP Server lets agents search models and datasets via MCP — modelcontextprotocol · 2026-11-04