847 agent runs show instruction-following falls from 94% to 41% as context fills
Unique-Werewolf-2784 · reddit · 2026-09-23
A developer tracked 847 agent runs and found instruction-following starts at 94% but collapses to 41% as the context window fills — a cliff, not a slope. Attention is U-shaped, so bigger windows just create a bigger ignored middle; some teams waste 60%+ of token budget re-injecting stale context. Aggressive compaction backfires: compressing an 18,282-token conversation to 122 tokens performed worse than no memory at all, since summaries keep the story but drop actionable facts. The author also cites a silent failure: 3,000+ docs in a memory layer, dashboard showed 91 memories, yet every search returned empty with no error. Their fix (Synap): retrieval-based memory across vector/graph/file stores under a token budget, with compaction checks reporting fact-survival rates, P75 latency under 15ms, and roughly halved token spend. Suggested self-check: measure accuracy at turn 5 vs turn 50 on the same task, then diagnose whether it's the attention cliff or lossy compaction. (Disclosure: author works on Maximem Synap; the 847-run data is from another r/LocalLLaMA developer.)
More from coding & agent
- Pigment Agents now reads and posts in Slack, inheriting user permissions — mattturck · 2026-11-12
- Map of the agent economy: 50+ startups across payments, spend control, identity, memory — venturetwins · 2026-11-12
- OpenCode launches Browse, an on-device browser agent with local logins — Saul_Loveman · 2026-11-12
- VentureTwins maps the B2A economy: dozens of agent infra and app companies in first series installment — venturetwins · 2026-11-12
- Google releases 76-page 'Agents Companion' whitepaper on agent engineering — mdancho84 · 2026-11-12
- Spinning up containers for 20-line agent tools is backwards: Safescript compiles provable, 50ms-deploy apps — uriwa · 2026-11-12