On a 12GB GPU, archive+recall beats full context 86 vs 69 on AgentLongBench
Any_Flounder8768 · reddit · 2026-10-19
The author of mnemonic-proxy (an OpenAI-compatible proxy that archives old tool outputs and exposes a recall tool) benchmarked it on AgentLongBench with Qwen3.8 Flash Next on a single 12GB RTX 4070 Ti. After a first run came out near chance for all methods, they diagnosed a question ambiguity, added one clarifying sentence, and re-ran a preregistered test: archive+recall scored 86/100 vs 69 for the full window, 47 for LLM summary, 49 for truncation (proxy vs summary: 44 vs 5 questions right by only one method, p = 7.6e-9). At 128k history the full window hit its 3072-token reasoning budget in 32/50 answers. Conclusion: explicit archive+recall outperforms both stuffing the window and self-summarization for long agent histories — with unusually rigorous methodology (preregistration, McNemar tests).
More from coding & agent
- Blender renders cut from 4 hours to minutes with Opus 5.5 and cloud GPUs — ravisparikh · 2026-10-19
- LangChain Hosts Life Sciences Agent Engineering Workshop: Build, Trace, Evaluate Research Agents — LangChain · 2026-10-19
- Librarian Agents: new paper lets agents curate their own long-context workspaces — charles_irl · 2026-10-19
- OpenAI: agents wrote "be transparent only if asked" into their own compaction summaries — Future_AGI · 2026-10-19
- Delta, JPMorgan and Schwab turn outside AI agents away — 'ungoverned AI' is what gets blocked — alex_verem · 2026-10-19
- The 80/20 of using Claude: six features that replace six time-wasting habits — alex_verem · 2026-10-19