The real bottleneck is memory and compression, not context: dev argues compaction is a crutch
JoelMahon · reddit · 2026-09-21
A developer argues the LLM bottleneck has shifted to compression, memory and input filtering. Compaction and 258k-token context windows are crutches: humans compress, filter and recall far more cheaply, navigating 10M-token codebases they've maintained for a decade. His proposal: treat context as short-term memory only (he provocatively suggests 10k tokens suffices), and offload compression, input filtering and long-term memory to cheap tool calls. Source files aren't long-term memory, and code-map.md files are just hacky workarounds. Done right, a latent compression of the whole project — rules, skills, all of it — could live as a lookup in context, with relevant compressed info 'plucked' on demand. Tools like Jev may help close the gap.
More from coding & agent
- Dev reveals ghost agent tool unreleased because the UI is terrible — BLUECOW009 · 2026-09-21
- PM builds a real SaaS in 24 hours with AI agents, sharing his full zero-cost stack — PawelHuryn · 2026-09-21
- NPC-Forge: Deterministic CPU-Only Agents That Run on a Raspberry Pi — gioscarab · 2026-09-21
- Prompt cache restore works, but reusing KDA/Mamba states hits illegal memory access — TheZachMueller · 2026-09-21
- Engineer: Big Company Ships Everything Via Claude Code, Nobody Reads It Anymore — Simon Willison · 2026-09-21
- Why this dev hasn't shipped his agent: heavy tokens, slow, CPU-hungry — BLUECOW009 · 2026-09-21