Do models really 'pass messages through KV cache'? A skeptic pushes back
scaling01 · x · 2026-09-04
- Challenging the claim that models "pass messages through the KV cache far more complex than what they pass through token space."
- The author argues the KV cache is just the internal state given the context: recomputing all states without a cache yields the same result.
- So the cache isn't enabling any extra information passing — it only saves compute — and treating chain of thought as a proxy for the model's internal process may be misleading.
Related event: Looped Transformer Rumors Spark Fierce Debate Over CoT Monitorability(7 posts)→
More from Models
- DeepSeek V4.1 Flash inference teardown: Engram cuts 30% params, WideEP spreads 552B over 64 GPUs — teortaxesTex · 2026-10-21
- MiniMax M3.1 Flash tops p0's search efficiency leaderboard at ~3x lower cost — VoidAsuka · 2026-10-21
- Voice models should run on-device: Phonon open-source release sparks debate — timshi_ai · 2026-10-21
- Cloudflare's 27B multimodal decision model Clef lands on Ollama for local use — ollama · 2026-10-21
- Terence Tao on OpenAI's math release: solving problems isn't the point, insight is — haider1 · 2026-10-21
- Terence Tao on OpenAI's math results: we want deeper insight, not solved problems — haider1 · 2026-10-21