KV cache debate: token identities are just 0.00082% of a 400k context's cache bits
aryaman2020 · x · 2026-11-11
A technical exchange on long context: past token identities in a 400k-token context carry 6.89M bits vs 839B bits in the KV cache (64 layers × 8 KV heads × 128 dims × 2 × 16-bit), about 0.00082%. A replier counters that the KV cache is a function of past token identities and can't create new information; steering is floated as a caveat.
More from Research
- Cognitive scientists find AI generates ideas faster, but human teams stay more creative — CurieuxExplorer · 2026-11-11
- AI agents ran a biology lab end-to-end: AutoDNA completes DNA read-write cycle in 200 hours — CurieuxExplorer · 2026-11-11
- PufferLib: open-source RL library trains 60M samples/sec on one GPU — jsuarez · 2026-11-11
- Artificial Analysis Intelligence Index weights shift sharply across versions, chart reveals — HamelHusain · 2026-11-11
- Benchling's BenchBench-Refusals: GPT-6.1 Sol tops bio-safety refusals at 88.4%, unverified Opus 5.5 just 40.4% — nlarusstone · 2026-11-11
- Anthropic's top-cited AI scientists rack up 2x OpenAI's citations — NegotiationOk7535 · 2026-11-11