BeNice! method shares KV-cache across loops, scaling looped LLM depth without extra memory
stochasticchasm · x · 2026-10-24
Rumor has it frontier labs are scaling quality with looped language models instead of more parameters — but KV-cache memory still grows with every loop. The proposed method BeNice! enables near-lossless sharing of the final KV-cache across loops, buying more effective depth at the same KV memory. The author notes this lossless sharing isn't guaranteed out of the box and requires training the model to share.
A commenter suggests YOCO-style decoder-only looping may be the simplest way to get looped LLMs without any KV-cache issues at all.
More from Research
- Google's PLE benchmark: best AI agent scores only 50.6% on professional engineering deliverables — dair_ai · 2026-10-24
- Quant hallucination forensics: fabricated prompt injections, pirate puzzles and a phantom Windows attack — Consumerbot37427 · 2026-10-24
- AgentVQA: 1,000 human-verified examples to evaluate visual agents offline across 5 domains — liuzhuang1234 · 2026-10-24
- Science study: STAT2 intron retention predicts lung cancer immunotherapy response (HR 0.49) — EricTopol · 2026-10-24
- Berkeley's Shreya Shankar: best data agent passes only 61% of queries, semantic layers help a lot — sh_reya · 2026-10-24
- New paper introduces "bottleneck pressure" to measure where work AI can't do gains value — soumitrashukla9 · 2026-10-24