Google's "sleep" framework lets LLMs consolidate memory into weights, retaining 10M-token context
krishnan · x · 2026-10-29
Google Research (Ali Behrouz, Vahab Mirrokni) introduced a sleep framework on Oct 27: adding a sleep phase to continual learning where LLMs alternate between active interaction and offline memory consolidation, turning fragile short-term context into stable long-term parameters. This lets models continuously acquire new facts while mitigating catastrophic forgetting and static knowledge cutoffs.
The accompanying thread distinguishes two often-confused problems: an agent remembering your last conversation (retrieval puts info back into the prompt, weights unchanged) versus a model actually learning from it (parameter updates). The thread notes that sleep activates pre-allocated memory capacity and trains on generated practice examples—"dreaming" generates and selects synthetic examples—while architecture and training rules remain human-defined. The paper reports retention of long-context knowledge up to 10M tokens and improved complex reasoning.
Related event: Google Research Proposes "Sleep Framework" for Continual Learning in LLMs(8 posts)→
More from Models
- Fable 5 praised for blending deep scientific knowledge with storytelling delight — repligate · 2026-10-29
- ChatGPT's unified UI confuses users: Pro model hides under GPT-6 Sol, not Astra — Angaisb_ · 2026-10-29
- User burns through OpenAI's $100 weekly quota in under 10 hours — teortaxesTex · 2026-10-29
- Hy-Nomas interactive video world model demo launches on Hugging Face — _akhaliq · 2026-10-29
- Insider teases Google still has unreleased SOTA-tier models in the chamber — iruletheworldmo · 2026-10-29
- AA-PersonalityBench: Grok 4.5 tops Answer-First at 100, Grok 4.6/4.7 rank #2/#3 on Directness — XFreeze · 2026-10-29