Google's sleep framework lets LLMs keep learning, retaining up to 10M tokens
ZeroStateReflex · x · 2026-10-30
Google Research (Ali Behrouz and Vahab Mirrokni) introduced the "sleep framework," a continual learning paradigm inspired by how the brain consolidates memory during rest.
- Problem addressed: deployed LLMs are static with fixed knowledge cutoffs; continual parameter updates cause catastrophic forgetting, while in-context learning is lost when the session ends.
- Approach: alternate between active interaction phases and offline sleep phases that consolidate fragile short-term context into stable long-term parameters.
- Results: models can continuously acquire new facts, retain long-context knowledge up to 10M tokens, and improve complex reasoning.
- Boosters argue that once scaled, this could open the door to AGI and conscious machines.
More from Models
- ChatGPT's cyber classifier rejection UX confuses users into applying for Daybreak — jd_pressman · 2026-10-30
- Shift-1 coding router cuts agent costs by up to 55% with no quality drop — ypatil125 · 2026-10-30
- Artificial Analysis launches Search Index benchmarking 26 Search APIs for AI agents — ArtificialAnlys · 2026-10-30
- Mistral Large 4 architecture teardown: sparse attention plus quirky KV/indexer sharing — eliebakouch · 2026-10-30
- Particle-1 matches DeepSeek V4.1 Flash with 43% fewer output tokens on Real-SWE — ycombinator · 2026-10-30
- Rumor: Alongside Google's Carbon, OpenAI's Internal Model "Bel" Is Coming — bindureddy · 2026-10-30