Paper Proposes 'Sleep Phase' for LLMs to Consolidate Memory Offline

rohanpaul_ai · x · 2026-08-13

To address the slowdown and rising costs of Transformer agents caused by attention mechanisms over long contexts, a new paper proposes introducing a sleep phase for language models.

Core Mechanism

Offline Consolidation: The model pauses to reread recent context multiple times, extracting useful information into fixed-size memory layers (fast weights within state-space blocks), and then clears the short-term attention cache.
Inference Efficiency: This design shifts the extra compute cost to the 'sleep' phase. During actual prediction and answering, normal inference still occurs with just 1 forward pass.

Results

The authors evaluated the approach on cellular automata, graph lookup, and GSM-Infinite math problems requiring the use of older information no longer in the attention cache. Longer sleep periods significantly improved performance, especially on harder cases requiring deeper reasoning rather than simple fact retrieval.

This suggests long-horizon agents may not need endlessly growing raw context windows; instead, they can operate efficiently by consolidating important memories and safely forgetting raw tokens.

Original post →

Paper Proposes 'Sleep Phase' for LLMs to Consolidate Memory Offline

More from coding & agent