Sleep-time Compute paper cuts test-time compute ~5x by pre-thinking offline
RLanceMartin · x · 2026-11-07
The author cites Letta's work on memory, linking to the arXiv paper "Sleep-time Compute: Beyond Inference Scaling at Test-time" (Kevin Lin, Charles Packer, Sarah Wooders, Ion Stoica, Joseph E. Gonzalez, et al.).
- Lets models "think" offline about contexts before queries arrive: anticipate likely questions and pre-compute useful quantities, reducing test-time compute.
- On Stateful GSM-Symbolic and Stateful AIME, sleep-time compute cuts the test-time compute needed for the same accuracy by 5x; scaling it further boosts accuracy up to 13% and 18%.
- Multi-Query GSM-Symbolic amortizes sleep-time compute across related queries on the same context, cutting average cost per query 2.5x.
- Efficacy correlates with how predictable user queries are; includes a real-world case study.
Takeaway: offline pre-computation ("sleep-time") is a viable axis beyond test-time scaling for the compute-accuracy tradeoff.
More from Research
- RFD4-Proteina: 158 of 188 designed TrkA binders bound below 1 µM, far ahead of rivals — AllThingsApx · 2026-11-07
- Default to human-robot teams: why research should be judged on quality, not AI use — _akpiper · 2026-11-07
- Dozens of AI-discovered drugs in the clinic: rebutting Vertex chairman's claim that AI doesn't work — chaitjo · 2026-11-07
- Dev plans to SFT a 4B model into a specialized coding agent, starting with 100-200 mini SaaS apps — ConfidentDinner6648 · 2026-11-07
- Berkeley's ADA aligns simulation-trained generative models to real experiments at NeurIPS — berkeley_ai · 2026-11-07
- Diffusion language models get an AACL-IJCNLP 2026 tutorial, slides coming soon — Bollegala · 2026-11-07