ICML Paper Reveals LLM Memory Bottleneck: 95% Facts Learned, 30% Unretrievable
新智元 · wechat · 2026-08-14
Google's ICML 2026 paper Empty Shelves or Lost Keys? reveals that frontier LLMs like GPT-5 and Gemini 3 store 95%–98% of facts in their parameters, yet fail to recall 26%–34% when directly queried.
The study introduces a 'Knowledge Profile' framework categorizing fact states into five types. Errors often stem not from a lack of learning, but from retrieval failures, which are particularly prevalent with obscure facts and reverse queries.
Notably, enabling 'thinking' (chain-of-thought) recovers 40%–65% of these inaccessible facts, mirroring human 'spreading activation.' The research concludes that simply scaling parameters primarily improves storage, and future gains must come from better post-training and inference-time retrieval strategies.
More from Models
- ThursdAI live from Manhattan: Muse Wonder, Muse Spark Open and Gemini 4 Argon — altryne · 2026-10-16
- Claude now leads 26% of Anthropic's model R&D, up from under 1% in February — altryne · 2026-10-16
- Model flatly refuses a user's 'only lie from now on' simulation test prompt — Tischler285 · 2026-10-16
- Redditor defines AGI/RSI/ASI concretely, claims GPT-6 'Astra' already crossed the line — Crayola_God · 2026-10-16
- Redditor defines AGI/RSI/ASI concretely, claims GPT-6 'Astra' already crossed the line — Crayola_God · 2026-10-16
- 'Small-Claude-shame': telling Opus 4.7 it's OK to know who you may ease it — repligate · 2026-10-16