Paper: Long Context Weakens Parametric Learning in LLMs
SonglinYang4 · x · 2026-08-16
A new preprint proposes the "Information Abundance Paradox": when relevant information is abundant in the training context, the model has less incentive to internalize it into parameters. This weakens parametric memory and holds across model sizes trained on 10B tokens.
More from Research
- HWREBench: AI researcher hacks Amazon smart devices daily to benchmark hardware reverse engineering — johnowhitaker · 2026-09-22
- TMLR submissions quadruple on AI-generated influx; ICLR 2027 caps single authors — petitegeek · 2026-09-22
- Stanford's VirtualBiotech puts tens of thousands of AI scientist agents in Science, NYT reports — StanfordAILab · 2026-09-22
- Irit Dinur wins Gödel Prize for her landmark 2005 proof of the PCP theorem — willcb · 2026-09-22
- Researcher proposes using RL to teach AI when to give up — sqcai · 2026-09-22
- Nature publishes Delphy: scalable near-real-time Bayesian phylogenetics for outbreak tracking — burny_tech · 2026-09-22