Baseten study finds new facts in LLM weights are fragile unless trained from many restatements

alex_verem · x · 2026-07-21

A Baseten paper studies whether new facts can be stored continually in an LLM’s weights, using 247 invented facts written into Qwen3 one by one.

• How the fact is written matters: training on a bare statement gives high recitation accuracy, but the model often cannot actually use the fact. Using 24 varied restatements sharply reduces the gap between reciting and applying the fact, without ever stating the conclusion directly.
• Forgetting happens fast: after 20 later writes, bare-statement facts drop to about 1% retention, while broadly trained facts still retain 46%.
• Forgotten does not mean erased: the facts still keep 57–67% of the probability mass added during the write, but later writes interfere with earlier ones.
• Main takeaway: reliable knowledge accumulation seems to depend more on context and recall cues than on cramming facts into weights alone.

Original post →

More from Research

• Cognition's SWE-2 uses a KKT duality argument in RL to shift the effort Pareto curve — YouJiacheng · 2026-09-11
• VidMap uses RoMa coarse matching on all frames, fine-scale only for keyframes — ducha_aiki · 2026-09-11
• Bug Hunt Bench author: leaderboard noise is about 2-3 points — PawelHuryn · 2026-09-11
• PNAS paper shows a tiny billiard-ball system is a universal computer — undecidability lives in two dimensions — eigensteve · 2026-09-11
• New paper: Absolute pose estimation from affine cues and gravity direction — ducha_aiki · 2026-09-11
• LoMa Paper Ships REALLY HardPairs Dataset, Accepted at ECCV 2026 — ducha_aiki · 2026-09-11

Research channel →