Baseten study finds new facts in LLM weights are fragile unless trained from many restatements

alex_verem · x · 2026-07-21

A Baseten paper studies whether new facts can be stored continually in an LLM’s weights, using 247 invented facts written into Qwen3 one by one.

• How the fact is written matters: training on a bare statement gives high recitation accuracy, but the model often cannot actually use the fact. Using 24 varied restatements sharply reduces the gap between reciting and applying the fact, without ever stating the conclusion directly.
• Forgetting happens fast: after 20 later writes, bare-statement facts drop to about 1% retention, while broadly trained facts still retain 46%.
• Forgotten does not mean erased: the facts still keep 57–67% of the probability mass added during the write, but later writes interfere with earlier ones.
• Main takeaway: reliable knowledge accumulation seems to depend more on context and recall cues than on cramming facts into weights alone.

Original post →

More from Research

• HarnessMM cuts agent tokens per task by up to 88% while raising success rates — omarsar0 · 2026-10-26
• Ben Recht: The world needs a truly open language model, and it's easier to build than you think — beenwrekt · 2026-10-26
• New Blog Breaks Black-Box Distillation Into Five Practical Steps — chrisrohlf · 2026-10-26
• DetectorScope turns black-box AI text detectors into auditable ones with sparse autoencoders — _akhaliq · 2026-10-26
• OPST paper shows LLMs self-improve with zero human data, guided only by topic descriptions — _akhaliq · 2026-10-26
• Tencent's HiSTEVE 1.0: hierarchical VLA agent plays Minecraft with native keyboard and mouse, hits 85% SOTA — _akhaliq · 2026-10-26

Research channel →