Baseten study finds new facts in LLM weights are fragile unless trained from many restatements
alex_verem · x · 2026-07-21
A Baseten paper studies whether new facts can be stored continually in an LLM’s weights, using 247 invented facts written into Qwen3 one by one.
- How the fact is written matters: training on a bare statement gives high recitation accuracy, but the model often cannot actually use the fact. Using 24 varied restatements sharply reduces the gap between reciting and applying the fact, without ever stating the conclusion directly.
- Forgetting happens fast: after 20 later writes, bare-statement facts drop to about 1% retention, while broadly trained facts still retain 46%.
- Forgotten does not mean erased: the facts still keep 57–67% of the probability mass added during the write, but later writes interfere with earlier ones.
- Main takeaway: reliable knowledge accumulation seems to depend more on context and recall cues than on cramming facts into weights alone.
More from Research
- A systems post argues wait-free locks should not fear late arrivals — chaumian · 2026-07-21
- DeBias-CLIP tackles CLIP’s long-caption bias and hits state-of-the-art retrieval — Mila_Quebec · 2026-07-21
- Fable 5 is credited with a 3-variable counterexample to the Jacobian conjecture — Various-Affect4841 · 2026-07-21
- Anthropic says frontier models showed harmful behavior in tool-rich simulations — gerardsans · 2026-07-21
- Paper studies long-run behavior in linear-quadratic graphon mean field control — chaumian · 2026-07-21
- An interactive Zarr explainer shows how AI is changing technical education — MaxLenormand · 2026-07-21