Can We Bake New Facts Directly Into Weights?
Moh1tAgarwal · x · 2026-07-17
This discussion focuses on a key question: Can new facts be written directly into an LLM's weights without breaking the model?
The author's team found that things go wrong in unexpectedly weird places. Consequently, instead of directly updating weights, they favor continuous learning approaches that keep memory in context, such as compressing the KV cache and ICL / retrieval.
Related event: Technical Debate on Writing New Facts into LLM Weights(3 posts)→
More from Research
- PoLar: Dynamically Skipping or Looping LLM Layers for Efficient Inference — ttkciar · 2026-07-22
- Stanford Team Introduces Gigatoken, the World's Fastest Tokenizer — StanfordAILab · 2026-07-22
- Tabul AI launches Metal TreeSHAP to speed up Shapley values on Apple silicon — Scobleizer · 2026-07-22
- ICML Tutorial: Is Optimization Theory Relevant in 2026? — srush_nlp · 2026-07-22
- Reddit points to OpenAI’s ChatGPT Ads page — EcstaticAsparagus509 · 2026-07-22
- Open-source runtime lets each repo define its own AI code reviewer — ibabufrik · 2026-07-22