HazyResearch claims a closed-form recipe for storing facts in Transformers
HazyResearch · x · 2026-07-23
HazyResearch and collaborators report a closed-form way to store facts in Transformer-ready MLPs without training.
- The work starts from the observation that MLPs store facts in language models.
- It asks whether those facts can be written into Transformers directly, without gradient descent.
- The team says it has a closed-form recipe and notes the paper has been accepted at COLM 2026.
- A blog post and paper are linked for readers who want the details.
Related event: HazyResearch Proposes Training-Free Method to Write Facts into LLMs(5 posts)→
More from Research
- TRL adds Score Centering, a simple fix for RLHF training instability — lvwerra · 2026-10-27
- Latent-Foresight: world models that learn to predict their own future — abursuc · 2026-10-27
- UK AISI papers: no good safeguards yet against misuse of biological AI tools — davidmanheim · 2026-10-27
- CMU's XYBench: LLMs answer queries literally and reinforce user misconceptions — valentina__py · 2026-10-27
- arXiv advice from T. Dietterich: wrong primary category will almost surely get your paper on hold — tdietterich · 2026-10-27
- arXiv has 4,000+ papers on hold amid submission surge; moderator shares how to avoid it — tdietterich · 2026-10-27