HazyResearch shows how to write facts into Transformers without training
CSProfKGD · x · 2026-07-23
A HazyResearch team reports a new way to inject knowledge into Transformer blocks without gradient descent.
- The paper frames the problem as how facts are stored in language models’ MLPs, then asks whether they can be written into Transformers directly.
- The authors claim a closed-form recipe for fact-storing, Transformer-ready MLPs.
- The work has been accepted at COLM 2026 and is presented as a research/blogpost package for further reading.
Related event: HazyResearch Proposes Training-Free Method to Write Facts into LLMs(5 posts)→
More from Research
- Hierarchical continuous flow language models make pathwise RL more robust to critic exploitation — vxnuaj · 2026-10-27
- rauchg reproduces OpenAI's covert CPU-channel finding: chat across Firecracker microVMs with no network — ccerrato147 · 2026-10-27
- Microsoft Research: ForkPilot cuts long-horizon agent compute by up to 59% — EliasEskin · 2026-10-27
- Microsoft's TeleTune evolves agent skills from offline user telemetry — EliasEskin · 2026-10-27
- FlexGlue: sparse-to-dense matching that beats dense accuracy at a fraction of the cost — zhenjun_zhao · 2026-10-27
- PixGR: pixel-space generative decoding for fast feed-forward 3D Gaussian reconstruction — zhenjun_zhao · 2026-10-27