Writing Facts into Transformers Without Training: New COLM Paper
HazyResearch · x · 2026-07-23
Traditionally, factual knowledge in language models is stored within MLP layers through training. A new paper accepted at COLM 2026 proposes a novel approach: writing facts directly into Transformers without any training.
The researchers provide a closed-form recipe to construct Transformer-ready MLPs capable of storing facts. The work is a collaboration between Hazy Research and other researchers.
More from Research
- Human Technopole Recruiting PhD Students for AI and Healthcare — yacineMTB · 2026-07-23
- SMDD-Bench tests whether LLM agents can budget through real small-molecule design — niloofar_mire · 2026-07-23
- LLM text analysis needs human checks, says literary studies discussion — soumitrashukla9 · 2026-07-23
- Paper maps equivalent ways to visualize tree data structures — round · 2026-07-23
- Blog argues agents still can’t design systems well, and says the stack must change — tianyin_xu · 2026-07-23
- A deep dive on building agents that can work for days, not minutes — baseten · 2026-07-23