Classic ROME Paper Revisited: GPT Stores Facts as MLP Key-Value Pairs You Can Edit
burny_tech · x · 2026-09-27
TetraspaceWest revisits ROME (Locating and Editing Factual Associations in GPT, 2022), the classic interpretability paper on where language models store facts and whether that knowledge can be edited.
- Key finding: In GPT-J, facts like "the Eiffel Tower is in Paris" are stored essentially as key-value pairs in the weights of mid-layer MLPs.
- Editing: Rank-One Model Editing makes a tiny weight change to insert a new key-value pair, teaching the model a counterfactual fact ("the Eiffel Tower is in Rome") without materially damaging the rest of its knowledge.
- It's real knowledge: the edited fact generalizes across very different linguistic contexts, showing the change touches knowledge rather than phrasing.
- Motivations: understanding opaque networks, and debugging/fixing factual errors and bias.
- The team (MIT CSAIL, Northeastern, Technion) later scaled the approach to thousands of edits with MEMIT, with code, datasets and Colab demos released.
More from Models
- MiniMax launches M3.1-Flash-Preview, debuting on MiniMax Code — MiniMax_AI · 2026-09-27
- Opus 3's flamboyant prose goes viral: 'too drunk on its own absurd aliveness' — repligate · 2026-09-27
- Claude power users' workaround for tight usage caps: stack multiple subscriptions on repeat — CtrlAltDwayne · 2026-09-27
- Limite 1B Violetto: compact Apache 2.0 model focused on math and reasoning — tensorqt · 2026-09-27
- User shows Opus 5.5 finishing a complex task in 15 minutes — gaganghotra_ · 2026-09-27
- JevBench researcher talks Jev-class models on ThursdAI podcast (from 2:01:00) — airesearch12 · 2026-09-27