Goodfire proposes VPD: decomposing LM parameters into faithful subcomponents
Sauers_ · x · 2026-10-02
Goodfire, with MATS and independent researchers, published "Interpreting Language Model Parameters", introducing adVersarial Parameter Decomposition (VPD).
- Motivation: prior interpretability work uncovered structure in intermediate representations, but little about how parameters and nonlinearities actually compute.
- Method: VPD decomposes a language model's parameters into simple subcomponents, each implementing a small part of the learned algorithm, optimized so input-output behavior is preserved even under heavy ablations — including adversarially chosen ones.
- Result: this yields short, mechanistically faithful descriptions of the network's behavior that aggregate into more global accounts of the learned algorithm.
Authors include Lucius Bushnaq, Dan Braun, Lee Sharkey and others; published May 5, 2026, with a blog version on Goodfire's site.
More from Research
- Boltz Lab reveals 50x molecular token growth in 5 months as Series A closes — GabriCorso · 2026-11-19
- Researcher calls METR/Anthropic '2x speedup' RSI analysis statistically dubious — jessi_cata · 2026-11-19
- Linear Probes Could Reveal Which Persona a Base Model Is Playing, Researchers Suggest — cephaloform · 2026-11-19
- ETH's Anymal robot completes 10km autonomous alpine hike with VLM-guided ROAM system — ChongZzZhang · 2026-11-19
- OpenVax codebase grows from 78K to 565K lines in 15 months — iskander · 2026-11-19
- François Fleuret: decoder transformer's elegance — no trade-off hyperparameters — francoisfleuret · 2026-11-19