Goodfire proposes VPD: decomposing LLM parameters into interpretable subcomponents
CatAstro_Piyush · x · 2026-10-27
Goodfire's new essay introduces adVersarial Parameter Decomposition (VPD), a mechanistic interpretability method that splits a language model's parameters into subcomponents, each implementing a small part of the learned algorithm. Decompositions are optimized so input-output behavior survives mass ablations—including adversarially chosen ones—yielding short, mechanistically faithful descriptions. Commenters note SAE weaknesses like feature splitting that VPD addresses, while debating whether superposition still needs accounting for.
More from Research
- figwig format ships with CLI tools and an agentic skill for bioinformatics sparse data — jmschreiber91 · 2026-10-28
- A bigwig-style file format built for genomic ML with direct GPU loading — jmschreiber91 · 2026-10-28
- More details on the genomic ML file format: sparsity and pseudobulk support — jmschreiber91 · 2026-10-28
- Cortex benchmark shows financial AI finds the right document 92-99% of time but ranks it first only 42-69% — Exp_Mark · 2026-10-28
- Benchmark: figwig genomic data format loads GPU batches ~10x faster than bigWigs — jmschreiber91 · 2026-10-28
- Prompt Wording Flips GPT-5.4 Novelty Verdicts From 93.5% to 40.9% in Ai2's Eval — allen_ai · 2026-10-28