DoM steering success isn't strong evidence for interpretability localization
johnhewtt · x · 2026-10-27
johnhewtt pushes back on arunasank's claim that behavior is evidence mediators are effective: DoM performing better doesn't strongly support the localization strategy, since neither the localization method nor DoM is necessarily the correct way to modify the network. arunasank cites their ReFT work showing gradient-based steering works without localization in limited settings. The thread stems from johnhewtt's blog on separating forms from estimators in interpretability.
Related event: Interpretability debate: steer by difference of means or by gradients?(6 posts)→
More from Research
- Kevin Bryan: maximizing raw accuracy is rarely the right way to train AI assistants — Afinetheorem · 2026-10-27
- Szegedy: a flaw in common training losses went unnoticed for over a decade — ChrSzegedy · 2026-10-27
- Claude research model claims to refute Exponential Time Hypothesis with 14M-token autonomous run — scaling01 · 2026-10-27
- "ETH is false": CS community buzzing over claim that upends SAT exponential-time assumption — akbirthko · 2026-10-27
- RL environments essay: knowing how to operate software isn't knowing how to achieve goals — ShenRaphael · 2026-10-27
- Δτ-Bench: Agents Face a Moving Tool Frontier, Best Model Scores Just 42.3% — UniversityCollegeLondon · 2026-10-27