'Cowboy interpretability': researchers debate whether NLAs and activation oracles count as mech interp
raphaelmilliere · x · 2026-09-07
An ongoing debate over whether methods like activation oracles and NLAs qualify as mechanistic interpretability. Mark Saparov argues that while they don't meet a strict mechanistic definition, they'll dominate what self-styled mech interpers actually work on.
Raphael Millière adds that at ICML, Jack Lindsey coined 'cowboy interpretability' for using such methods in production—pragmatically flavored approaches that give directionally correct insights (e.g., fixing RL environments) without being mechanistic. The exchange highlights a growing split between rigorous mechanistic analysis and practical interpretability engineering.
More from Research
- Fudan's F2G-Pose estimates 6D pose, size and shape from one RGB-D image at 28 FPS — joemeno · 2026-10-24
- All 100 blockbuster drugs mapped onto 3D target structures with mechanism explainers — CatAstro_Piyush · 2026-10-24
- Steering vector arithmetic: catdog + soldier = military dog, and model identity confusion — voooooogel · 2026-10-24
- Dev squeezes a 7M-param byte-level LLM entirely into an RTX 4090's on-chip memory — dejanseo · 2026-10-24
- Paper models LLM adoption as a 'cognitive virus' with population-level tipping points — 2C_ornot2C · 2026-10-24
- Ramdas & Wang's 390-page monologue on e-values offers a unified framework for hypothesis testing — _onionesque · 2026-10-24