Researchers Clash Over Interpretability's Value: Do Probes Count, and Why Isn't RL Held to the Same Standard

A debate over the value of interpretability research unfolded among researchers: Arthur Conmy argued that RL's progress is clearly visible from o1 to o3 to astra, while interpretability's value-add remains unclear, noting other safety directions are also progressing. aryaman, saprmarks and others joined the exchange, sparring over whether probes count as interp achievements and whether a "messy" methodology implies a failed field.

Confirmed

• aryaman rebutted the claim that probes are simple and unimproved by mechinterp research: those deploying and iteratively improving probes in production at frontier labs are mostly interpretability researchers, so probes should count as a real interp win. Arthur Conmy agreed with this rebuttal, while still conceding that interp overall remains "a mess" but that RL's trajectory of improvement is clearer
• aryaman pressed further: the methodology behind today's successful RL is entirely different from pre-o1 RL theory, making RL equally "a messy technique"; its incremental algoslop draws no criticism while interp does—a double standard. His core argument: in an empirically-driven, active science, methodological messiness has no necessary bearing on whether the field succeeds
• saprmarks (a co-author of the activation oracles paper) clarified that he does not consider activation oracles or NLA to be "mechanistic interpretability," pointing out the paper itself contains a section stating activation oracles are not of this type; he added that, for similar reasons, supervised activation probes also do not count as mech interp
• Per raphaelmilliere's account of the discussion, Mark Saparov believes such methods, while not meeting the strict definition of mechanistic interpretability, will become the path most self-described mech interp researchers actually take (the "cowboy interpretability" debate)
• Arthur Conmy laid out his position on mech interp's prospects: he hopes mech interp wins, sees improvements in no-CoT time horizons as an opportunity, but mech interp must get dramatically faster and cannot wait 8.5 years; he does not think interp's success is guaranteed, nor is it necessary

Why it matters

• The debate touches a persistent question in the AI safety community: how to assess whether interpretability research has produced real value, with probes' production deployment at frontier labs offered as key evidence
• The RL-versus-interp comparison raises questions of inconsistent standards, relevant to resource allocation and direction-setting in safety research
• The activation oracles author personally clarifying the method's classification—and excluding supervised activation probes from mech interp as well—shows the boundary of "what counts as mech interp" is itself being redrawn, directly affecting how the field's achievements are evaluated

2026-09-07 ~ 2026-09-07 · 10 related posts

• [source] Interp researcher pushes back: probes have advanced well beyond pre-LLM-era techniques — aryaman2020 · 2026-09-07
• Interp researcher defends probes as a real win while downgrading the field's overall progress — ArthurConmy · 2026-09-07
• Why do RL researchers get away with incremental algoslop while interp doesn't? — aryaman2020 · 2026-09-07
• [source] Interp researcher: RL clearly progressed from o1 to astra, but interp's value add is unclear — ArthurConmy · 2026-09-07
• RL methods are a mess too — and that didn't stop the field from succeeding, argues aryaman — aryaman2020 · 2026-09-07
• [source] Activation Oracles Author Weighs In: 'A Fundamentally Non-Mechanistic' Interp Technique — saprmarks · 2026-09-07
• saprmarks: Supervised Activation Probes Aren't Mechanistic Interpretability Either — saprmarks · 2026-09-07
• Interp researcher: mech interp must work far faster than 8.5 years, or CoT monitoring wins — ArthurConmy · 2026-09-07
• Researchers debate whether mech interp is essential as CoT monitorability scales — ChrisGPotts · 2026-09-07
• 'Cowboy interpretability': researchers debate whether NLAs and activation oracles count as mech interp — raphaelmilliere · 2026-09-07