DeepMind researcher: CoT interpretability is too fragile to anchor long-term AI safety
cephaloform · x · 2026-09-04
jachiam0 (DeepMind interpretability) argues chain-of-thought interpretability was always too fragile to serve as a long-term AI safety backstop. While efforts to protect CoT fidelity were worthwhile, elevating "CoT must stay human-legible" as a principle is a doomed strategy that will eventually fail. The field should not depend on it for enduring reassurance; making models legible must go far beyond CoT fidelity.
More from AGI Musings
- Kent Beck: Let the Genie Carry the Toil, But Keep the Struggle — josh_wills · 2026-10-21
- Nearly $3B mapped: the money behind the AI existential risk ecosystem — KeyNaive2982 · 2026-10-21
- Three reasons AI is not conscious and may never be, per Nature essay — anilkseth · 2026-10-21
- Everything in AI is downstream of a global status graph, says founder — gabriel1 · 2026-10-21
- Should Claims That AI Agents Aren't Humanlike Carry a Burden of Proof? — davidmanheim · 2026-10-21
- Mathematician Daniel Litt concedes his AI bet, but says AI won't solve math — ziv_ravid · 2026-10-21