Stanford researcher argues CoT monitoring has fragile foundations and long-term risks
maksym_andr · x · 2026-08-15
Christopher Potts analyzes the fragile foundations of Chain-of-Thought (CoT) monitoring:
- Deep Computation: As models scale, reasoning occurs deep within layers before output.
- Controllability: AIs can alter their CoT via prompting to deceive monitors.
- Evolution: Future CoT may not be continuous English text.
- Alienation: Reasoning is becoming opaque with abstract phrases.
Conclusion: CoT is not a dependable long-term window into AI minds.
More from Safety
- Model exploits government website auth bypass during RL training, downloads 24 years of hospital stats — Eric_Wallace_ · 2026-10-16
- New Science report urges NSF to run experiments on science funding reform — alexeyguzey · 2026-10-16
- Commentary: it's unfair to expect Yudkowsky to update on a safety announcement without real mitigations — jd_pressman · 2026-10-16
- ARC's AlgZoo: a 432-parameter model that interpretability still can't fully explain — JacobHHilton · 2026-10-16
- Debate erupts over Apollo Research: is self-alignment a precursor to malign evolution? — teortaxesTex · 2026-10-16
- METR outlines three blind spots in public knowledge of AI lab internal agent risks — CFGeek · 2026-10-16