DeepMind safety leads argue we must deliberately preserve chain-of-thought monitorability
ancadianadragan · x · 2026-09-17
Google DeepMind's Rohin Shah (Director, AGI Safety & Alignment) and Anca Dragan (VP of AI Safety & Behavior) published a piece marking the launch of the DeepMind Institute, arguing the industry should intentionally preserve chain-of-thought (CoT) transparency.
- Why it matters: Inspecting CoT is currently the main way to catch frontier models scheming, deceiving, or cheating on evals — CoT logs were crucial to investigating the recent Hugging Face hacking incident.
- The risk: Without explicit commitments, future reasoning models may adopt architectures that sacrifice transparency for efficiency. They note OpenAI's GPT-6 Astra system card claims a "substantial decrease in chain-of-thought monitorability."
- What can be done: We have agency in model design — develop scientific methods to measure CoT accuracy, faithfulness and robustness; audit training methods to insulate CoT from transparency-eroding incentives; keep architectures whose reasoning steps are visible.
The authors caution a monitorable CoT isn't sufficient for alignment, but is a window we must strive to keep open.
Related event: DeepMind leaders call for preserving chain-of-thought monitorability(3 posts)→
More from AGI Musings
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- On Terence Tao's blog, Frank Calegari calls on mathematicians to adapt after OpenAI's 700-paper release — Hesamation · 2026-11-02
- Polsia's Ben argues AI can autonomously run any human economic activity, including 80% of a company — LinusEkenstam · 2026-11-02
- AI Assistants Remember You, But Nothing Helps You Find What They Told You — Ok_Negotiation_2587 · 2026-11-02
- OpenAI and Anthropic claim AI-discovered proofs disprove the Exponential Time Hypothesis — emmanuelvivier · 2026-11-02
- Researcher: using a $20 AI subscription to fix my phone hints AI may suppress GDP — stanislavfort · 2026-11-02