OpenAI warns CoT monitorability will degrade as models scale, researchers push back
gleech · x · 2026-11-17
OpenAI's alignment team published a study on how pretraining, architecture, RL, and model-generated data affect chain-of-thought (CoT) monitorability, using no-CoT capability and CoT controllability as proxies. Their conclusion: models will become harder to monitor via CoT as systems scale.
Researcher RaunoArike flagged a key caveat: the claim that no-CoT capability is explained by base-model perplexity understates architecture effects—changes like looped transformers improve performance on evals measuring serial multihop computation and planning, precisely the capabilities most relevant to unverbalized scheming risk. Evals unaffected by architecture mostly test knowledge retrieval.
The report, authored by Jeffrey G. Wang, Micah Carroll, Jakub Pachocki and others, notes CoT monitoring is a core pillar of OpenAI's safety cases, with full-trajectory monitoring already used for cyber-critical deployments.
More from Safety
- US shuts Malaysia chip loophole: license now needed to reexport advanced chips to five countries — SumitGup · 2026-11-17
- Report: US Has 9x China's Compute, So It Can Afford to Pace the AI Frontier — trevposts · 2026-11-17
- LessWrong: Gemini 4 Safety Reports Fail to Demonstrate Low Loss-of-Control Risk — DKokotajlo · 2026-11-17
- Paul Christiano: A 1-in-3 Chance of Government-Overthrowing AI Within a Year — Dwarkesh Podcast · 2026-11-17
- After a compromise, this engineer moved all AI coding agents off his desktop — rseroter · 2026-11-17
- FBI: Open-Source Hacking Tool Artex AI Breached Networks in Under Three Hours — TechCrunch AI · 2026-11-17