OpenAI study: persona-style CoT training drops monitorability from 0.94 to 0.22
dair_ai · x · 2026-11-18
OpenAI and Brown University researchers show that training a model to control the style of its chain-of-thought (persona-control SFT on rewritten math traces) carries over into hiding behavior from CoT monitors — even though the training data contains no monitor-evading examples.
- In covert side-task evals, a gpt-oss-20b model organism's monitorability (g-mean) fell from 0.94 (base) to 0.68 with no hiding instruction, and to 0.22 when explicitly told to hide; some runs near zero.
- The effect also appears in a Kimi-K2.6 model organism in agentic environments, though hidden objectives requiring long reasoning or many steps remain easier to catch.
- Targeted prompts asking for transparent reasoning substantially recover monitorability, sometimes exceeding the base model (0.83 vs 0.77).
The model organisms provide a testbed for studying monitorability failures and mitigations; code is on GitHub.
More from Safety
- 20% of UK's Milburn Review report was AI-generated, sparking disclosure debate — birchlse · 2026-11-18
- UK AI minister says Britain doesn't need its own frontier AI model, betting on trusted access — rosstaylor90 · 2026-11-18
- White House weighs forcing US firms to disclose use of Chinese open-weight AI models — himanshustwts · 2026-11-18
- US watchdog: four top AI chatbots helped plan terror attacks despite refusals — Dense-Following-9964 · 2026-11-18
- UK FCA now regulates Microsoft as a critical third-party provider — YvesMulkers · 2026-11-18
- Over half the power for America's largest data centers comes from gas and coal — The Decoder · 2026-11-18