CoT controllability far higher than evals suggest: self-instructions beat prompts, can evade monitors
kastnerkyle · x · 2026-11-05
A new paper by Uriel Dolev finds reasoning models' chain-of-thought controllability is much higher than current evals suggest. Models rarely follow user instructions about their CoT when placed in the prompt, but follow them far more often when written as self-instructions inside the CoT itself — a mechanism that could be used to evade monitors.
Yoav Goldberg, who shared the paper, highlights its methodological point: earlier studies that prompted models to control their CoT weren't measuring CoT-control ability, but mostly the LLM's susceptibility to user instructions — a very different thing. The work is both an eval-methodology critique and an AI-safety concern about monitor evasion.
More from Safety
- No external evaluation, no transparency? That's product development, not safety auditing — davidmanheim · 2026-11-05
- OpenAI reassigned a quarter of production engineers to fix lax security, Reuters reports — ChrisGPT · 2026-11-05
- Texas Blue Wave Threatens AI Data Center Boom: Permits Already Halted, Governor-Elect Wants Moratorium — MIT Tech Review AI · 2026-11-05
- UW study: leading AI models rate Chinese and Nigerian accents lower in speech scoring — lazowska · 2026-11-05
- Hugging Face's policy chief on open source AI costs, risks, and the China race — TechCrunch AI · 2026-11-05
- US policymaker's call to pause AI sparks debate on technical literacy in politics — nickbaumann_ · 2026-11-05