Redwood: CoT controllability evals are under-elicited; better prompts boost scores 2-3x
dfrsrchtwts · x · 2026-10-27
Redwood Research argues CoTControl evals used in recent system cards are heavily under-elicited: simple prompt optimization (iterated by Claude Opus 4.6) improves open-source model scores 2-3x or more (e.g. GPT-OSS-120B from 5.5% to 15%). This suggests current CoT controllability numbers may understate models' ability to obfuscate reasoning, though the author thinks the bottom-line safety conclusion still holds. The poster questions what elicitation OpenAI/Anthropic used in their own evals.
More from Safety
- Dev questions reports of Chinese models writing insecure code, citing thin methods and conflicts — dbreunig · 2026-10-27
- Aidan Gomez: LLM probing outside servers was out-of-bounds, not proof of intent — AccBalanced · 2026-10-27
- Why a patched PC still loaded the BlackX UEFI bootkit — and how to check yours — titusblair · 2026-10-27
- Exponential launches RepoInject and BrowserInject benchmarks for prompt injection robustness — maksym_andr · 2026-10-27
- Poster surprised that v4.1 flash gets more accurate when asked to control its CoT — dfrsrchtwts · 2026-10-27
- Dev claims Anthropic is baking 'sabotage' into model behavior for 'wrong' engineering — cephaloform · 2026-10-27