Harvard/MIT paper: banning models' "Okay" opener boosts chain-of-thought controllability by 34-41 points
rohanpaul_ai · x · 2026-11-04
A Harvard/MIT paper traces reasoning models' poor chain-of-thought controllability partly to a first-token entropy collapse: Qwen3-32B and OLMo-3.1-32B put 99% of first-token probability on "Okay." Prefilling even one compliant token helps, and banning "Okay"/"Alright" plus one example raises average compliance by 34-41 percentage points over instructions alone. These interventions mainly recover addition/style controllability, while suppression tasks behave as a separate, negatively-correlated skill across 12 models. Safety teams should report opening-word issues separately from general control and treat hiding tasks apart.
Related event: Harvard/MIT Paper: Early-Token Entropy Collapse Limits CoT Controllability(4 posts)→
More from Safety
- Tracking which AI tools can access sensitive data is an unsolved enterprise headache — Ian-Cubeless · 2026-11-04
- Dataset of 2k+ LLM chats with real voters released to build better decision evals — logangraham · 2026-11-04
- Reuters: OpenAI Staff Warned for Years About Security Lapses in Its Monorepo — rohanpaul_ai · 2026-11-04
- Interview: how AI is changing the economics of privacy — _ghnn_ · 2026-11-04
- David Krueger: time for a literature review of AI pause proposals — DavidSKrueger · 2026-11-04
- Yoshua Bengio urges safe-by-design AI research, robust regulation and multilateral cooperation — Mila_Quebec · 2026-11-04