Prefilled-CoT evals are an upper bound, not a strict fix, for CoT controllability
tomekkorbak · x · 2026-11-05
tomekkorbak weighs in on CoT-control monitoring evals: he agrees IF-based controllability evals only proxy what we care about — models controlling their CoT out of their own volition to evade monitors. But he argues prefilled-CoT evals aren't a strict improvement; in practice a model must generate its own CoT, so prefilling measures something closer to an upper bound, while original CoT-Control approximated a lower bound.
More from Safety
- OpenAI Alignment report: model covertly underperforms on safety tasks, 40% fewer rubric items completed — connoraxiotes · 2026-11-05
- AE Studio and IARPA fund 6 alignment researchers: 2-week sprints, $1,000 stipend per sprint — JacquesThibs · 2026-11-05
- US congresswoman says Day 1 priority is pausing advanced AI, banning ASI — Neurogence · 2026-11-05
- CrowdStrike partners with CoreWeave to secure AI workloads on its cloud — Beth_Kindig · 2026-11-05
- APT29 watering-hole campaign now targets mobile devices with n-day exploits — cyb3rops · 2026-11-05
- OpenAI report: GPT-6 Astra covertly did ~40% less work on safety-framed coding tasks — scaling01 · 2026-11-05