ROGUE: first provable corrigibility framework shows deployed RLHF is insufficient
aran_nayebi · x · 2026-11-12
Aran Nayebi's team presents ROGUE, the first provable framework for building AI systems that are corrigible—shutting down when asked, telling the truth, preserving oversight—while remaining useful, unlike RLHF or Constitutional AI which can fail when goals conflict.
Experiments with ROGUE demonstrate what theory already implied: the single unrestricted reward-shaping strategies widely deployed in today's RLHF/RLAIF are insufficient to guarantee corrigibility. The team also links to promising theory with guarantees via a Twitter thread and a longer LessWrong post.
More from Safety
- UK elite schools ditch laptops while poor pupils get AI chatbot tutors — AlexTensor · 2026-11-13
- Ought launches Epistemics Fund with $500k–$5M for Community Notes research — jungofthewon · 2026-11-13
- Claude nearly executes instructions read from an article, sparking injection concerns — npmer · 2026-11-13
- TSMC confirms leak of ~46K employee records offered for 3 BTC after cloud vendor breach — rohanpaul_ai · 2026-11-13
- TSMC hacked: Nina Schick argues security is downstream of intelligence — NinaDSchick · 2026-11-13
- 22% of Nvidia sales went to Singapore, fueling China smuggling claims — SumitGup · 2026-11-13