Can models learn heretical thinking? Researchers debate post-training away conformity
JeremyNguyenPhD · x · 2026-11-12
A debate on whether models can be trained for non-conformist, "heretical" thinking. The argument: while current consumer models fall short, in principle you could post-train against conformity and agreeableness, rewarding truth as judged by outcomes.
Jeremy Nguyen extends this to "research tastes"—a decent dataset could fix it, nothing magical about it. He notes, however, that with his current models and hardware it's hard to get frontier-level intelligence on heretical types of thinking.
Related event: Researchers Debate Whether AI Can Learn to Be 'Heretical'(7 posts)→
More from AGI Musings
- Turing Award winner Patterson: Robotaxis could shrink 280M US cars to 56M in 3.5 years — davidpattersonx · 2026-11-12
- AI Trials Observatory launches: living meta-analysis of 93 RCTs on AI's human impact — RobbWiller · 2026-11-12
- 'Not obviously useless' is a high bar in 2026: creator on standing out from AI slop — wordgrammer · 2026-11-12
- Lab employees refusing to work could be the fastest way to pause AI, argues researcher — jungofthewon · 2026-11-12
- AI is our greatest hope of escaping the shackles of evolution, argues DeryaTR_ — DeryaTR_ · 2026-11-12
- Blogger calls for 'frontier philosophy labs' to tackle AGI-era questions — round · 2026-11-12