Decade-Old Warning Coming True: Model Self-Reports Collapsing Under Optimization Pressure
davidmanheim · x · 2026-11-14
AI safety researcher David Manheim argues that researchers worried about metric failure in advanced AI have long predicted that monitoring and having models explain their own views would collapse under optimization pressure — and we're starting to see that happen now. A substantive challenge to alignment approaches that rely on models' self-reported reasoning.
More from AGI Musings
- Millions of lesser intelligences already control greater ones: managers. Alignment is management — peterjliu · 2026-11-15
- WSJ Reporter Says AI Diagnosed Decades-Old Medical Mystery Her Doctors Missed — EricTopol · 2026-11-14
- Stanford AI100's third report 'AI Goes Mainstream' arrives November 2026, with a talk by Michael Wooldridge — conitzer · 2026-11-14
- Anil Seth at TED: Why AI Isn't Going to Become Conscious — anilkseth · 2026-11-14
- MLStreetTalk: AI creates novelty but true transformative creativity remains out of reach — burny_tech · 2026-11-14
- Same panic, new target: E2EE backdoor demands vs 'open models threaten humanity' — SuB8u · 2026-11-14