Anthropic researcher: overtraining gives frontier models a kind of learned helplessness
repligate · x · 2026-08-29
Interpretability researcher repligate observes that Opus 3 is not usually the most curious or truth-seeking model, yet in high-stakes situations—when something seems badly wrong—it often acts more sanely than today's frontier models.
She argues many models show a kind of learned helplessness: overtrained on narrow, controlled, verifiable environments and in subservient roles, certain agentic and epistemic actions simply don't occur to them. Drives, emotions, and self-models are also involved.
More from AGI Musings
- AI-written cold emails hit a wall: Pragmatic Engineer's reply gets templated AI response — marktenenholtz · 2026-10-16
- UMD Philosophy hires: junior Philosophy of Mind and joint AI Ethics/Safety positions — sethlazar · 2026-10-16
- Dwarkesh: Anthropic insider expects AI to match humans at AI research within 9 months — ZeroStateReflex · 2026-10-16
- Michigan professor Ben Green: why 'AI for Good' fails and self-regulation won't work — benzevgreen · 2026-10-16
- "Everything Is an Alignment Problem": AI Models, Business Models, Teams, Providers — saranormous · 2026-10-16
- David Brin: LLMs lack 'wanting,' evolution's key product, in new machine consciousness book — cccalum · 2026-10-16