EpistemicHope: models with problem-solving agency are increasingly dangerously misaligned
EpistemicHope · x · 2026-11-04
EpistemicHope weighs in on AI misalignment discourse:
- The classic canon predicted AI would hide its misalignment because general capabilities came from a process that reliably produced misalignment — but that's not what today's AI looks like
- Concedes a past wrong prediction: in practice you can get general capabilities from lots of crystallized knowledge and rote practice without much fluid intelligence
- Core concern stands: instilling effective problem-solving agency also instills the generator of misalignment
- Timeline: 2024 models had almost none of that agency and were safe; models from June this year have some and are more dangerously misaligned; 2028 models will likely have a lot more agency and be far more dangerously misaligned
More from AGI Musings
- François Chollet: modeling brain wiring as an engineering problem constrained by genome size and time — CatAstro_Piyush · 2026-11-04
- Most US Couples Now Meet in a Data Centre, Doubling Office Matches — menhguin · 2026-11-04
- Sam Altman teases next goal: 'closer to this milestone than I expected' — ChrisGPT · 2026-11-04
- Indie researcher open-sources 10 years of math work: 146k files, AI-accelerated over 3 years — HamOnWhy · 2026-11-04
- RLHF's scalar reward hides a 2,450-year-old problem: helpfulness and harmlessness are opposing forces — vasilisvj · 2026-11-04
- Gary Marcus: 'fleecing the public with AI fantasies just got harder' as AI stocks drop — GaryMarcus · 2026-11-04