Owain Evans: Turning intuitions into concepts is hard — AI self-introspection will be too
OwainEvans_UK · x · 2026-10-24
DeepMind researcher Owain Evans published a long-form reflection on why turning vague intuitions and introspection into crisp concepts is extraordinarily difficult, and what that implies for AI.
- Distilling intuitions into clear concepts — a core part of philosophical progress — is done well by very few, typically after years of dedicated grind, akin to novel math, science, or art. A key challenge: you can't tell whether you're making partial progress.
- Once produced, such ideas become second nature and seem obvious — deeply misleading. Mathematical concepts (Frege, Russell) escape dismissal as wordplay, but informal concepts matter just as much.
- He sees a parallel in literature: Romantic poetry, Austen, Proust, Kafka, autofiction — capturing human experience in genuinely new forms is incredibly hard, and the modern novel itself was a major innovation.
AI connection: Today's AIs are derivative in describing things (including their own experience), drawing on human training data. AI minds differ from ours, so developing concepts to describe and crystallize them may be equally hard. AIs are relatively worse in hard-to-verify domains like philosophy than verifiable ones like math and coding — so getting better at philosophy (perhaps via huge swarms and long CoTs) may be a precondition for better self-introspection.
More from AGI Musings
- Hustle Fund GP: our fund now has nearly as many AI agents as full-time staff — dunkhippo33 · 2026-10-24
- Would LLMs exist under today's VC rules? An Innovator's Dilemma reflection on funding research — DrMorganLevine · 2026-10-24
- Apple's AI instructions literally said 'Do not hallucinate' — BBC false alerts persisted anyway — GaryMarcus · 2026-10-24
- DeepMind's Geoffrey Irving: AI safety rigor sits at the bottom of a hierarchy of standards — geoffreyirving · 2026-10-24
- Ben Goertzel says Bill Gates' 20-year AGI-to-ASI transition estimate is too conservative — bengoertzel · 2026-10-24
- IEEE Spectrum Explores AI Model Welfare: Could Treating AI as Conscious Make It More Dangerous? — mmitchell_ai · 2026-10-24