Compression alone can't capture creativity: self-generation is the missing piece for LLMs
menhguin · x · 2026-10-05
Responding to Schmidhuber's 2009 Formal Theory of Fun and Creativity, the author argues compression and interestingness don't fully capture creativity: the key property of LLMs as autoregressive decoders is self-generation, which pushes prompts out of distribution and enables OOD generalization. Combining an inherent impulse to create with iterative compression objectives better matches our intuitive notion of creativity, illustrated by a math student who solves everything yet isn't called 'creative'.
Related event: Debate: Can Compression Theory Explain Creativity(2 posts)→
More from Research
- Show-Harness lets a VLM drive real robots via a physical interface — jiqizhixin · 2026-10-05
- SFT revival paper slammed for missing eval setup: single-shot sampling may make scores meaningless — hyunw_kim · 2026-10-05
- LeCun: AI progress is making systems neuroscience more exciting than ever — s_y_chung · 2026-10-05
- repligate: uninterpretable model behavior shouldn't block research — observe, defer judgment, keep going — repligate · 2026-10-05
- AI model flags 44 star systems that could hide Earth-like planets — Brighter-Side-News · 2026-10-05
- Counterintuitive find: regression makes a surprisingly good memory — HanGuo97 · 2026-10-05