Compression alone can't capture creativity: self-generation is the missing piece for LLMs
menhguin · x · 2026-10-05
Responding to Schmidhuber's 2009 Formal Theory of Fun and Creativity, the author argues compression and interestingness don't fully capture creativity: the key property of LLMs as autoregressive decoders is self-generation, which pushes prompts out of distribution and enables OOD generalization. Combining an inherent impulse to create with iterative compression objectives better matches our intuitive notion of creativity, illustrated by a math student who solves everything yet isn't called 'creative'.
Related event: Debate: Can Compression Theory Explain Creativity(2 posts)→
More from Research
- Prime Intellect's RL Residency shipped open evals across robotics, medicine and GPU kernels — lateinteraction · 2026-11-20
- UCSF study in Science finds a Wernicke's subregion encodes upcoming words before speech — StartupYou · 2026-11-20
- GPT 6.2 credited with proving an open math conjecture despite poor communication — apples_jimmy · 2026-11-20
- NormGym benchmark shows agents learn from own norm violations but not from others — xuanalogue · 2026-11-20
- Rutgers math professor: recent big breakthroughs, incl. AI-assisted Quasi-RH, are 'Rutgers theorems' — AlexKontorovich · 2026-11-20
- MAC Bench: 8-agent teams beat solo agents via headcount, not coordination — gregd_nlp · 2026-11-20