New paper: LLM creativity is 'jagged' — test scores don't predict real-world diversity
burny_tech · x · 2026-11-11
A new paper by Sam Schapiro and colleagues argues LLM creativity is "jagged": a high score on a creativity test does not imply diverse creative responses on real-world open-ended questions.
Key findings:
- No semantic distance test significantly predicts output diversity
- The popular human creativity test RAT does not measure LLM creativity well — it mostly reflects general capability
- The DAT (divergent association test) does predict creative writing performance
The authors argue LLM creativity evals need to be rethought from first principles.
More from Models
- Dev trains small CoreML decision model, beats his baselines by a wide margin — cyrilzakka · 2026-11-11
- Fable 5.2 audits an Opus 5.5-built codebase and finds issues Opus missed — CtrlAltDwayne · 2026-11-11
- New model Carbon looks strong but runs 3x slower than Opus 5.5, says Bindu Reddy — bindureddy · 2026-11-11
- Nat Lambert: Jensen Huang leaving NVIDIA CEO role would badly hurt US open models — natolambert · 2026-11-11
- Gemini quietly renames models and appears to debut unannounced 4 Carbon and 4 Argon — Star-Light-9698 · 2026-11-11
- GPT Image 2.6 Extra High called unusable by Pro user amid influencer praise — StudyDemon · 2026-11-11