Kombine benchmark of 35 LLMs finds an abstraction bottleneck and inventive homogeneity
profjamesevans · x · 2026-10-17
An arXiv paper introduces Kombine, a benchmark for combinatorial creativity that tasks LLMs with discovering associations, analogies, and blends between unrelated entities. A large-scale study across 35 LLMs found: models are 10-16 percentage points better at associations/analogies than blends; even frontier models show an abstraction bottleneck in forming coherent blended spaces; about one in five invented concepts reuses properties from other models; reuse is 14x more likely on blends than analogies; and models from the same provider are 2.3x more likely to reuse each other's properties. The authors say the framework can track progress toward creative AI systems and inform human creativity research.
More from Research
- Work in progress: geometric structure of GPG projections as Markov chain coarsening — michaelchchoi · 2026-10-18
- Backprop-free pretraining gets 2.3x speedup as Noise method crushes Dust record on 8 GPUs — inductionheads · 2026-10-18
- q3dviewer: open-source Qt+OpenGL tool for painter-like point cloud editing — 4310sy · 2026-10-18
- Smarter Every Day's Destin to defend PhD dissertation on impact fusion — CatAstro_Piyush · 2026-10-18
- Brain implants are restoring speech to people with ALS, Science reports — EricMTrautmann · 2026-10-18
- Formal Verification of LLM-Generated Code: Slides from FATTW 2026 Talk — CatAstro_Piyush · 2026-10-18