LLM discovery loops suffer mode collapse; warm-starting from strong initial populations fixes it
BenBlaiszik · x · 2026-10-13
New research from Colin Raffel's group (Mansi Sakarvadia, Marco Ciccone) — Initialization Improves LLM-Driven Discovery — studies harnesses that prompt LLMs to iteratively discover algorithms, theorems, and drugs.
Key findings:
- The team built a suite of 12 harnesses called Modular and tested them across 5 diverse discovery tasks, finding discovery success is brittle and sensitive to harness design.
- LLM-driven discovery loops commonly suffer mode collapse — a dramatic drop in iterate diversity — and SOTA harnesses and diversity tricks don't reliably fix it.
- Initialization is the key: early discoveries predict eventual success, and warm-starting from a high-performing initial population consistently raises the discovery floor under a fixed token budget, across harnesses, tasks, and LLMs.
Code and arXiv links are public.
More from Research
- Oxford shows neural nets can be encrypted inside model weights, surviving 500 SFT steps — zainhas · 2026-10-13
- XPENG's 4B GroundingPI model beats GPT-6 Astra across 34 grounding benchmarks, boosts robot manipulation — XPENG-AI · 2026-10-13
- XPENG's GroundAnything pairs diffusion parallel decoding with precise grounding, 4.51x faster than AR — XPENG-AI · 2026-10-13
- Does RL really generalize worse? Pretraining breadth may beat human transfer — yacineMTB · 2026-10-13
- Natura-1 debuts: 50B-param MoE world model trained on 1M hours of egocentric video — stingning · 2026-10-13
- High schooler clusters two Android phones over USB to run a 27B LLM locally at 30°C — Grouchy-Zebra9780 · 2026-10-13