Re-Search paper: LLMs can judge good research ideas but struggle to generate them
DanielKhashabi · x · 2026-11-17
LLMs now write research papers, resolve decades-old Erdős problems, and even produce claimed proofs of Navier–Stokes — but we only see final results, never the ideation behind them. Re-Search (arXiv:2611.14487, Muhan Gao, Daniel Khashabi, et al.) turns this into a game of Twenty Questions:
Method
- Given only a published paper's research problem, a model must rediscover its core idea by asking yes/no questions (100 max)
- Built from 200 ICML 2026 papers; tracks how the model's idea evolves with binary oracle feedback, isolating ideation from execution
Key findings
- Across six recent LLMs, nearly all progress happens within the first 30 of 100 questions; most models plateau afterward
- Controlled interventions show the bottleneck is generation, not judgment: models recognize a good idea or question when offered but often can't produce one themselves
- Failure modes: over-exploiting one direction, or over-exploring too many
Design implication: since models can judge candidates they cannot produce, autoresearch systems should enrich search with externally sourced ideas and questions.
Related event: Re-Search: A 20-Questions Benchmark for LLM Research Ideation(2 posts)→
More from Research
- BrickFlow uses SE(3) flow models to reconstruct buildable LEGO assemblies from images — Peter Kulits · 2026-11-17
- Meta-model writes weight updates from one paragraph, beating LoRA on many benchmarks — gdm3000 · 2026-11-17
- SPARE preprint: 2.55x faster decoding for linear attention on a 1.3B model — krishnan · 2026-11-17
- Study estimates ~12% of philosophy published today may be AI-generated — leland_mcinnes · 2026-11-17
- Berkeley EECS & Statistics open all-rank faculty search in AI for Physical Sciences — berkeley_ai · 2026-11-17
- Origami as physical reservoir computing: powering robotic crawling in Scientific Reports — cephaloform · 2026-11-17