Re-Search benchmark shows LLMs can spot good research ideas but can't generate them
Muhan Gao · hf · 2026-11-17
Researchers at Texas A&M and Johns Hopkins introduce Re-Search, a benchmark that adapts the Twenty Questions game to research ideation: given only a published paper's problem statement, a model must rediscover its core idea by asking yes/no questions.
Key findings:
- LLM progress is front-loaded, with most gains early and plateaus later;
- Models can recognize good questions and ideas but often cannot generate them;
- The authors suggest automated research systems may benefit from externally sourced candidate ideas rather than purely self-directed exploration.
The work offers a measurable framework for pinpointing where LLM ideation breaks down.
Related event: Re-Search: A 20-Questions Benchmark for LLM Research Ideation(2 posts)→
More from AGI Musings
- Silicon Valley's new obsession: agent swarms that beat frontier models via parallelism and specialization — briannekimmel · 2026-11-17
- Will 95% of People Stop Reading Books and Just Chat with Agents? — miniapeur · 2026-11-17
- Gary Marcus: the real risk is superintelligence before alignment, not lacking it — GaryMarcus · 2026-11-17
- David Patterson predicts no personal income tax by 2030 because there will be no jobs — davidpattersonx · 2026-11-17
- Armin Ronacher uses Princess Mononoke's Irontown as a metaphor for AI's societal role — mitsuhiko · 2026-11-17
- Study estimates ~12% of philosophy published today may be AI-generated — leland_mcinnes · 2026-11-17