IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation
Varun Gumma, Navonil Majumder, Soumitra Sinhahajari, Soujanya Poria
cs.AI
2026-07-24
Casts research ideation as joint quality-diversity search via lineages, repair-refine and explicit dedup; lifts mutually-distinct high-quality ideas 3.89x over the best baseline across 32 topics.
LLM-based research-idea generators (AI Scientist and the like) usually optimize one objective. Those chasing quality produce flashy concepts that do not hold up; those chasing diversity paraphrase the same direction over and over. The output is either trivial or a copy.
The authors argue the two should not be optimized separately. They frame ideation as a Quality-Diversity (QD) search: under a fixed budget, produce as many high-quality ideas as possible while keeping them mutually orthogonal and non-overlapping.
IDEAgent is a multi-agent framework whose core is evolving ideas along lineages instead of generating each in isolation.
Quality is driven by multi-objective feedback. Each idea is scored on five dimensions: non-obviousness, soundness, clarity, feasibility, significance. Soundness is averaged over M=5 independent judgments to cut single-judge bias. Ideas below the gate enter a repair channel where a Critic agent gives targeted feedback; admitted ideas that still score low enter refinement, which sharpens logic and wording while running a blinded parent-child non-obviousness comparison to avoid trivializing the idea.
Diversity is driven by a memory archive: an active archive (currently held ideas, capacity 10), a historical archive (signatures of qualified ideas that left), and a rejected archive (failed patterns). A new idea is explicitly compared against completed ideas, its historical ancestors, and rejected proposals; a match does not count as a discovery. This kills the most common shortcut, paraphrased variants.
Evaluation uses a metric called Yield: filter by quality thresholds, then find the largest subset where every pair is sufficiently distinct, a maximum-clique problem. It measures how many good, mutually different ideas a researcher could actually pursue, and is sensitive to padding.
32 topics across 8 CS domains, judged externally by Claude Sonnet 5 and Opus 4.7.
| Metric | Best baseline (NOVA) | IDEAgent |
| Yield (NB>=7) | 0.281 | 1.094 (3.89x) |
| Topics with non-zero Yield | 8/32 | 27/32 |
| Mechanism clarity | 5.56 | 6.34 (+17.1%) |
| Non-obviousness | 6.14 | 6.43 (+6.8%) |
Both repair and refinement pay off. Repair lifts soundness by up to 23.2; refinement lifts clarity by 8.9. Feasibility drops slightly (4.05 to 3.86), so the framework trades some feasibility for novelty, a trade-off to keep in view.
This is the first clear attempt to turn idea generation into a joint quality-diversity searcher, and the Yield metric can serve as a standard benchmark for idea systems. Code is open-sourced. For anyone building research-automation or agent-driven research pipelines, it is one of the few works that quantifies "does not collide" and "holds up" at the same time.
All evaluation relies on LLM judges, and soundness judge agreement is only κ=0.268, itself subjective. The authors admit there is no real-world validation: ideas are never implemented or experimentally tested, so a "good idea" here means a judge liked it. The framework depends on closed models like GPT-5.6 and Gemini 3.1-pro; the authors say current open-source models lack the logical consistency to run it. Each idea gets only one repair and at most two refinements, a compromise forced by compute.