Agentic AI Scientists Are Not Built For Autonomous Scientific Discovery
Harshit Bisht, Vinay Kumar, Kevin Maik Jablonka, Mausam, N. M. Anoop Krishnan
NeurIPS 2026 (Position Paper Tr
cs.AI
2026-05-09
NeurIPS position paper: agentic AI scientists aren't built for autonomous discovery; 12 models from 4 providers converge on near-identical hypotheses, adding 0.7 distinct ideas.
A growing family of systems claims end-to-end autonomous scientific discovery: pick the problem, form hypotheses, run experiments, write it up, with no meaningful human involvement. Funding bodies are buying in, and the Nobel Turing Challenge names such a system as its endpoint. This NeurIPS 2026 position paper (IIT Delhi and Friedrich Schiller University Jena) argues the opposite: current agentic systems are co-scientists at best, and the gap is structural, not a matter of scale or scaffolding. The dividing line is how fast a system learns it is wrong. Proof assistants answer in seconds, which is why autonomy claims in mathematics get a pass; natural science closes the loop through physical experiments that are slow, partial, and need human interpretation, and that is where the paper plants its argument.
The paper is mostly argument, organized around four failure modes: problem selection bends toward the McNamara fallacy, so measurable, data-rich problems get done; published corpora omit tacit laboratory knowledge and failure knowledge; RLHF and DPO compress output diversity toward annotator consensus; and scientific benchmarks reward single-turn prediction with no feedback from physical experiments back to the model.
The one new experiment, hypothesis hivemind, tests the diversity claim. Twelve frontier models from Anthropic, Google, Moonshot AI, and OpenAI (four open-weight, spanning GPT-5, Gemini 3.1, Claude Sonnet 4.6, and Kimi K3) each produced 10 outputs per paper over 50 NeurIPS 2025 AI4Mat papers and 52 PLOS Biology papers. Task A reconstructs a paper's hypothesis from an experiment summary, a determinate-answer baseline where convergence is expected. Task B reads the full text and proposes new hypotheses, where diversity is the point. Outputs are embedded, and the authors compare within-model against cross-model cosine similarity, plus Vendi scores for the effective number of distinct hypotheses.
| Metric | Setting | Result |
| Cross-model vs within-model similarity | AI4Mat, hypothesis generation | 0.64 vs 0.72 |
| Cross-model vs within-model similarity | PLOS Biology | 0.69 vs 0.76 |
| Vendi effective hypotheses (10 models vs 1 model, 10 samples) | AI4Mat | 3.6 vs 2.9 |
| Vendi effective hypotheses (same setup) | PLOS Biology | 3.1 vs 2.6 |
Switching providers buys almost no marginal diversity. The cited external evidence is just as blunt: across 41.3M papers, AI-augmented scientists publish 3.02x more and collect 4.84x more citations, while the collective volume of topics studied shrinks 4.63% and scientist-to-scientist engagement drops 22%; across 25,000+ agent runs, evidence was ignored in 68% of traces and refutation-driven belief revision appeared in only 26%, with the base model explaining 41.4% of task-success variance against 1.5% for the scaffold; and of GNoME's 2.2M predicted crystal structures, 736 had been experimentally realized at publication time.
Two takeaways for AI4Science teams. First, do not count on querying multiple providers to hedge a single model's bias: outputs from four providers cluster in the same semantic region, so synthesis and characterization resources flow toward candidates the literature already emphasizes (the running example is familiar solid-state-electrolyte families such as LLZO and NASICON). Second, a high score on a scientific benchmark says little about running a sustained investigation; single-turn accuracy skips the step where an anomaly revises the hypothesis and the reward. The proposed fixes are infrastructure-shaped: scientific simulators such as OpenMM and MuJoCo as scalable verifiers for training, a persistent mutable epistemic state (a Bayesian network over hypotheses), and a centralized preregistration repository for AI-generated hypotheses. This is a framing paper rather than an incremental improvement, and its value is the chain it builds from scattered evidence.
The authors concede the hard parts themselves: the preregistration repository needs community consensus and compliance, annotated laboratory video is expensive and hard to standardize, and training on public reviews raises reviewer-consent and copyright issues. No implementation path is given; all six recommendations stay at the direction level. The experiment is also open to challenge: two domains and roughly a hundred papers, 10 samples per cell; embedding cosine similarity can be inflated by shared phrasing and terminology, so semantic proximity does not prove the hypotheses are substantively identical; and models converging on the obvious extension of a paper could be reasonable convergence, an explanation the paper does not separate from pathological homogeneity.