Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
Kevin Murphy
cs.AI
2026-08-10
MDA couples an LLM proposer with Bayesian value-of-information experiment design; on physics, chemistry, and a new neuron benchmark it reaches ceiling accuracy in about 8 experiments, beating pure-LLM agents that need about 41.
Answering "what if I did a?" takes more than a model that fits observed data. Two mechanisms can explain all historical data perfectly yet disagree sharply under an intervention never tried. Interventional questions need a mechanistic, causal model, not a curve fit. The catch is that passive data cannot identify the mechanism; only intervening and watching the response breaks the degeneracy. Experiments are expensive, so the central problem is data efficiency, pinning the mechanism in as few interventions as possible.
Kevin Murphy (UBC) proposes the Model Discovery Agent (MDA), splitting labor between an LLM and Bayesian machinery:
The headline finding is that discovery and design reinforce each other: design identifies the mechanism discovery proposes, the identified mechanism sharpens predictions, and the sharper predictions expose subtler residuals that trigger the next discovery. The Yukawa world makes this vivid. The screened kernel is nearly indistinguishable from a power law at short range, so only a long-range probe separates truth from near-miss. VoI picks the long-range probe automatically, and the true model drops to the corner of the accuracy-complexity frontier, an "aha" moment in the authors' words.
Three benchmarks, budget B of at most 8 experiments:
| Benchmark (domain) | Key result |
| ForceBench (physics, wrapper on DiscoverPhysics, 6 two-particle worlds) | MDA lifts the cheap DeepSeek-v4 to near-Opus accuracy; the pure LLM agent stays weak and needs about 41 experiments in unthrottled mode to reach nMSE 0.013 |
| ChemBench (chemistry, 36 tasks) | MDA hits its ceiling in about 8 experiments at roughly 56% symbolic accuracy; the prior SOTA, LLM-AutoSciLab, reaches only 42% at B=60 |
| NeuronBench (biology, new, 6 mystery Hodgkin-Huxley neurons) | The Bayes forecaster beats the in-context forecaster on every world; random design is worst |
The chemistry result has a counterintuitive edge worth pulling out. LLM-AutoSciLab's symbolic regression (PySR) fits expressions with RMSLE as low as 0.001, but they are mechanistically meaningless nested logs and stretched exponentials. MDA returns interpretable mechanisms and hits the exact true form on substrate inhibition. The paper calls this the high-numeric, low-symbolic pathology, the classic failure mode of pure fitting.
NeuronBench also has a stochastic variant (channel-gating noise turns the model into an SDE with intractable likelihood), which needs a particle filter and is heavy to compute. A 1D CNN learns a summary statistic and speeds it up by about 10 to the 4th power.
For AI4Science and automated discovery, the division of labor is clean: the LLM contributes hypothesis proposal from priors, Bayesian methods contribute uncertainty quantification, experiment design, and interpretable models. Neither is new, but the combination, plus M-open self-correction that expands the hypothesis space, is the point. The recovered models are readable mechanistic equations, not black-box curves.
All three benchmarks are synthetic (the truth is known), which enables symbolic-equivalence checks but leaves a gap from real wet labs. The budget B is at most 8. NeuronBench is newly built here; the other two are wrappers on existing benchmarks. The stochastic variant is computationally heavy, with a particle filter nested inside SMC. Results depend on LLM proposer quality, and M-open correction only expands the pool once. Single-author paper.