Apodex Discovery: Reality Benchmarks and Environments for Evaluating and Building Discoverative Artificial Intelligence
Brian Wang, Bin Feng, Xiaoman Pan, Chenyang An, Felix Liu, Tangqi Fang, Gongbo Sun, Lingfeng Shen, Ning Wang, Handuo Zhang, Feng Chen, Fuchao Yang, Xiang Wang, Jiacheng Lin, Siting Li, Zixuan Liu, Chi Han, Zhenhailong Wang, Kunlun Zhu, Lawrence Zhao, Yueqi Guo, Kailong Wen, Feng Xing, Yiling Guo, Lidong Bing, David Tan, Bo An, Heng Ji, Sheng Wang
cs.AI
2026-08-12
A framework that turns 20 real-world problems from a 561-industry survey into executable environments; AAV capsid design beats published SOTA on all four tasks, and adding a drug-repurposing environment lifts GPT-5.6-sol from 54.2 to 61.8 normalized score.
Frontier models score higher every year on exams, math olympiads, and software-engineering benchmarks, but those arenas share a hidden precondition: the problem, the tools, and the success criteria arrive pre-specified. Real research and industrial problems do not come in that shape. There is no data interface, no verifier, no feedback loop. This paper builds the missing layer, and the authors reach for the Apollo analogy: reaching the Moon was not about engineers solving hard equations, it was about converting a distant ambition into a mission architecture with explicit objectives, simulation, telemetry, and failure criteria. AI needs the same infrastructure before problem-solving turns into discovery.
Three layers.
Problem scouting first. A ten-person team of STEM PhDs spent two months surveying 561 industries across 16 sectors, assembled a registry of 423 high-value problems, and selected 20 for the initial release. Selection criteria were reasoning depth, technical structure, a verifiable pathway, and real-world value.
Environments second. The core abstraction is environment-task-episode: an environment supplies data, tools, constraints, and feedback, records the full trajectory, and verifies intermediate artifacts and final submissions. The unit under test is the complete solver, foundation model plus harness plus agent loop, never the model alone. Hidden ground truth is structurally isolated from the solver; answers cannot be retrieved from outside.
Process scoring third. HDS6 rates six dimensions independently of task success on a 0-4 scale: Tools (selection, invocation, correct interpretation), Repair (responding to verification feedback and confirming fixes), Alternatives (making competing hypotheses explicit and adjudicating among them), Coherence (state and constraint consistency over a long horizon), Evidence (grounding claims in observations and tool outputs), and Scope (identifying where a conclusion holds and where it stops). Scores read the recorded trajectory only, with the outcome and the solver's private chain of thought hidden. Across pooled environments, process scores correlate with outcomes at Spearman rho +0.51.
AAV capsid design beats the published state of the art on all four tasks:
| Task | apodex-1.1 | Published SOTA |
| Viability prediction (OOD AUROC) | 0.904 | CAP-PLM 0.878 |
| Tropism prediction | 0.635 | Fit4Function 0.622 |
| Structure prediction (3-level mean) | 0.649 | AlphaFold 3 + symmetry expansion 0.605 |
| Generative design | 0.180 | AAVDiff 0.116 / AAVGen 0.109 / ALICE 0.110 |
On drug repurposing and reformulation, the same closed-book backbone plus a task-specific biomedical environment moved GPT-5.5 from 53.78 to 56.30 normalized score (+2.52) and GPT-5.6-sol from 54.21 to 61.81 (+7.60). These are three runs on the public development set, which the authors themselves label descriptive rather than definitive.
The harness rankings deserve a look too. In controlled ablations over 11 LLM environments, the balanced setting puts ApodexHarness first (0.548), ahead of A-Evolve (0.544) and DeerFlow (0.521), with OpenHands last; on models, GPT-5.6-sol leads at 0.595. Harness effects persist when the model is swapped.
Two directly usable conclusions. First, there is now hard evidence on biology tasks that investing in the environment can beat investing in the model: the same GPT-5.6-sol went from 54.2 to 61.8 with the right environment, a larger delta than most model swaps. Second, process scores work as diagnostics. GPT-5.5 trailed kimi-k3 by 20% on structure prediction (0.544) because it skips the self-validation step the task requires and submits its first-pass answer; its HDS6 process score fell to 1.38. Failures can be attributed to a specific component instead of a vague "the model is not good enough".
The authors list their own: 17 environments and 218 episodes cover a small corner of the 423-problem registry; ground truth for high-value problems may be delayed or incomplete, forcing proxies such as held-out experimental fitness instead of in-vivo efficacy; frontier problems still need expert adjudication. Two things this reader flags. The relationship between apodex-1.1 and the baselines stays vague: in the SOTA tables apodex-1.1 is a solver training models inside the environment, while the comparison methods are task-specific specialists, and whether that framing is fair depends on reproduction details the paper does not fully surface. And the benchmark comes from a single company that built the environments, the verifiers, and the process scorer, so external reproduction will not be cheap.