ATHENA Hits 94.7% Drug Reasoning Accuracy, Beating GPT-5 by 17.8 Points
marinkazitnik · x · 2026-07-07
On 3168 drug reasoning questions (DrugPC) and 456 patient-specific treatment cases (TreatmentPC), ATHENA achieved accuracies of 94.7% and 82.9% respectively, outperforming GPT-5 by 17.8 and 10.7 percentage points, and significantly surpassing DeepSeek-R1 (671B). The study found that providing GPT-5 with optional tool access barely improved its accuracy—it proactively used tools only about 1% of the time. This indicates that tool access itself is not the bottleneck; the key factor is whether the model can recognize its own knowledge boundaries and actively query information, rather than confidently providing answers when uncertain.
More from Research
- UniAction: unified action conditioning lets one world model handle mixed action types — ZeYanjie · 2026-10-27
- Artificial Analysis breaks down all sub-evaluations in Intelligence Index v4.3.2 — ArtificialAnlys · 2026-10-27
- Infini-AI Lab open-sources Hermit, a framework for agent harness evolution and continual learning — BeidiChen · 2026-10-27
- Meta FAIR's Self-Direction turns agent failures into reusable LoRA skills, +13.8pp on Terminal-Bench — karen_ullrich · 2026-10-27
- DONIC hits 86.79% on bimanual dexterous tracking, crushing prior SOTA's 9.43% — liuziwei7 · 2026-10-27
- Why don't more LLMs ship 10M-token context? A researcher asks about attention tradeoffs — agarwl_ · 2026-10-27