1,500 AI-for-Science Papers in a Week Share One Question: Compared With What?
bravo_abad · x · 2026-11-12
Discovery at Scale reviews 1,500 AI-for-science papers published Nov 4–10 and distills one lesson: a result is only as strong as its comparison. Five cases:
- Three foundation models (RNA, human gait, collider physics) shipped, but none reports the same model trained from scratch on the same inputs — so pretraining credit can't be separated from better architecture or extra information.
- Fusing two cell foundation models rarely helped: a single gene-expression model fine-tuned on 1% of labels beat every fusion method.
- Extreme rain forecasting: GraphCast beats a 3km physics model point-by-point, but over matched areas the physics model leads at short lead times — rankings depend on scoring.
- An exoplanet radiative-transfer emulator is >800x faster than its reference on GPU with large batches, but only 6x on CPU one profile at a time.
- A delirium alert held AUROC 0.91 at a second hospital, with about one true case per four alerts.
Method takeaways: choose comparisons that match the decision; retrain from scratch with identical raw inputs to test pretraining claims; a close fit still needs independent evidence.
More from Research
- Bronstein releases 78-page book: Mathematical Foundations of Geometric Deep Learning — caglar_ee · 2026-11-12
- Most biology AI models learn only one layer, and the big bet hinges on mapping — anshulkundaje · 2026-11-12
- Anthropic details agent memory: dreaming organizes Claude's memory, wiki+search hits 61% — RLanceMartin · 2026-11-12
- Talk preview: scaling 1,000 Fable 5.2 agents to formalize Fermat's Last Theorem — RLanceMartin · 2026-11-12
- Probing Qwen3 with a height ruler: each model's implicit "I" has a different height — Sauers_ · 2026-11-12
- Claude agent finds dwarf planet candidate in public sky data; $100K grants launched — jzlegion · 2026-11-12