AI for Science weekly: why model performance isn't scientific evidence yet
bravo_abad · x · 2026-10-09
This week's Discovery at Scale briefing ties together enzyme engineering, charged-defect modeling and machine-checked proofs around one question: what makes a model's output usable as scientific evidence? Highlights: a positive-control study showing duplicate perturbation samples lose to a mean predictor on 95% of perturbations in a widely used K562 dataset under MSE across 14 datasets and 18 metrics; force fields that describe crystals accurately yet miss the electronic variable driving defects; and formal proofs that can pass checkers while not proving the intended statement. Evaluation design, not raw performance, is the bottleneck.
More from Research
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23
- Perceptic launches PercepticOS, an AI workspace for 'scientific superintelligence' — nathanbenaich · 2026-10-23
- EPFL's Bosselut to talk long context as test-time learning at ELLIS seminar — andre_t_martins · 2026-10-23
- DRATS: Adaptive Task Sampling Boosts Worst-Task Performance in Multi-Task RL — Cohere · 2026-10-23