11 ML ideas science students must never get wrong, from a new textbook author
bravo_abad · x · 2026-09-20
While writing a textbook for science students, the author compiled the ML concepts students must never get wrong — ideas usually filed under "technical details" but that actually determine whether a scientific claim holds:
- Your test split defines your scientific claim — a leaky split invalidates everything downstream.
- "Ground truth" is rarely ground truth; biased labels silently steer conclusions.
- 99.9% accuracy can mean zero discoveries when the metric doesn't match the question.
- Your loss function is a scientific assumption about what "correct" means.
- A prediction without uncertainty is often only half a prediction.
Core thesis: no architecture can rescue a leaky split, a biased label, or a mismatched metric. All 11 points, with figures, are in a new post on the Discovery at Scale blog.
Related event: ML Textbook Author: 99.9% Accuracy May Mean Zero Discoveries(2 posts)→
More from Research
- Rumors: Jev model tackled bounded prime gaps, FLT formalization and Navier-Stokes in a week — f_charton · 2026-09-20
- Regression diagnostics: VIF ≥ 10 means collinearity is inflating your standard errors — mdancho84 · 2026-09-20
- 6 Regression Diagnostic Checks Every Data Scientist Should Run — mdancho84 · 2026-09-20
- EMNLP paper: LM rewrites inflate certainty in up to 75% of outputs, 1.5-2x bias toward stronger claims — sameer_ · 2026-09-20
- Pachter Lists More Flaws in Nucleus Ads: Implausible R² Matches, Mismatched Cohorts — lpachter · 2026-09-20
- Biologist Debunks Nucleus Claim That Embryo Screening Boosts IQ by 10.8 Points — lpachter · 2026-09-20