Pretraining science likened to steam engines before thermodynamics
Researcher tszzl draws on Andrew McCalip's essay on Watt's steam engine, arguing that pretraining science resembles engineers building good steam engines before thermodynamics was understood, and is now stumbling toward interpretability and monosemanticity.
2026-10-16 ~ 2026-10-16 · 2 related posts
- tszzl: Pretraining science is like steam engines before thermodynamics, slouching toward interpretability — tszzl · 2026-10-16
- Pretraining Is Steam Engine Before Thermodynamics: Field Slouches Toward Monosemanticity — ZeroStateReflex · 2026-10-16