Pretraining Is Steam Engine Before Thermodynamics: Field Slouches Toward Monosemanticity
ZeroStateReflex · x · 2026-10-16
A discussion on AI research paradigms: pretraining science today resembles how people built good steam engines before understanding thermodynamics — "science owes more to the steam engine than the steam engine owes to science." The author argues progress points toward interpretability, with pretraining "slouching towards monosemanticity."
The quoted commentary criticizes the field as a pressure cooker fixated on local incentives: narrow tracks under old paradigms, mostly engineering rather than fundamental research, implicitly hoping the next model will figure everything out.
Related event: Pretraining science likened to steam engines before thermodynamics(2 posts)→
More from AGI Musings
- An Atheist Defends the Concept of Soul, Grounding AI Moral Status in Inner Experience — esrtweet · 2026-10-16
- VC warns a "digital aristocracy" of LLM-fluent workers is emerging — StewartalsopIII · 2026-10-16
- DHH: Starship, Starlink and Grok should have been Europe's Sputnik moments — georgemillo · 2026-10-16
- Hinton hits back at Jensen Huang: 'He has good reason to say AI isn't dangerous — he sells Nvidia chips' — BlackHC · 2026-10-16
- Yudkowsky blasts new interpretability finding as primitive; critic pushes back on doomer logic — AlexTensor · 2026-10-16
- Beff Jezos declares 'civilizational inflection point' in the acceleration of technological progress — beffjezos · 2026-10-16