Researchers debate: pretraining gains may owe more to human curation than compute scaling
davidmanheim · x · 2026-09-09
Responding to Toby Ord's critique of the "just add compute/data" framing, davidmanheim argues marginal evidence suggests more of pretraining improvement comes from human-driven quality work, implying pure scaling trajectories are flatter than claimed. He also raises whether "quality-scaling" itself is bounded.
Related event: Scholars Debate Whether Pretraining Scaling Is Slowing Down(4 posts)→
More from AGI Musings
- Khosla wants FDA to certify AI that beats the median doctor; ER physicians push back — DrDatta_AIIMS · 2026-09-09
- Elon Musk: AI and robots will more than double the global economy within 10 years — DimaZeniuk · 2026-09-09
- OpenAI plans ~$1T compute spend; expert sees it fueling AGI, not inference — NinaDSchick · 2026-09-09
- Anthropic staffer: >10% chance AI kills humanity this decade; a16z's Casado pushes back — venturetwins · 2026-09-09
- Mockery Turns to Confusion as Rogue-AI Skeptics Watch What Lab Insiders Do — jeremiecharris · 2026-09-09
- Sandberg: The Right Goal May Be Simple, Thanks to Kolmogorov Priors — anderssandberg · 2026-09-09