Researchers debate: pretraining gains may owe more to human curation than compute scaling
davidmanheim · x · 2026-09-09
Responding to Toby Ord's critique of the "just add compute/data" framing, davidmanheim argues marginal evidence suggests more of pretraining improvement comes from human-driven quality work, implying pure scaling trajectories are flatter than claimed. He also raises whether "quality-scaling" itself is bounded.
Related event: Scholars Debate Whether Pretraining Scaling Is Slowing Down(4 posts)→
More from AGI Musings
- After Microsoft's 60K layoffs, pundit predicts 50-60% headcount cuts at big companies — krishnan · 2026-10-27
- 'Explain how AI isn't impacting productivity': 27% workforce cut sparks debate — alvelda · 2026-10-27
- David Patterson: by 2035, superintelligence brings 100x incomes and post-scarcity housing — davidpattersonx · 2026-10-27
- Resy's scorched-earth war on booking agents sets up a clash with big platforms — matt_slotnick · 2026-10-27
- Michael Levin on AI Emails: Read the Papers First, Then Write Me — drmichaellevin · 2026-10-27
- Resy goes scorched earth on agent-based bookings; big agent platforms may force the issue — kevinakwok · 2026-10-27