Researcher Challenges GPT-4-Era 'Winner-Takes-All' Scaling Thesis as Pretraining Shrinks to Minority Compute Share
jd_pressman · x · 2026-10-26
AI researcher jdpressman says he's broadly bullish on AI but struggles to buy the GPT-4-era 'winner takes all' parameter-scaling investment thesis. His core argument: every scaling axis introduced after raw parameters has been sparser than the last, and traditional pretraining now accounts for a minority of training compute. That weakens the case for a natural monopoly in AI of the kind railroads once had — scaling advantages are diffusing across multiple axes rather than concentrating.
More from AGI Musings
- mflux: talk about the cool stuff you're building, not how it's built — mflux · 2026-10-26
- Economists replicate papers with Claude and Codex: your past errors will be found — paulnovosad · 2026-10-26
- Engineer asks: are we building better products or better funding pitches? — TajyMany · 2026-10-26
- voooooogel: AI will purge 50 years of tech debt and finish 90s cyberidealism — voooooogel · 2026-10-26
- NYT Opinion: The AI Boom Looks a Lot Like the Housing Bubble — JoshuaZ1 · 2026-10-26
- Blogger claims tipping point crossed: next 6 months to beat past 60 years — SydSteyerhart · 2026-10-26