Compute and Data are the True R&D Bottlenecks
willccbb · x · 2026-07-16
This discussion questions the vagueness of a "1000x R&D speedup," noting that improvements in architecture, optimizers, infrastructure, kernels, chips, and algorithms all have their own ceilings.
The author argues that the variables seemingly without upper limits are compute and data, yet they are the most heavily constrained by physical reality. Replies also suggest that the lifespan of architectures like DSV3 should prompt a reality check for those basing world models on the infinite acceleration of "recursive self-improvement of algorithms."
Related event: Researchers cool expectations around 1000x AI R&D speedups(8 posts)→
More from AGI Musings
- Bindu Reddy says the industry still lacks a way to train 20T models and scale post-training RL — bindureddy · 2026-07-22
- Advanced AI Models Are Becoming Impossible to Plug and Play — emollick · 2026-07-22
- The Thimble and the Waterfall: AI's Data Bottleneck and Feedback Loops — dyamins · 2026-07-22
- Researcher Admits Kurzweil Was Right About AI Scaling Laws All Along — davidmanheim · 2026-07-22
- AI is still not at a maturity plateau, the author argues — generativist · 2026-07-22
- Essay argues LLMs are externalized metacognition, not standalone intelligence — lnsip9reg · 2026-07-22