Pretraining Potential: Coding Agents and the Compute Bottleneck
zeeshanp_ · x · 2026-08-21
Zeeshan discusses the current landscape and challenges in AI pretraining.
Key Points:
- Untapped Potential: There are still many "low-hanging fruits" left in pretraining optimization.
- Agent Leverage: Coding agents have made it 100x easier to explore these directions at scale.
- Physical Constraint: The only limiting factor now is the speed of physically standing up compute. With GPUs and power in high demand, the bottleneck is infrastructure deployment speed.
More from Infra
- CUHK Team Open Sources Libra: 3x Throughput for Agentic Training — jiqizhixin · 2026-08-21
- Open-source x402-cleanweb-agent saves 80% tokens by cleaning web content — EstablishmentTough18 · 2026-08-21
- The Math: Claiming 100T Tokens/Day Would Need ~580K GPUs — teortaxesTex · 2026-08-21
- Moore's Law Fading: Non-Silicon Computing and Novel Architectures to See Capital Influx — MikePFrank · 2026-08-21
- "Why do we need more datacenters? Just write faster kernels" — basedjensen · 2026-08-21
- Cornell Nested Architecture Cuts Training Compute by 36% — burkov · 2026-08-21