Training on 1,000 GB300s costs $125-150M over 3 years and still trails frontier by 1-2 orders of magnitude
FinanceYF5 · x · 2026-10-28
FinanceYF5 breaks down the compute math for a "Neolab"-style startup:
- The bill: 1,000 GB300s ≈ 14 NVL72 racks, costing roughly $125-150M over three years, with 15-30% prepaid and 2-2.5MW of power.
- The output: 10²⁵ FLOPs per quarter — enough to train a GPT-4-class model, but still 1-2 orders of magnitude behind frontier pretraining.
- The alternative: post-training on strong open-source models offers a better shot at catching up, but carries two risks — RL environments alone require millions more in spend, and the team gets locked to a base model that a newer release could quickly obsolete.
Related event: The 'Neolab' Compute Math: $150M Over Three Years and Still Behind(3 posts)→
More from Infra
- CXMT to start small-scale HBM3E production within two months, yield at 20% — zephyr_z9 · 2026-10-28
- Video Transcription Costs Under 1 Yuan per Hour, CPU Free but Error-Prone — huangyun_122 · 2026-10-28
- A forgotten supercomputer design: bare dies, gold post interconnects, 3D stacking, immersion cooling — jwt0625 · 2026-10-28
- Reddit predicts a GPU price crash next year as the compute bubble deflates — sayamss · 2026-10-28
- Karpathy recommends a YouTube series building a CPU from a single transistor — blaizedsouza · 2026-10-28
- Idle GPUs burn cash: Neolabs lose money below ~60% spot utilization — FinanceYF5 · 2026-10-28