JAX speedrun PR trains to loss 4.179 in 6.76s on 8×H100 — 2.27× faster than prior best
RobertTLange · x · 2026-10-21
Robert Lange opened PR #29 for qlabs-eng/noise's speedrun track: training to val loss 4.1792 in 6.76 seconds on 8×H100 80GB, timed from first update to end of last.
- That's 2.27× faster than PR #25's 15.35s on the same recipe, and 12.3× faster than track record #7 (83.3s).
- Robustness: five more seeds average 6.74±0.18s with val 4.162±0.016, all below 4.2.
- Method: the estimator and recipe are untouched — every signal is the same forward measurement, just computed faster via moment kernels with one head per GPU; the diff against #25 is mostly train.py and one new kernel file. The branch is stacked on #25 and should merge after it.
Worth a read for anyone into JAX kernel optimization and training speedruns.
More from Infra
- Super Micro guides FY27 Q1 revenue to $18.5–19.5B on AI demand, crushing prior guidance — firstadopter · 2026-10-21
- PyTorchCon Demo Theater afternoon: IBM Spyre, Modal, Arm and 5 more sponsor demos — PyTorch · 2026-10-21
- SPCX compute hits 1.7 GW at end of Q3, 2.5 GW by Q4 looks achievable — JOBhakdi · 2026-10-21
- Dev Ships Free LLM Cost Calculator Pulling Live OpenRouter Pricing for 250+ Models — zackku · 2026-10-21
- Musk targets 13GW AI compute by 2027 with 440,000-GPU Vera Rubin cluster — scaling01 · 2026-10-21
- VectorDash Rewrites Windows Search in Rust, Adds Local Reverse Image Search and AI Chat — andy2na · 2026-10-21