FineWeb-30M speedrun record falls again: 3.50 val loss in 14.2s on one 8xH100 node
RobertTLange · x · 2026-11-05
RobertTLange opened PR #56 in qlabs-eng/noise, cutting the FineWeb-30M Track 2 speedrun record from 15.9s to 14.24s (seed 0, single 8xH100 node) with a final validation loss of 3.4965 — ×1.12 faster than the prior record (#55) and ×1.64 faster than the earlier 22.9s run.
Key stepping stones:
- 22.7s: #52 rerun — identity-path phase, then Cut with single roots at blocks 4/6, fused chain kernels, one CUDA graph per estimate;
- 19.9s: 16-row updates — each GPU natively estimates two rows per block from 10% of Cut onward;
- 14.2s (final): root 4 on every 2nd update early, tensor-core and Gluon credit kernels, per-head q/k polar, and a scheduled update tail. Identity phase 1.18s, end merge 0.15s, run with the default uv run torchrun --standalone --nprocpernode=8 train/speedrun/run.py.
Related event: FineWeb-30M Speedrun Record Cut to 14.2 Seconds on Single 8xH100 Node(2 posts)→
More from Infra
- Meta president says midterms will make building new US data centers 'very challenging' — ShakeelHashim · 2026-11-05
- Outsiders are signing $50M B300 cluster deals with zero space or power lined up — andersonbcdefg · 2026-11-05
- Local voice agent stack on RTX 3060 gives veteran user his 'magic' AI moment — 1or4s · 2026-11-05
- NASA makes history with first-ever quantum link between aircraft and ground — Polymarket · 2026-11-05
- Offline survival AI packs 1.5M articles into 8.8GB, works with no internet — mhdfaran · 2026-11-05
- LZ4, the extremely fast C compression algorithm, still pulling stars — lz4 · 2026-11-05