NVIDIA to Demo NVFP4 Pretraining Recipes Approaching BF16 Quality
PyTorch · x · 2026-09-01
NVIDIA researchers will present the latest NVFP4 pretraining recipes at PyTorch Conference North America 2026, aiming to accelerate large-scale LLM training while closing the quality gap to BF16. The October 21 session will cover recipe design, kernel choices, and the API surface required for native PyTorch integration. Dense linear NVFP4 training is currently available via TorchAO, with the recipes being upstreamed into the PyTorch ecosystem through TorchAO and TorchTitan.
More from Infra
- RL train-sample systems are frontier AI's hardest infra problem — and a trade secret — brianryhuang · 2026-10-16
- Local tuning of GLM-5.3-Flash on 2×GB10 lifts decode from 49 to 58 tok/s — EAccelerate_42 · 2026-10-16
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16
- Optuna creator Akiba: when 2 hyperparameters cost $20M, what's left for BO/HPO — Stefania_druga · 2026-10-16
- Amazon Proposes Per-Request Validated Evidence Caching for Production RAG — _reachsumit · 2026-10-16
- LinkedIn's TransX report: sequence transduction for recsys, +6.02% CTR in A/B tests — _reachsumit · 2026-10-16