H100 Memory Study Gets Errata After DeepSeek's Open-Source V3 Trace Reveals SwiGLU Quantization Fusion Details
ezyang · x · 2026-09-02
PyTorch core dev ezyang confirmed a fix and errata in the H100 memory study after Vlad Savinov flagged an issue: SwiGLU doesn't consume e4m3 directly since DeepGEMM outputs bf16 — quantization is likely fused into the SwiGLU kernel.
Vlad backed this up with DeepSeek's open-sourced V3 perfetto trace (the profile-data repo on GitHub, 1.2k stars): SwiGLU time matches topKseqlenintermediate7 / 2.8e12 ≈ 0.16ms. He infers the kernel reads two bf16 tensors and outputs three fp8 tensors — two quantized inputs plus a quantized silu(a)b feeding the next DeepGEMM call (2.8e12 is 85% of Hopper's 3.35 TB/s spec bandwidth).
More from Infra
- Tencent open-sources CubeSandbox v0.7.0, keeping thousands of agents alive through node failures — SucceededMind · 2026-09-03
- Magnitude open-source inference server swaps coding agents to free local models automatically — nickbaumann_ · 2026-09-03
- MiniMax H3 video generation runs fully local on a single RTX 5060 Ti 16GB — apoke890 · 2026-09-03
- With 98% Cache Hits in Coding, 400-600 t/s Prefill Already Hits Diminishing Returns — nomorebuttsplz · 2026-09-03
- Dell COO: inference tokens to grow 87x to 3,600 quadrillion by 2030 — Beth_Kindig · 2026-09-03
- Beating hipBLASLt on a gaming GPU: a GEMM optimization deep dive — Moist_Weird_42067 · 2026-09-03