Software magic: 4x V100s match RTX 5090 running Qwen 3.8 NVFP4
Simple_Library_2700 · reddit · 2026-08-19
A developer achieved performance parity between four Tesla V100s (2017) and an RTX 5090 when running Qwen 3.8 NVFP4 by writing a custom QPN kernel. Although V100s lack native FP4/FP8 support, the kernel translates fragments on the fly to FP16 for Volta's Tensor Cores. In single-request decoding, the V100 setup reached 219.1 tok/s, slightly edging out the 5090's 214.7 tok/s by verifying more tokens per round (5.89 vs 4.27).
More from Infra
- Large data centers employ only ~50-150 permanent on-site staff, Microsoft says — brezina · 2026-10-16
- Prediction: Opus 5.5-level models on laptops by 2027, backed by Qwen 27B data — cephaloform · 2026-10-16
- Oracle down 12% in a week, nearly 50% off June high as OpenAI exposure spooks market — alvelda · 2026-10-16
- US data center moratoriums top 400 — an AI bull explains why they're actually good for AI — kevinsxu · 2026-10-16
- Microsoft Research unveils Magnetar, the fastest zkVM at 10-39x speedups and post-quantum security — jedisct1 · 2026-10-16
- Bonsai 2 2B hits 120 tok/s on iPhone 18 Pro, outpaces Qwen 3.5 2B at half the memory — cephaloform · 2026-10-16