NVFP4-to-MXFP4 online conversion hits 5,809 tok/s prefill on AMD R9700
whodoneit1 · reddit · 2026-09-16
A developer added online NVFP4-to-MXFP4 conversion to their vLLM MXFP4 build on Radiance images, fully exploiting the MXFP4 fast path on AMD R9700 GPUs for big speedups on any NVFP4 quantized model. Running Unsloth Dynamic v3 Qwen3.8-27B NVFP4, they measured 5,809 tok/s prefill and 276 tok/s decode. Code is open-sourced on Codeberg and GitHub, with results from their BetterBench tool.
More from Infra
- Perplexity details CobbleDB: 2 engineers plus hundreds of AI agents built core infra in 2 months — inductionheads · 2026-09-16
- NVIDIA exec: global AI penetration only ~2%, Vera Rubin ramp still 'insanely hard' — BenBajarin · 2026-09-16
- Inference providers offload idle GPU risk by forcing throughput reservations — AAAzzam · 2026-09-16
- First M5 Ultra Geekbench leak: 52,516 multi-core, ~35% faster than M3 Ultra — jedisct1 · 2026-09-16
- Apple execs on podcast: 2nm silicon, doubled Neural Engine, edge AI future — BenBajarin · 2026-09-16
- Sentdex's get-rich trick: buy 8 RTX Pro 6000s, resell 4 boxed, keep 4 free — Sentdex · 2026-09-16