NVFP4-to-MXFP4 online conversion hits 5,809 tok/s prefill on AMD R9700
whodoneit1 · reddit · 2026-09-16
A developer added online NVFP4-to-MXFP4 conversion to their vLLM MXFP4 build on Radiance images, fully exploiting the MXFP4 fast path on AMD R9700 GPUs for big speedups on any NVFP4 quantized model. Running Unsloth Dynamic v3 Qwen3.8-27B NVFP4, they measured 5,809 tok/s prefill and 276 tok/s decode. Code is open-sourced on Codeberg and GitHub, with results from their BetterBench tool.
Related event: Single AMD R9700 Hits 153 tok/s on Qwen3.8 27B with NVFP4(2 posts)→
More from Infra
- To dethrone Postgres, new databases must prioritize ecosystem integrations from day one — mgill25 · 2026-11-02
- ESP32-powered 4-inch touchscreen builds a TUI dashboard for monitoring a DGX inference cluster — CATLLM · 2026-11-02
- OpenRouter crosses 10 quadrillion annualized token run rate, doubling every 10.5 weeks — deedydas · 2026-11-02
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- Disable MTP for 20-30% faster local LLM inference on limited RAM — InternalMode8159 · 2026-11-02
- Kioxia up 216% in a year but down 68% from its June 2026 peak — zephyr_z9 · 2026-11-02