R9V: Custom RDNA4 Kernels Boost Qwen3.8 Throughput by 30x
Public_Umpire_1099 · reddit · 2026-08-31
The author released R9V, a set of highly custom inference kernels tuned specifically for AMD's RDNA4 (R9700s) architecture, integrated into vLLM-Radiance to significantly boost performance for Qwen3.8-Flash-Next and Muse Glimmer 30B.
Key Benchmarks (Dual R9700, 128GB RAM):
- Qwen3.8 Flash Next: PP8192 hit 1510 tok/s (30x increase), TG256 hit 78 tok/s (3x increase).
- Muse Glimmer 30B: Outperformed llama.cpp (ROCm/Vulkan) across the board.
Technical Details:
- Deep utilization of RDNA4's Wave32 design, DPP operations, and integer dot instructions.
- Optimized MTP (Speculative Decoding) by reusing token routes for hot experts, achieving 27% bandwidth optimization.
- Optimized Prefill by grouping prompt tokens by expert (group size 16).
- For dense models, implemented effective reuse of multivector weights and HyperConnection fusion.
More from Infra
- RL train-sample systems are frontier AI's hardest infra problem — and a trade secret — brianryhuang · 2026-10-16
- Local tuning of GLM-5.3-Flash on 2×GB10 lifts decode from 49 to 58 tok/s — EAccelerate_42 · 2026-10-16
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16
- Optuna creator Akiba: when 2 hyperparameters cost $20M, what's left for BO/HPO — Stefania_druga · 2026-10-16
- Amazon Proposes Per-Request Validated Evidence Caching for Production RAG — _reachsumit · 2026-10-16
- LinkedIn's TransX report: sequence transduction for recsys, +6.02% CTR in A/B tests — _reachsumit · 2026-10-16