AMD MI350X Runs Qwen3.6-35B: Open Source Kernel Achieves 78.5k tok/s on 8 GPUs
SmilingGen · reddit · 2026-08-26
NetraRuntime optimized Qwen3.6-35B-A3B for AMD MI350X GPUs at the kernel level to close the gap with NVIDIA's CUDA stack. Benchmarks show single-card speeds of 11,161 output tok/s and an 8-card peak of 81,331 tok/s (mean: 78,498 tok/s), achieving 2.16x the throughput of vLLM. The kernels are open-sourced, and the team details their findings, noting that once kernels are fast enough, bottlenecks shift to scheduling, graph coverage, and HTTP serialization.
More from Infra
- ASML CFO says US Match Act restricting DUV sales to China would backfire — pstAsiatech · 2026-10-15
- Bloomberg breakdown: a 1GW Texas AI data center may need nearly $3B for overruns alone — rohanpaul_ai · 2026-10-15
- 7 System Design Algorithms to Know: Merkle Trees to CAP/PACELC — goyalshaliniuk · 2026-10-15
- MiniCPM-V 4.7 runs a 35B MoE multimodal model on phones with just 3GB memory — nevrekaraishwa2 · 2026-10-15
- Sipeed MaixStation product page goes live for edge AI dev — xlltt · 2026-10-15
- Nvidia Is Now Worth More Than the Entire Indian Stock Market — theteknosaur · 2026-10-15