TileRT and AMD hit 469 tok/s decode on GLM-5.3 with vLLM on 8x MI355X, 40% faster than GB300
vllm_project · x · 2026-09-25
The vLLM project reports that TileRT and AMD teams pushed GLM-5.3 single-user decode to 469 tok/s on 8x MI355X in SemiAnalysis's AgentX benchmark — over 40% faster than GB300 TRTLLM on FP4. The setup is disaggregated: vLLM handles prefill while TileRT handles latency-critical decode via vLLM's V1 connector interface, keeping the rest of the deployment stock vLLM.
Related event: vLLM Integrates TileRT, AMD MI355X Outpaces GB300 by 40% on GLM-5.3(2 posts)→
More from Infra
- Altman says OpenAI aims to replace Nvidia in the medium to long term — zephyr_z9 · 2026-11-11
- Chinese University Bought Nvidia H200 Chips Despite US Blacklist — pstAsiatech · 2026-11-11
- Baidu's ERNIE Evolve tops public bests on NanoGPT Speedrun, brings RSI to real inference optimization — xiaohu · 2026-11-11
- Baidu says RSI auto-optimized ERNIE inference, 6.24x decode speedup on Kunxin P800 — xiaohu · 2026-11-11
- Nvidia's supply commitments jump from $119B to $279B in one quarter, locking up 2027-28 memory — tengyanAI · 2026-11-11
- Train 27B models on a 24GB GPU: ternary optimizer cuts memory to 1 byte/weight — oguzeldereli · 2026-11-11