GLM-5.2 hits 12.2 tok/s on 16 AMD MI50s with llama.cpp RPC
Legal-Ad-3901 · reddit · 2026-07-23
- A user reports GLM-5.2 UD-Q4KXL GGUF running on a 16× AMD MI50 32GB setup via llama.cpp RPC.
- On a real 10.7k-token document, it reached 12.2 tok/s output and 30.9 tok/s input; with two parallel requests, aggregate throughput was 14.5 tok/s.
- At 10.7k context, output stayed at 10.2 tok/s with coherent long-form generation.
- The cluster uses 2× 8-GPU nodes, 512 GB total VRAM, 100W cap per card, and 10 GbE DAC interconnect.
- The author says earlier attempts with a custom vLLM-gfx906 build hit decent speed but degraded past 10k context, and they hope someone can get the Moby Dick repo working better with it.
More from Infra
- Nearly half of Claude revenue now flows through cloud marketplace commitments, up from 11% — BenBajarin · 2026-10-26
- Anthropic ARR goes vertical after price cuts: tokens up 4x, $100B+ ARR back in play — GavinSBaker · 2026-10-26
- Alex Banks argues power caps GPU supply, real risk is demand cooling first — thealexbanks · 2026-10-26
- Analyst: Ant has the most fungible AI workload of any model provider today — BenBajarin · 2026-10-26
- Anthropic S-1: training infra edge cut per-task cost 40% and doubled revenue growth — BenBajarin · 2026-10-26
- Rewriting scaling law study: optimal synthetic data share rises from 10% to 32% with compute — heghbalz · 2026-10-26