Testing Local Inference on GLM-5.2-FP8
AlpinDale · x · 2026-07-09
The author reported running GLM-5.2-FP8 on a cluster of 4 machines equipped with 4090 48GB GPUs, achieving a decoding speed of approximately 28 tok/s.
They also noted that the current nodes are only connected via a 10Gb Ethernet network, meaning there is still room for speed improvements. Additionally, they are working on getting DSpark up and running and plan to upstream the relevant kernels to vLLM.
More from Infra
- Surprising Ubuntu Setup: NVIDIA 5090 PC Becomes the Easiest AI Rig — _xjdr · 2026-07-27
- TSMC reportedly plans 5%–10% price hikes in 2027 to cover rising costs — Beth_Kindig · 2026-07-27
- Anthropic says Claude Code can drop 80% of its system prompt with no coding loss — krishnan · 2026-07-27
- Dhruv Bhatia joins fal to work on video and world models — gorkem · 2026-07-27
- Moonshot’s Kimi K3 lands on Together with reserved throughput and 65% lower cost — togethercompute · 2026-07-27
- NVIDIA says Vera CPU is speeding up next-gen CPU and GPU design cycles — nordicinst · 2026-07-27