Local Deployment of GLM-5.3-Flash: 206 tok/s and 1M Context on DGX Station
funding__secured · reddit · 2026-08-27
The author successfully deployed and tested the GLM-5.3-Flash model on a DGX Station GB300.
Performance:
- Speed: 206 tokens/s in single-stream mode.
- Context: Supports 1 million (1M) context length.
- Precision: Using NVFP4 quantization, optimized for HBM3e memory.
Deployment Configuration (Docker):
A complete docker run command is provided with key parameter optimizations:
- VLLMKVCACHELAYOUT=HND
- --compilation-config '{"mode":3}'
- --enable-prefix-caching
- --attention-config '{"usefp4indexercache": true}'
- --speculative-config '{"method":"mtp","numspeculativetokens":3}'
Gotcha:
The current image has a bug preventing automatic model downloads. Weights must be pre-downloaded locally and pointed to via the --model flag.
Related event: GLM-5.3 Flash Benchmarks Hit GPT-5.6 Territory at Fractional Cost(7 posts)→
More from Infra
- ASML CFO says US Match Act restricting DUV sales to China would backfire — pstAsiatech · 2026-10-15
- ASML sold out through 2028 but stock dips 2%; TL20 index up 57% YTD — TiernanRayTech · 2026-10-15
- Bloomberg breakdown: a 1GW Texas AI data center may need nearly $3B for overruns alone — rohanpaul_ai · 2026-10-15
- 7 System Design Algorithms to Know: Merkle Trees to CAP/PACELC — goyalshaliniuk · 2026-10-15
- MiniCPM-V 4.7 runs a 35B MoE multimodal model on phones with just 3GB memory — nevrekaraishwa2 · 2026-10-15
- Sipeed MaixStation product page goes live for edge AI dev — xlltt · 2026-10-15