Local Deployment of GLM-5.3-Flash: 206 tok/s and 1M Context on DGX Station
funding__secured · reddit · 2026-08-27
The author successfully deployed and tested the GLM-5.3-Flash model on a DGX Station GB300.
Performance:
- Speed: 206 tokens/s in single-stream mode.
- Context: Supports 1 million (1M) context length.
- Precision: Using NVFP4 quantization, optimized for HBM3e memory.
Deployment Configuration (Docker):
A complete docker run command is provided with key parameter optimizations:
- VLLMKVCACHELAYOUT=HND
- --compilation-config '{"mode":3}'
- --enable-prefix-caching
- --attention-config '{"usefp4indexercache": true}'
- --speculative-config '{"method":"mtp","numspeculativetokens":3}'
Gotcha:
The current image has a bug preventing automatic model downloads. Weights must be pre-downloaded locally and pointed to via the --model flag.
Related event: GLM-5.3 Flash tests show near-GPT-5.6 performance at rock-bottom cost(6 posts)→
More from Infra
- QNX partners with Hailo for edge Physical AI: 14x performance consistency — pdamodaran · 2026-08-27
- US Holds 15-20x Compute Advantage, But May Not Matter for Some Threats — ohlennart · 2026-08-27
- Hark partners with NVIDIA for gigawatt-scale compute on Vera Rubin platforms — adcock_brett · 2026-08-27
- Minimax H3 Local Benchmark: 5-Second Clip Takes 4 Minutes on AMD 7900XT — thevictor390 · 2026-08-27
- US right-leaning groups push bills to curb China's AI chip access — ohlennart · 2026-08-27
- GLM-5.3-Flash Runs at 160t/s with 1.4M Context on RTX6000 Pro — AutonomousHangOver · 2026-08-27