Full SGLang config for GLM-5.3-Flash NVFP4 on 4x RTX 6000 Max-Q
TheZachMueller · x · 2026-10-02
Zach Mueller shared his complete SGLang config for running GLM-5.3-Flash NVFP4 on 4x RTX 6000 Max-Q: tensor parallel 4, MoE backend flashinfercutlass, modeloptfp4 quantization, fp8e4m3 KV cache.
Speculative decoding uses EAGLE (4 steps, top-k 1, 5 draft tokens), max 8 concurrent requests with CUDA graphs up to batch 8, and NCCLP2PDISABLE=1. The config pairs with his concurrency sweep benchmarks and is directly reproducible.
Related event: GLM-5.3-Flash NVFP4 Benchmark: Gains Flat Beyond 8 Concurrency(2 posts)→
More from Infra
- MachGen pushes MiniMax H3 past its 15s cap with 30-second continuous video — MiniMax_AI · 2026-10-02
- Redditor builds fully local LLM-powered radio site on two DGX Sparks and a 5090 — jwhh91 · 2026-10-02
- VC quip: many neoclouds are closer to 95% than five nines of reliability — saranormous · 2026-10-02
- Report: lenders demand up to 25% collateral from Nvidia as GPU-backed loans wobble — GaryMarcus · 2026-10-02
- Microsoft Backs Snowflake-Led Effort to Standardize Business Metrics for AI — xiaosun86 · 2026-10-02
- GLM-5.3-Flash NVFP4 benchmarks show no per-user speedup beyond 8 concurrent requests — TheZachMueller · 2026-10-02