NVIDIA shares guide to run GLM-5.3 NVFP4 locally on two DGX Stations with vLLM
vllm_project · x · 2026-11-17
The vLLM project shares NVIDIA's video walkthrough and deployment guide for running GLM-5.3 NVFP4 locally on two DGX Stations with vLLM. All model weights stay in GPU memory with tensor parallelism across two GB300 GPUs, served via a single OpenAI-compatible API, with MTP speculative decoding, FP8 KV cache, and prefix caching enabled.
More from Infra
- Michael Burry's Oracle short pays off: stock down 17.4% since August — SumitGup · 2026-11-17
- Megakernels dissected: launch overhead, rare compute overlap, and lots of barrier waiting — simran_s_arora · 2026-11-17
- Torch Compile underrated: CUDA Graphs + PDL hits 98% of megakernel speed on Llama 1B — simran_s_arora · 2026-11-17
- DeepSeek v4.1 Flash hits ~2,500 tok/s on Vera Rubin in Baseten low-latency tests — baseten · 2026-11-17
- ARK Newsletter Flags Jensen Huang's Resignation as Major Test for Nvidia — downingARK · 2026-11-17
- Tracing Nvidia's circular financing: $170B in sales, only ~$35B unsettled — tengyanAI · 2026-11-17