NVIDIA shares guide to run GLM-5.3 NVFP4 locally on two DGX Stations with vLLM

vllm_project · x · 2026-11-17

The vLLM project shares NVIDIA's video walkthrough and deployment guide for running GLM-5.3 NVFP4 locally on two DGX Stations with vLLM. All model weights stay in GPU memory with tensor parallelism across two GB300 GPUs, served via a single OpenAI-compatible API, with MTP speculative decoding, FP8 KV cache, and prefix caching enabled.

Original post →

More from Infra

Infra channel →