vLLM Day-0 Support for Qwen3.8-2.4T-A95B: Runs on Single NVIDIA/AMD Nodes
vllm_project · x · 2026-08-13
Congratulations to Alibaba Qwen on the release of Qwen3.8-2.4T-A95B, one of the largest open-weight models to date, featuring 2.4T total parameters, 95B active parameters, and 512 experts.
vLLM project has announced Day-0 support for this massive model, verified on both NVIDIA and AMD hardware. Ready-made 4-bit checkpoints are provided out of the box:
- NVIDIA: Runs on a single 8xB300 node (1.32 TiB).
- AMD: Runs on a single 8xMI355X node (1.45 TiB).
Related event: vLLM Announces Day-0 Support for Qwen3.8 2.4T Model(2 posts)→
More from Infra
- AI-Driven Timing-Aware Power Optimization Uses Learned Ranking With Baseline-Preserving Repair — Amit Sharma · 2026-10-16
- Nscale AI datacenter plagued by coolant leaks, transformer fire, 10 safety violations — saranormous · 2026-10-16
- WSL 3.0 ships with native containers and Linux 6.18 kernel, syscall overhead down ~16% — solyarisoftware · 2026-10-16
- Are 22 GB modded RTX 2080 Ti cards a viable budget local LLM option? — Dave86ch · 2026-10-16
- Intel's neural-compressor: SOTA low-bit LLM quantization with FP4/NVFP4 support — HaihaoShen · 2026-10-16
- Tencent squeezes 770B-parameter Hunyuan Hy4 Preview into 214 GiB at ~2.38 bits per weight — coolmum18 · 2026-10-16