vLLM Announces Day-0 Support for Qwen3.8 2.4T with Ready-to-use 4-bit Checkpoints
vllm_project · x · 2026-08-13
The vLLM project has announced Day-0 support for Alibaba's massive new open-weight model, Qwen3.8-2.4T-A95B (2.4T total params, 95B active, 512 experts).
Verified on both NVIDIA and AMD hardware, vLLM has collaborated with Inferact to provide ready-made 4-bit quantized checkpoints:
- NVFP4 (1.32 TiB): Runs on a single 8xNVIDIA B300 node.
- MXFP4 (1.45 TiB): Runs on a single 8xAMD MI355X node.
Users can deploy directly using vllm serve without needing any manual conversion or calibration.
Related event: vLLM Announces Day-0 Support for Qwen3.8 2.4T Model(2 posts)→
More from Infra
- AI-Driven Timing-Aware Power Optimization Uses Learned Ranking With Baseline-Preserving Repair — Amit Sharma · 2026-10-16
- Nscale AI datacenter plagued by coolant leaks, transformer fire, 10 safety violations — saranormous · 2026-10-16
- WSL 3.0 ships with native containers and Linux 6.18 kernel, syscall overhead down ~16% — solyarisoftware · 2026-10-16
- Are 22 GB modded RTX 2080 Ti cards a viable budget local LLM option? — Dave86ch · 2026-10-16
- Intel's neural-compressor: SOTA low-bit LLM quantization with FP4/NVFP4 support — HaihaoShen · 2026-10-16
- Tencent squeezes 770B-parameter Hunyuan Hy4 Preview into 214 GiB at ~2.38 bits per weight — coolmum18 · 2026-10-16