vLLM ships Day-0 support for Tencent Hy4 Flash with 8x H200 deployment recipe
vllm_project · x · 2026-11-18
vLLM announces Day-0 support for Tencent's Hy4 Flash: a 220B-total-parameter (incl. 64B Engram embeddings) MoE with 12B active per token, 1M-token context and multimodal understanding, aimed at productivity and agent workflows.
Architecture: 42 decoder layers with hybrid sliding-window + compressed sparse attention, Latent MoE, Engram embeddings, and a five-layer DFly speculative decoding head (on by default). Deployment notes:
- Dedicated vllm/vllm-openai:hy4-flash image; requires Hopper/Blackwell data-center GPUs (H100/H200/B200/GB300); AMD Instinct not supported yet
- Default recipe is 8x H200 tensor parallel + BF16, with a full vllm serve command including speculative decoding config
- Engram tables can be offloaded to host RAM
More from Infra
- 曝 Nvidia 2022 年牵线阿里与芯片商 Alice Huang,绕道美国出口管制 — teortaxesTex · 2026-11-18
- Microsoft launches Azure AI Gateway Labs: 27 hands-on labs for governing models and agents — adnan_hashmi · 2026-11-18
- SpaceX CFO says power is the real compute bottleneck, eyes top AI infra spot — XFreeze · 2026-11-18
- AI2 releases MoE training tech report: rebuilt Olmo-core scales to 1.2T params on 512 B300 GPUs — AkariAsai · 2026-11-18
- SGLang hits v1.0 as NVIDIA, AMD, Intel, Google and Meta form its consortium — nikola_mr64990 · 2026-11-18
- Nvidia's CFO walks back defense of 'circular financing' investments — rohanpaul_ai · 2026-11-18