Tencent Hy4 lands in vLLM day-0: 774B MoE with 49B active, 1M context, 8-GPU FP8 recipe
vllm_project · x · 2026-11-20
The vLLM project announced day-0 support for Tencent's Hy4 with a full deployment recipe.
- Multimodal MoE: 774B total params, 49B active per token, 78 layers, 256 routed experts (top-8) + 1 shared expert, gated sparse MLA attention, up to 1M-token context
- Includes a 1.23B vision encoder and 3.3B DFly2 speculative-decoding draft; FP8 and BF16 checkpoints available
- Runs on one 8-GPU NVIDIA node (FP8 944GB, BF16 1859GB aggregate memory); Docker image vllm/vllm-openai:hy4 plus official recipe with hyv4 tool/reasoning parsers and FLASHMLASPARSE backend
- AMD ROCm unsupported; Tencent led model implementation, Inferact contributed integration and serves Hy4 in production
In Tencent's earlier blind evaluation across 579 financial tasks with 213 professionals, Hy4 scored 3.04, above DeepSeek-V4.1-Flash (2.97), Kimi K3.1 Pro and GLM-5.3 (both 2.68).
More from Infra
- Local inference users want 512GB RAM and 1M context, says current boxes fall short — AIFlow_ML · 2026-11-20
- BitDecoding-FP4: native FP4 KV cache decoding via algorithm-system co-design — kalyan_kpl · 2026-11-20
- Prefill beats decode: SGLang wins agent runs on DGX Spark despite slower tok/s — AIFlow_ML · 2026-11-20
- Nvidia supplier commitments jump from $87B to $203B as forecasts miss low — tengyanAI · 2026-11-20
- Qwen3.8-Flash-Next on a single Intel Arc B70: 2.5K prefill, 47 tok/s — WizardlyBump17 · 2026-11-20
- SQLite is ~35% faster than the filesystem for 10KB blobs — here's why the syscall overhead adds up — arpit_bhayani · 2026-11-20