Hy3 Launches 1bit/4bit Quantized Versions
腾讯混元 · wechat · 2026-07-14
Tencent's Hunyuan team has performed quantization and deployment optimizations for the open-source Hy3 model, focusing on making the 295B flagship model runnable on more limited hardware.
Key highlights include:
- Releasing 1bit and 4bit GGUF quantized versions for use with the llama.cpp ecosystem.
- The 1bit version compresses the 598GB BF16 weights down to 85.5GiB, which the author says can be deployed on a single 96GB inference card.
- The 4bit version is approximately 169.9GiB, suitable for dual inference cards.
- Additionally, a GPTQ Int4 version is provided, which can be directly deployed using vLLM.
- The team also added support for Hy3's MTP speculative decoding to llama.cpp; when enabled, the acceptance rate is around 60%, delivering a speedup of roughly 50% for 1bit and nearly 60% for 4bit.
- Official Hugging Face links for the base model, quantized models, and patches are also provided.
Related event: Tencent Hunyuan ships 1-bit/4-bit quantized Hy3 for single-GPU deployment(5 posts)→
More from Infra
- Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100 — sguada · 2026-10-27
- DeepSpeed shows AutoEP/AutoTP/AutoSP parallelism gains, hosts office hours — PyTorch · 2026-10-27
- Running a local 9B as a coding agent: brief the big model to plan, let the small model execute — Electrical_Ninja3805 · 2026-10-27
- Cloudflare Durable Objects validated as top architecture for AI agent WebSocket servers — tristanbob · 2026-10-27
- Gary Marcus: With Q3 data added, Anthropic's compute-efficiency growth shows deceleration — GaryMarcus · 2026-10-27
- SpaceX locks in $40B GPU financing deal with Apollo, the largest ever committed — TechCrunch AI · 2026-10-27