Hy3 Launches 1bit/4bit Quantized Versions
腾讯混元 · wechat · 2026-07-14
Tencent's Hunyuan team has performed quantization and deployment optimizations for the open-source Hy3 model, focusing on making the 295B flagship model runnable on more limited hardware.
Key highlights include:
- Releasing 1bit and 4bit GGUF quantized versions for use with the llama.cpp ecosystem.
- The 1bit version compresses the 598GB BF16 weights down to 85.5GiB, which the author says can be deployed on a single 96GB inference card.
- The 4bit version is approximately 169.9GiB, suitable for dual inference cards.
- Additionally, a GPTQ Int4 version is provided, which can be directly deployed using vLLM.
- The team also added support for Hy3's MTP speculative decoding to llama.cpp; when enabled, the acceptance rate is around 60%, delivering a speedup of roughly 50% for 1bit and nearly 60% for 4bit.
- Official Hugging Face links for the base model, quantized models, and patches are also provided.
Related event: Tencent Hunyuan ships 1-bit/4-bit quantized Hy3 for single-GPU deployment(5 posts)→
More from Infra
- 12 KV Cache Reduction Techniques Every AI Engineer Should Understand, Explained — blaizedsouza · 2026-09-11
- The shadow GPU capacity market is formalizing, with Meta selling excess compute to outside buyers — DavidLinthicum · 2026-09-11
- Engram's random reads don't suit SSDs; CPU-memory over NVLink could serve all 72 GPUs — bookwormengr · 2026-09-11
- 80% of the DIY LLM inference hype posters have already quit — it's brutally hard systems work — abhijithneil · 2026-09-11
- Hugging Face's Ultra Scale Playbook: a free book on training LLMs on GPU clusters — mdancho84 · 2026-09-11
- Is inference latency becoming the biggest bottleneck for production AI agents? — Euphoric_Sea632 · 2026-09-11