Tencent Hunyuan AngelSlim: Compressing Hy4 Model to 214GB with Heterogeneous Inference
腾讯混元 · wechat · 2026-09-01
Tencent's Hunyuan team released the AngelSlim quantization scheme, successfully compressing the Hy4-preview model (originally 1.5TB) down to 214GB, significantly lowering hardware barriers. Key technologies include:
- Sherry Sparse Ternary Quantization: Aggressively compresses MoE routing experts to an average of 1.25 bits per weight.
- MIX-STQ10 Mixed Precision Strategy: Dynamically allocates bits per layer based on sensitivity, balancing compression rate and accuracy.
Evaluations show minimal capability loss in long-context, math, and coding tasks, with some retrieval tasks even outperforming the original. Furthermore, the team validated heterogeneous device collaborative inference with prima.cpp, distributing the 214GB model across a laptop (RTX 4090) and a server (4x A4000). The setup achieved 1.02 token/s, proving that flagship models can run without expensive homogeneous clusters.
Relevant GGUF libraries and code have been open-sourced.
Related event: Tencent Hunyuan's AngelSlim Shrinks Hy4-preview from 1.5TB to 214GB(5 posts)→
More from Infra
- Auction-based LLM routing paper finds fixed token pricing wastes 10%-71% in costs — _onionesque · 2026-10-18
- Running Ling 3.1 Flash (560B MoE, 25B Active) Across 4 DGX Sparks via llama.cpp RPC — BerryBlush_Rogers691 · 2026-10-18
- SGLang Summit sponsors jump from 11 to 36 in one week — ying11231 · 2026-10-18
- Oracle CDS hits record 300bps, called first AI trade domino after Barclays sell downgrade — arthurcolle · 2026-10-18
- Agent Coding Workflow Would Cost ~$40k/Month on API Pricing, Dev Reveals — ChrisRackauckas · 2026-10-18
- PS5LM: llama.cpp port turns a jailbroken PS5 into a home LLM server (500+ GitHub stars) — wateriscoding · 2026-10-18