Tencent squeezes 770B-parameter Hunyuan Hy4 Preview into 214 GiB at ~2.38 bits per weight
coolmum18 · reddit · 2026-10-16
yghstill, a member of Tencent Hunyuan's quantization team, detailed how Hy4 Preview's 1.5TB of weights (770B parameters) was compressed to 214 GiB post-training, with parameter count unchanged.
Key points
- Sherry algorithm + STQ10 format: groups of 4 weights from {-d, 0, +d} with exactly one zero encode 32 patterns in 5 bits (1.25 bits/weight; 1.3125 with FP16 shared scales; 2.38 bits/weight mixed precision).
- Sensitivity-based precision: across 77 MoE layers × 256 experts, 48 sensitive layers use IQ2XXS while 29 insensitive ones use STQ10, beating uniform IQ1M at the same bit budget.
- Full-Hessian scoring: H = XXᵀ captures off-diagonal correlations; ranking correlates only -0.115 (Spearman) with diagonal-only imatrix; layers chosen greedily by error reduction per byte.
- Joint scale/zero fitting: alternating weighted least squares and imatrix-weighted zero selection cuts weighted reconstruction error 90% in three rounds on 1,200 rows of real expert weights.
- Runtime: STQ10 CUDA kernels in patched llama.cpp run about as fast as IQ1M; MRCR retrieval nearly unchanged, small math drop; beats UD-IQ1M across evals by 5+ points on MRCR.
More from Infra
- Intel's neural-compressor: SOTA low-bit LLM quantization with FP4/NVFP4 support — HaihaoShen · 2026-10-16
- TokenSpeed MLA 2 hits NVIDIA Vera Rubin: up to 3.44x faster decode vs B200 — zhyncs42 · 2026-10-16
- US vs Chinese hyperscalers: Americans fund and schmooze, Chinese demand your BoM and cap you at 7% margin — jwt0625 · 2026-10-16
- How LiteLLM routes one OpenAI-shaped API across every model provider — Mahmoud_Zalt · 2026-10-16
- OpenAI-Broadcom custom AI chip Jalapeño named a TIME Best Invention of 2026 — itsclivetime · 2026-10-16
- Developer live-runs continual pretraining of a Luganda LLM on Colab, 41% done with loss dropping fast — weskambale · 2026-10-16