Tencent compresses 770B Hunyuan Hy4 Preview into 214 GiB at ~2.38 bits per weight
OpeningRock8761 · reddit · 2026-10-07
A technical breakdown from yghstill of Tencent Hunyuan's quantization team explains how Hy4 Preview's 1.5TB of weights (770B parameters, unchanged) was compressed to 214 GiB.
Four weights, five bits
- The Sherry algorithm + STQ10 format encodes groups of four ternary weights {-d, 0, +d} with exactly one zero: 32 patterns in 5 bits (1.25 bits/weight), or 1.3125 with a shared FP16 scale per 256 weights; mixed-precision MIX-STQ10 averages 2.38 bits/weight.
Sensitivity-driven precision
- Across 77 MoE layers (256 experts each), expert gate/up projections use IQ2XXS on 48 sensitive layers and STQ10 on 29 insensitive ones, beating uniform IQ1M at the same bit budget.
Full-Hessian scoring
- Sensitivity uses the full Hessian (XXᵀ), capturing correlations diagonal-only imatrix misses (Spearman correlation between rankings: -0.115); layers chosen greedily by error reduction per byte.
Fitting and runtime
- Post-training quantization: alternating weighted least squares for scale and imatrix-weighted zero-position selection cuts weighted reconstruction error 90% over the original ternary encoder in three rounds.
- CUDA kernels in a patched llama.cpp build run at roughly IQ1M speed; MRCR retrieval nearly unchanged, slight math drop; MIX-STQ10 beats UD-IQ1M across evals, by 5+ points on MRCR.
More from Infra
- MIT Lincoln Lab tracks 120+ AI accelerators from GPUs to ASICs in LAICS survey — nordicinst · 2026-10-07
- K8 unveils $250M prepayment facility to finance AI compute contract deposits — Saul_Loveman · 2026-10-07
- Beff Jezos to join SF Tech Week panel on the future of inference — beffjezos · 2026-10-07
- ChipForge NPU Challenge Goes Live: Crowdsource AI Accelerator Designs Headed to Real Silicon — bittingthembits · 2026-10-07
- Open models are the answer: how Fireworks and OpenRouter cut AI's soaring costs — karlfreund · 2026-10-07
- Actual ships toks, an asm-powered tokenizer claiming 151x speedup over Hugging Face — markjeffrey · 2026-10-07