Tencent's Hy4 Cites Zhihu Research: Identity Matrix Beats DeepSeek's mHC
tokenbender · x · 2026-08-29
DeepSeek's mHC paper proposes constraining residual connections with a doubly stochastic matrix. Tencent's new Hy4 preview cites a Zhihu study showing that simply setting Hres to an Identity matrix outperforms complex mHC variants, based on experiments training Qwen3 1.7B and 8B models from scratch.
Author tokenbender notes that his own January implementation of mHC included a tunable alpha parameter for Hres (0 to 1). While he only validated it at the 250M scale due to resource limits, Tencent's release 8 months later confirms the effectiveness of the alpha=0 (Identity) case at scale.
More from Research
- Cohere Labs world model course session 4: DreamerV1-V4 architectures in one go — Cohere_Labs · 2026-10-16
- Reasonable post-trains Nemotron 3.5 on 3,000 Verus proofs, lifting pass@1 ~30 points to 73.3% — fhuszar · 2026-10-16
- GPT-6.1 Sol audit finds 6% of arXiv papers had false main theorems, down to 1.5% recently — nitarshan · 2026-10-16
- One tiny word changes the conversation: how LLMs pick up on 'just' — StellaLisy · 2026-10-16
- SEAR: first large-scale controlled study of self-evolving robot agents, 7 LLMs, 330 tasks — xiye_nlp · 2026-10-16
- After Claude-assisted elliptic curve records fell, EC Rank challenge opens 156 record slots — Jsevillamol · 2026-10-16