Tencent's Hy4 Cites Zhihu Research: Identity Matrix Beats DeepSeek's mHC
tokenbender · x · 2026-08-29
DeepSeek's mHC paper proposes constraining residual connections with a doubly stochastic matrix. Tencent's new Hy4 preview cites a Zhihu study showing that simply setting Hres to an Identity matrix outperforms complex mHC variants, based on experiments training Qwen3 1.7B and 8B models from scratch.
Author tokenbender notes that his own January implementation of mHC included a tunable alpha parameter for Hres (0 to 1). While he only validated it at the 250M scale due to resource limits, Tencent's release 8 months later confirms the effectiveness of the alpha=0 (Identity) case at scale.
More from Research
- Priming Models: Teacher-Free One-Step Generation Hits FID 1.19 With 12x Less Compute — du_yilun · 2026-10-15
- vLLM Achieves Bitwise-Consistent RL Training, Fixing KL Blowups and Score Collapse — vllm_project · 2026-10-15
- David Deutsch: Proofs Are No Longer a Useful Proxy for Mathematical Progress — francoisfleuret · 2026-10-15
- STEMBench launches: 6 STEM domains, zero full-target successes across all runs — mariofilhoml · 2026-10-15
- OpenAI gives $5M to Simons Foundation's $15M push to help mathematicians navigate AI-era results — OpenAI · 2026-10-15
- Using AI to reverse engineer hardware: a path to faster open-source drivers — ErenayDev · 2026-10-15