Tencent's Hy4 Cites Zhihu Research: Identity Matrix Beats DeepSeek's mHC
tokenbender · x · 2026-08-29
DeepSeek's mHC paper proposes constraining residual connections with a doubly stochastic matrix. Tencent's new Hy4 preview cites a Zhihu study showing that simply setting Hres to an Identity matrix outperforms complex mHC variants, based on experiments training Qwen3 1.7B and 8B models from scratch.
Author tokenbender notes that his own January implementation of mHC included a tunable alpha parameter for Hres (0 to 1). While he only validated it at the 250M scale due to resource limits, Tencent's release 8 months later confirms the effectiveness of the alpha=0 (Identity) case at scale.
More from Research
- First backprop-trained CNN from 1988 approved by FDA for mammography — SchmidhuberAI · 2026-08-29
- pMLX Optimizes Qwen and GLM with Expert Paging, Runs Large MoEs on 37GB RAM — EyalToledano · 2026-08-29
- PILOT: New Framework Lets Long-Running Agents Self-Improve — badumtsssst · 2026-08-29
- Dactyl automates iOS engine dev using fleet of agents & pixel diffing — alexisgallagher · 2026-08-29
- FACT: Robots Learn from Failures Using Failure-Aware Causal Training — ZeYanjie · 2026-08-29
- Neural Network Basics: Train CNNs with PyTorch & TensorFlow — Al_Grigor · 2026-08-29