DeepSeek-V4 and ERNIE 5.5 Flash Said to Use 4 Residual Streams with Sinkhorn Mixing
RobertTLange · x · 2026-11-09
Robert Lange writes that DeepSeek-V4 models and Baidu's ERNIE 5.5 Flash all operate with 4 residual streams mixed via 20 Sinkhorn iterations, and walks through this architectural evolution:
- Residual connections: x + F(x) lets gradients propagate through deep layers, but capacity is tied to layer width
- Hyper-Connections (HC): copies input x into 4 streams and learns 3 small maps — Hpre reads streams into the layer, Hpost writes output back, Hres (4×4) mixes streams; F and its FLOPs stay the same
- Manifold-constrained HC (mHC): projects Hres onto doubly stochastic matrices via Sinkhorn-Knopp iterations, making each stream a convex mix of streams
More from Models
- Unverified claim: Opus 5.5 has only 10B active params; dev hands projects to Qwen-4-Flash — teortaxesTex · 2026-11-09
- Benchmarks are like perfume: why public leaderboards demand skepticism and private evals — intellectronica · 2026-11-09
- Running 8-25 Opus 5.5 threads all day used under 20% of a $200 Claude sub — cyrus_zei · 2026-11-09
- Grok Bot quietly throttled usage, then reset quotas after user backlash — xiaohu · 2026-11-09
- Two padding tokens doubled a hosted DeepSeek V4.1-Flash success rate, pointing to phase sensitivity — No_Pea_6625 · 2026-11-09
- Is Opus 5.5 nerfed? Users report thinking patterns changed on November 3 — rzk · 2026-11-09