TPHC: tensor-parallel grouped updates push Hyper-Connections beyond four residual streams
Xinying Hu · hf · 2026-11-18
ShanghaiTech University and Meituan propose TPHC, improving Hyper-Connections (HC) residual streams in Transformers.
- Manifold-constrained HC (mHC, adopted by the DeepSeek-V4 series) benefits little from more than four streams because each sublayer writes a single output to all streams, limiting independent updates.
- TPHC integrates tensor-parallel shards with HC streams to enable more independent stream updates.
- It improves language model loss and zero-shot reasoning with modest training overhead.
More from Research
- Voyager 1 Reaches One Light-Day From Earth, Signals Now Take 24 Hours — snowclipsed · 2026-11-18
- TCS Hiring Statement Urges Quality Over Quantity as AI Fuels Paper Arms Race — fortnow · 2026-11-18
- New paper: a sociocultural approach to linguistic creativity — abenitezburraco · 2026-11-18
- 27B Model Rewrites Its Own Weights Without Training or Gradients, Paper Claims — udmrzn · 2026-11-18
- From 'impossible' to a working Bluetooth beacon: open-source EDA chip design is real — IgorCarron · 2026-11-18
- AURA 2.0: a streaming video LLM that speaks first, calls tools, and grounds visually — Shuhan Li · 2026-11-18