xHC: Scaling Residual Streams by 16x
teortaxesTex · x · 2026-07-17
This content focuses on a new architecture called xHC, aiming to further expand residual streams to boost model capacity and expressiveness.
It notes that early Hyper-Connections and mHC already proved that "residual streams don't have to be single-path," but original designs remained limited when scaling up. The proposed xHC uses temporal feature augmentation and a sparse residual stream architecture to simultaneously solve shared write-back components and rapidly growing generation costs at scale. It claims to effectively widen the residual stream significantly (mentioned as roughly 16x).
More from Research
- Navier-Stokes, Riemann, P vs NP: what this week's math buzzwords mean for you — koltregaskes · 2026-09-11
- Fruit fly connectome LLM weights land on Hugging Face, transformers-compatible — ngxson · 2026-09-11
- Fruit fly brain as an LLM: connectome-driven language model demo goes live — ngxson · 2026-09-11
- Harry Collins: LLMs can't do frontier science because they can't invent new language — whoamisri · 2026-09-11
- The Waymo effect: how AI is quietly making research less collaborative — JohnHammersley · 2026-09-11
- Causal-only attention for non-generative tasks is wasteful, argues HF engineer — antoine_chaffin · 2026-09-11