Multi-Branch Transformer scales LLMs with parallel branches, 51.4% faster decoding
_akhaliq · x · 2026-11-19
A new paper proposes the Multi-Branch Transformer, which scales LLMs by branching and running full Transformer blocks in parallel instead of only making them wider or deeper. It reportedly reaches lower validation loss at matched compute and up to 51.4% faster decoding, offering a third scaling axis beyond width and depth.
Related event: Multi-Branch Transformer Scales LLMs via Parallel Branches(3 posts)→
More from Research
- Baseten hosts All-Gather, its first research conference on where AI's next gains will come from — baseten · 2026-11-19
- GPT-6.2 Astra produces first Lean proof of Smoktunowicz's Gap Theorem in 94 minutes — VraserX · 2026-11-19
- Interpretability solved—then what? Najoung Kim proposes filtering for recurrent constructs — najoungkim · 2026-11-19
- New preprint asks: if interpretability is "solved," what does it mean for the science of human language? — najoungkim · 2026-11-19
- Map of tabular foundation models shows only two are fully open — GaelVaroquaux · 2026-11-19
- Google open-sources TPU codesign guide: 8 principles for LLM/MoE performance — rseroter · 2026-11-19