Multi-Branch Transformer scales LLMs with parallel branches, 51.4% faster decoding

_akhaliq · x · 2026-11-19

A new paper proposes the Multi-Branch Transformer, which scales LLMs by branching and running full Transformer blocks in parallel instead of only making them wider or deeper. It reportedly reaches lower validation loss at matched compute and up to 51.4% faster decoding, offering a third scaling axis beyond width and depth.

Related event: Multi-Branch Transformer Scales LLMs via Parallel Branches(3 posts)→

Original post →

More from Research

Research channel →