Multi-Branch Transformer scales LLMs via parallel branches, saving up to 1.26x training compute

KyeGomezB · x · 2026-11-18

Researchers from UIUC, Rice, Michigan and Cornell propose the Multi-Branch Transformer (MBT), a scaling path beyond making blocks wider or deeper: composing full Transformer blocks in parallel.

Code and project page are open source.

Original post →

More from Infra

Infra channel →