Multi-Branch Transformer Scales LLMs via Parallel Branches

Researchers from UIUC, Rice and Cornell propose the Multi-Branch Transformer, which scales LLMs by running complete transformer blocks in parallel branches, saving 1.26x compute at equal loss and speeding up decoding by 51.4%.

2026-11-18 ~ 2026-11-19 · 3 related posts