Multi-Branch Transformer scales LLMs via parallel branches, saving up to 1.26x training compute
KyeGomezB · x · 2026-11-18
Researchers from UIUC, Rice, Michigan and Cornell propose the Multi-Branch Transformer (MBT), a scaling path beyond making blocks wider or deeper: composing full Transformer blocks in parallel.
- Better scaling: across a controlled pretraining ladder, MBTs beat standard Transformers at equal compute; a standard Transformer needs up to 1.26x more training compute to match MBT validation loss.
- Consistent pattern: with two branches, loss is lowest at 1/3 of blocks in parallel and only returns to baseline near 80%.
- Faster decoding: with Joint-KV cache sharing and a branch-aware inference engine, up to 51.4% higher throughput on one H200 and 34% less KV-cache storage.
Code and project page are open source.
More from Infra
- TokScale leaderboard: #1 user burned 4.65T tokens, est. $2.43M in cost — olcan · 2026-11-18
- Inside Microsoft's massive Outlook outage: what broke and why fixes took so long — tomwarren · 2026-11-18
- vLLM ships DSpark draft model for Kimi K3.1 Pro with up to 30% higher acceptance length — vllm_project · 2026-11-18
- Wonderful runs in-house voice models on Together AI at 274ms first sentence — togethercompute · 2026-11-18
- Anthropic's S-1 Shows Only 25% of Compute Goes to Inference — Sparking a Bear Case — toptickcrypto · 2026-11-18
- LanceDB Raises $57.5M Series B for AI-Native Multimodal Data Stack — pzakin · 2026-11-18