BigMac keeps LLM pipeline speed while capping multimodal activation memory
小红书技术REDtech · wechat · 2026-07-22
BigMac proposes a dependency-safe nested pipeline for multimodal training: keep the LLM pipeline as the backbone, then insert encoder and generator work only where dependencies allow, so throughput stays high while activation memory remains bounded.
Key points:
- It separates global scheduling from runtime execution with a scheduler/executor design.
- Model authors can wire modules without rewriting pipeline communication logic.
- A profiler, simulator, and visualization tooling help diagnose bubbles and test PP/VPP/microbatch choices.
- On two representative workloads, BigMac reports 1.08–1.1× speedups over a compute-efficient baseline and 1.6–1.9× over a memory-efficient baseline, while keeping memory stable as batch size grows.
More from Infra
- Poolside releases Laguna S 2.1 with 118B MoE, 1M-token context and open weights — nathanbenaich · 2026-07-22
- llama.cpp adds support for Laguna XS.2 and M.1 in release b10087 — LaurentPayot · 2026-07-22
- LightOn Search hits 5th place on BrowseComp-Plus with 86.27% accuracy — CShorten30 · 2026-07-22
- Nvidia’s Spectrum switches add CPO support as Lambda tests early units — TheZachMueller · 2026-07-22
- Hermes Cloud adds on-demand disk expansion and CPU/RAM upgrades — Teknium · 2026-07-22
- Cloud reserved-instance resale has effectively disappeared, leaving AI teams stuck with bad capacity bets — DavidLinthicum · 2026-07-22