BigMac keeps LLM pipeline speed while capping multimodal activation memory
小红书技术REDtech · wechat · 2026-07-22
BigMac proposes a dependency-safe nested pipeline for multimodal training: keep the LLM pipeline as the backbone, then insert encoder and generator work only where dependencies allow, so throughput stays high while activation memory remains bounded.
Key points:
- It separates global scheduling from runtime execution with a scheduler/executor design.
- Model authors can wire modules without rewriting pipeline communication logic.
- A profiler, simulator, and visualization tooling help diagnose bubbles and test PP/VPP/microbatch choices.
- On two representative workloads, BigMac reports 1.08–1.1× speedups over a compute-efficient baseline and 1.6–1.9× over a memory-efficient baseline, while keeping memory stable as batch size grows.
Related event: BigMac Accelerates Multimodal Training with Nested Pipelines(2 posts)→
More from Infra
- LLM Serving Metrics Thread: Why TPOT and Uptime Make or Break User Experience — abhijithneil · 2026-09-11
- PlanetScale launches sharded Postgres: 768 servers acting as one, 1PB scale — dhruv2038 · 2026-09-11
- Can a 7900 XTX 24GB run Qwen locally? Reddit seeks ROCm tok/s benchmarks — thenomadexplorerlife · 2026-09-11
- RTK Terminal Compression Cuts Tokens but Leaves Your AI Coding Bill Unchanged — Bartaseth · 2026-09-11
- SF Compute founder: buying compute is 'an absolutely awful experience' right now — IgorCarron · 2026-09-11
- SmolVM open-sources persistent computer infrastructure for agents that outlive chat sessions — aniketmaurya · 2026-09-11