BigMac keeps LLM pipeline speed while capping multimodal activation memory
小红书技术REDtech · wechat · 2026-07-22
BigMac proposes a dependency-safe nested pipeline for multimodal training: keep the LLM pipeline as the backbone, then insert encoder and generator work only where dependencies allow, so throughput stays high while activation memory remains bounded.
Key points:
- It separates global scheduling from runtime execution with a scheduler/executor design.
- Model authors can wire modules without rewriting pipeline communication logic.
- A profiler, simulator, and visualization tooling help diagnose bubbles and test PP/VPP/microbatch choices.
- On two representative workloads, BigMac reports 1.08–1.1× speedups over a compute-efficient baseline and 1.6–1.9× over a memory-efficient baseline, while keeping memory stable as batch size grows.
Related event: BigMac Accelerates Multimodal Training with Nested Pipelines(2 posts)→
More from Infra
- RTX 5080 price jumps from $1658 to $2318 in ten days on Amazon — Frizzy-MacDrizzle · 2026-10-27
- ComfyUI on 12GB VRAM: measured peaks show offloading turns 24s into 11 minutes — Impossible-Mail-4619 · 2026-10-27
- Neon rebuilds its docs search entirely on its own stack: Lakebase Search, Functions, AI Gateway — matei_zaharia · 2026-10-27
- PyTorch details 2.12x faster on-device streaming ASR on Arm SME2 via ExecuTorch — PyTorch · 2026-10-27
- NVIDIA shows speculative decoding speeding up LLM inference on Jetson with Qwen3.5 9B — NVIDIA Developer · 2026-10-27
- Revenue per GPU can rise as token prices fall, argues Gavin Baker — GavinSBaker · 2026-10-27