Peking University & Xiaohongshu Introduce BigMac: 1.9x Faster Multimodal Training with Stable Memory
jiqizhixin · x · 2026-08-04
Researchers from Peking University, independent researchers, and Xiaohongshu introduce BigMac, a new training pipeline for multimodal large language models. BigMac cleverly nests the encoder and generator work inside the standard LLM training pipeline, executing them in a safe, dependency-friendly order.
This design slashes the memory required during training to a constant amount, allowing the system to run at full speed without actually requiring unlimited memory. Experiments show that BigMac delivers up to 1.9x faster training compared to existing systems while keeping memory usage stable even as batch sizes scale up, breaking the long-standing Pareto frontier trade-off between compute and memory in multimodal AI training.
More from Research
- Humanoid RL Shows Scaling Laws: Picking Throughput Jumps 85% — lukas_m_ziegler · 2026-08-04
- CCN 2026 Preview: Scaling Laws for Visual Cortex and NeuroAI Frontiers — martin_schrimpf · 2026-08-04
- Mila Researcher Shares AfroXLMR and AfriqueLLM Models for 20 African Languages — hugo_larochelle · 2026-08-04
- Paper Explores a Third Pretraining Axis via Explorative Modeling — Benlus · 2026-08-04
- From Representation Learning to Continual Learning Agents: New AIxIA Paper — v_lomonaco · 2026-08-04
- Text Diffusion Models Show Latency and Throughput Gains for Longer Contexts — bodonoghue85 · 2026-08-04