Luma's MARBLE: exact sequence parallelism for packed multimodal masks, wins 47/48 vs FlashMask+Ulysses
mrjonfinger · x · 2026-10-20
Luma released MARBLE, an exact sequence parallelism solution for packed multimodal masks. It selects per layer and workload among load-balanced ring attention (with PTRR), an online FlashAttention-4 kernel, chunked ring attention, and sparse all-to-all, matching single-GPU numerics while beating a FlashMask + Ulysses hybrid in 47 of 48 cases.
More from Infra
- Diamond-embedded copper vias shown to boost chip thermal conductivity — jwt0625 · 2026-10-20
- Stealth startup Physis debuts world foundation model play with PhysisServe, core tech headed to vLLM-Omni — vllm_project · 2026-10-20
- Barcelona Supercomputing Center AI Factory Accelerator Names Three AI Champions — NVIDIA Blog · 2026-10-20
- Dual R9700 layer-split side effect: per-card load under 50%, fans stay quiet — yahbluez · 2026-10-20
- PyTorch Conference poster to break down KV cache offloading: when CPU beats shared storage — PyTorch · 2026-10-20
- What Gen Models Run Well on a 3060/4060? Reddit Weighs Local Setups — Haghiri75 · 2026-10-20