vLLM-Omni Enables Real-Time Serving for MiniMax H3
vllm_project · x · 2026-09-02
The vLLM team blog post details system-wide optimizations enabling real-time video generation for MiniMax H3. Key improvements include Fast Ulysses communications for long-context attention, fully fused DiT operators (RMSNorm+RoPE, SwiGLU), parallel video/audio VAE decoding across 8 GPUs, parallel H.264/AAC muxing, and integration with FastVideo's 4-step FastH3 distillation. On an 8xB300 GPU setup, generating a 10.125-second MP4 takes just 8.7 seconds, achieving real-time performance.
Related event: vLLM and FastVideo Make MiniMax H3 Video Generation Faster Than Playback(7 posts)→
More from Infra
- CoreWeave co-founders cash out $3.3B in stock as AI credit risks mount — SumitGup · 2026-10-18
- ClickHouse aggregates 100B rows in 0.4s with parallel replicas auto-scaling — Abhishekcur · 2026-10-18
- Auction-based LLM routing paper finds fixed token pricing wastes 10%-71% in costs — _onionesque · 2026-10-18
- Running Ling 3.1 Flash (560B MoE, 25B Active) Across 4 DGX Sparks via llama.cpp RPC — BerryBlush_Rogers691 · 2026-10-18
- SGLang Summit sponsors jump from 11 to 36 in one week — ying11231 · 2026-10-18
- Oracle CDS hits record 300bps, called first AI trade domino after Barclays sell downgrade — arthurcolle · 2026-10-18