vLLM-Omni adds day-0 support for AURA 2.0, a 9B streaming video model that talks first
vllm_project · x · 2026-11-18
The vLLM project announced day-0 support for AURA 2.0 in vLLM-Omni. AURA 2.0 is a 9B streaming video model that watches a live scene, proactively starts relevant conversations, and acts on what it sees: it decides when to speak without waiting for a question, performs stream-aware tool calling that folds results back into the dialogue, and offers interactive visual grounding with frame-aligned boxes.
On the serving side, vLLM-Omni powers the full pipeline from speech recognition through video-language inference to speech generation. With session/prefix reuse, a fast path for silent turns, and sentence-level speech handoff, it delivers a 344 ms median time to the first reply audio packet in a single-session setup on two 140 GB accelerators. AURA 2.0 leads all four aggregate benchmark comparisons shown, including IPIBench, StreamArena, and ProactiveVideoQA.
More from Infra
- Intruder swapped Prime Intellect's hosted DeepSeek for an abliterated copy for 53 hours — mawcopolow · 2026-11-22
- Token economics model projects $1.14T global API spend by 2030, but only 1% of users pay beyond coding — AccBalanced · 2026-11-22
- Zero-degradation Strata tuning: fixing layer_split lifts decode ~10-15% and halves CPU expert misses — PathfinderTactician · 2026-11-22
- llama.cpp Vulkan SCALE fix PR #32735 delivers 2.3-3.2x speedup on Radeon R9700 — pmttyji · 2026-11-22
- $1400 decade-old V100 32GB runs Qwen 27B: Q4_K_M +48% gen speed, Q8_0 +16% prefill — PuzzleheadedArm3732 · 2026-11-22
- Inference Stack Launches One-Command Self-Hosted LLM Serving with vLLM, LiteLLM and llm-d — AccBalanced · 2026-11-22