vLLM-Omni adds day-0 support for AURA 2.0, a 9B streaming video model that talks first

vllm_project · x · 2026-11-18

The vLLM project announced day-0 support for AURA 2.0 in vLLM-Omni. AURA 2.0 is a 9B streaming video model that watches a live scene, proactively starts relevant conversations, and acts on what it sees: it decides when to speak without waiting for a question, performs stream-aware tool calling that folds results back into the dialogue, and offers interactive visual grounding with frame-aligned boxes.

On the serving side, vLLM-Omni powers the full pipeline from speech recognition through video-language inference to speech generation. With session/prefix reuse, a fast path for silent turns, and sentence-level speech handoff, it delivers a 344 ms median time to the first reply audio packet in a single-session setup on two 140 GB accelerators. AURA 2.0 leads all four aggregate benchmark comparisons shown, including IPIBench, StreamArena, and ProactiveVideoQA.

Original post →

More from Infra

Infra channel →