AURA 2.0: a streaming video LLM that speaks first, calls tools, and grounds visually
Shuhan Li · hf · 2026-11-18
A team from HUST, CUHK, HKUST and Huawei released the AURA 2.0 technical report: a streaming video LLM that goes beyond wait-and-answer interaction with three core upgrades.
- Model-Initiated Conversation (MIC): identifies topics worth discussing in the evolving video stream and starts conversations without a user query
- Stream-Aware Tool Calling (SATC): invokes external tools based on current visual and conversational context, folding results back into the ongoing interaction
- Interactive Visual Grounding (IVG): grounding on both user inputs and model outputs, marking objects and linking responses to frame-aligned bounding boxes
It also adds Long-Horizon Interaction Memory (LHIM) for contextual continuity over extended streams, a dedicated streaming interaction dataset, and a three-stage training scheme. Evaluations show leading performance among open-source streaming models.
More from Multimodal
- New tool launches beta for cinematic science videos with every molecule verified — realmeetjames · 2026-11-19
- Multiple Google employees tease "Gemini", hinting at an imminent new model release — haider1 · 2026-11-19
- Meta's AVTR adds visual imagination routing: +4.5 pts, 33% lower latency — meta · 2026-11-19
- MOSS-Image SFT releases 4-step distilled LoRA using eDMD + LADD for high-detail T2I and I2I edits — TimeTruth2490 · 2026-11-19
- MOSS 4-Step Distill LoRA Cuts GPU Time 14x With Better Detail — TimeTruth2490 · 2026-11-19
- First Niji v8 previews surface ahead of imminent Midjourney release — LeelooAI_fr · 2026-11-19