Google preprint StreamVibe trains streaming-video assistants on when to speak up
jiqizhixin · x · 2026-11-11
StreamVibe, a preprint from Google and academic collaborators, tackles proactive assistants that must track which requests still matter rather than just remembering what users once said.
- StreamVibe Harness: tracks user-task progress, assigns each intent a lifecycle, and retrieves evidence from hierarchical memory (user preferences, past interactions, domain knowledge) into a unified interaction context.
- Two-stage training: Stage 1 is adaptive streaming SFT learning per-second wait-vs-speak decisions; Stage 2 is localized trajectory RL with point-level rewards (timing + content) and trajectory-level rewards (coverage, density, order, consistency, redundancy).
- StreamVibe-Bench: 4,100 evaluation samples, 20,546 help points, and 183.4 hours of video spanning explicit, implicit, and changing intents. Under full-trajectory evaluation with ±2s matching tolerance, the RL model reports Content F1 scores of 10.83, 19.13, and 8.91 across the three intent categories.
More from Multimodal
- MiniMax H3 Image generates a GTA6-style hurricane scene from a single prompt — socialwithaayan · 2026-11-11
- Dragon Ball reimagined as a 1971 movie: another hit AI retro trailer — techhalla · 2026-11-11
- Solo creator makes 20+ minute AI anime episode in three weeks — Kavanthekid · 2026-11-11
- Open-source PersoDub dubs videos in your own voice across 100+ languages, locally — bigaiguy · 2026-11-11
- Testing MiniMax H3-Image for fashion contact sheets: multi-frame consistency holds up — Hot-Purchase-3738 · 2026-11-11
- Midjourney --sref 1109104433: painterly urban crowd illustrations — sergeantsref · 2026-11-11