Qwen presents OmniVChat: synthesized data, benchmark and RL for native audio-visual dialogue
Qwen · hf · 2026-09-21
Qwen's team introduces OmniVChat, the task of native audio-visual dialogue where an omni model directly consumes simultaneous audio+video input (queries embedded in the stream, no separate text, captioning, or ASR), cutting latency while preserving perceptual cues. To tackle data scarcity and evaluation, the team builds OmniVChat-Studio, a multi-agent engine for synthesizing single- and multi-turn dialogues, and OmniVChat-Bench spanning five ability categories (plus a human-recorded variant). They also propose OmniVChat-RL, a reward design jointly targeting correctness, efficiency, and style. Training Qwen3-Omni-Instruct on synthesized dialogues improves scores on both benchmarks, validating the reward design and showing transfer to real-world conversations.
More from Multimodal
- Open-source full-duplex speech model caramel-1: 413M params, sounds human but can't hold a conversation — alexcovo_eth · 2026-11-12
- H3-Image touts 16MP generation and realtime inference via 2-step LoRA on ZeroGPU — linoy_tsaban · 2026-11-12
- MiniMax H3 Image renders 4K portrait in half the time Seedream 5.5 takes for 2K — heyronir · 2026-11-12
- Endless screensaver of self-assembling quantum cryostats, made with Gemini 4 Carbon — hapticdata · 2026-11-12
- Apple's ADM 3 Cloud image model ranks #42 in editing and #85 in text-to-image on Artificial Analysis arena — ArtificialAnlys · 2026-11-12
- Adobe Firefly debuts Story Builder tool for streamlined visual storytelling — LudovicCreator · 2026-11-12