VideoChat3 debuts as a fully open 4B video model for long and streaming clips
机器之心 · wechat · 2026-07-22
VideoChat3 brings fully open video understanding to 4B scale
Researchers from Nanjing University, Shanghai AI Lab, NTU, and Peking University released VideoChat3, a 4B-parameter multimodal model for general video understanding.
Main ideas
- I3D-ViT moves temporal modeling into the vision encoder, doing joint space-time attention before the LLM sees the tokens
- An adaptive streaming mechanism uses three states — Silence, Standby, and Response — to decide when to stay quiet and when to answer
- The model is designed for short video, long video, and online streaming understanding in one system
Data and training
The team also open-sourced three datasets:
- VideoChat3-Academic2M: 2.27M academic video QA/caption samples rewritten with evidence-bearing answers
- VideoChat3-LV116K: 116K long-video samples with timestamped evidence
- VideoChat3-OL617K: 617K streaming samples that teach the model when to respond
Training is done in four stages, from vision pretraining to long-video and streaming instruction tuning, using 25M samples in total.
Reported results
VideoChat3 says it matches or beats other open models across long-video and streaming benchmarks, including strong gains over Qwen3-VL-4B in direct comparisons. It also cuts latency and compute for longer inputs, with large wins at 2048 frames.
The project is fully open: data, code, and weights are all released.
More from Multimodal
- fable 5.1 recreates The Starry Night with 256,157 JavaScript brush strokes — cedric_chee · 2026-09-11
- GPT-6 Astra + Hyper3D Rodin MCP Generates 3D Assets in One Agent Flow — ahuja_priyank · 2026-09-11
- Five tells that still make AI video read as AI, from physics glitches to missing operators — NewPhoneWhotiz · 2026-09-11
- MiniMax + ComfyUI used to create Michael Jackson moonwalk-on-the-moon short film — Inside-Cantaloupe233 · 2026-09-11
- GPT Image 2 'Impossible Fit' prompt turns your product into the missing shape — aziz4ai · 2026-09-11
- Invideo launches agent-driven video editor that executes edits from plain descriptions — azed_ai · 2026-09-11