VideoChat3 debuts as a fully open 4B video model for long and streaming clips

机器之心 · wechat · 2026-07-22

VideoChat3 brings fully open video understanding to 4B scale

Researchers from Nanjing University, Shanghai AI Lab, NTU, and Peking University released VideoChat3, a 4B-parameter multimodal model for general video understanding.

Main ideas

I3D-ViT moves temporal modeling into the vision encoder, doing joint space-time attention before the LLM sees the tokens
An adaptive streaming mechanism uses three states — Silence, Standby, and Response — to decide when to stay quiet and when to answer
The model is designed for short video, long video, and online streaming understanding in one system

Data and training

The team also open-sourced three datasets:

VideoChat3-Academic2M: 2.27M academic video QA/caption samples rewritten with evidence-bearing answers
VideoChat3-LV116K: 116K long-video samples with timestamped evidence
VideoChat3-OL617K: 617K streaming samples that teach the model when to respond

Training is done in four stages, from vision pretraining to long-video and streaming instruction tuning, using 25M samples in total.

Reported results

VideoChat3 says it matches or beats other open models across long-video and streaming benchmarks, including strong gains over Qwen3-VL-4B in direct comparisons. It also cuts latency and compute for longer inputs, with large wins at 2048 frames.

The project is fully open: data, code, and weights are all released.

Original post →

VideoChat3 debuts as a fully open 4B video model for long and streaming clips

More from Multimodal