Fully Open-Source Video MLLM VideoChat3
MCG-NJU · hf · 2026-07-17
VideoChat3 is introduced as a fully open, more efficient, and highly generalizable video MLLM.
Main Goals
Common issues with existing open-source video models:
- Limited generalization, working well only in a few scenarios
- High computational overhead, making scaling difficult
- Partial openness, with incomplete training code/strategies/datasets affecting reproducibility
Method Design
The authors improved upon these from two directions:
1) Enhancing Efficiency
- Introduced Inflated 3D Vision Transformer (I3D-ViT)
- Adopted Adaptive Frame Resolution for streaming video perception
- Reduced the cost of processing video inputs during training and inference
2) Enhancing Performance
Built a scalable video data synthesis pipeline, organizing three training sets:
- VideoChat3-Academic2M
- VideoChat3-LV116K
- VideoChat3-OL617K
Covering general, long video, and streaming video scenarios.
Results
On general, long video, and streaming video benchmarks, VideoChat3 (with 4B parameters) achieved better results and higher efficiency than previous open-source models of the same or even larger scales.
Related event: Fully Open-Source Video MLLM VideoChat3 Released(3 posts)→
More from Multimodal
- Runway App Studio Demos: Character Swap and Production-Ready Character Sheets — runwayml · 2026-10-27
- Guy turns his girlfriend's selfie into a playable AI game ad about popping pimples — CurieuxExplorer · 2026-10-27
- Fibo 2 edit called unstable: even a color change alters the edited object — jd3k · 2026-10-27
- AIArtsy AI generates 3D cartoon-style sword fighting animation clip — Jealous-Associate-41 · 2026-10-27
- Seeking AI workflows for high-quality anime comics and colored manga pages — Worldly_Courage395 · 2026-10-27
- Open-source FableCut lets AI agents edit a real video timeline via remote MCP — FastPresence9799 · 2026-10-27