Fully Open-Source Video MLLM VideoChat3
MCG-NJU · hf · 2026-07-17
VideoChat3 is introduced as a fully open, more efficient, and highly generalizable video MLLM.
Main Goals
Common issues with existing open-source video models:
- Limited generalization, working well only in a few scenarios
- High computational overhead, making scaling difficult
- Partial openness, with incomplete training code/strategies/datasets affecting reproducibility
Method Design
The authors improved upon these from two directions:
1) Enhancing Efficiency
- Introduced Inflated 3D Vision Transformer (I3D-ViT)
- Adopted Adaptive Frame Resolution for streaming video perception
- Reduced the cost of processing video inputs during training and inference
2) Enhancing Performance
Built a scalable video data synthesis pipeline, organizing three training sets:
- VideoChat3-Academic2M
- VideoChat3-LV116K
- VideoChat3-OL617K
Covering general, long video, and streaming video scenarios.
Results
On general, long video, and streaming video benchmarks, VideoChat3 (with 4B parameters) achieved better results and higher efficiency than previous open-source models of the same or even larger scales.
Related event: Fully Open-Source Video MLLM VideoChat3 Released(3 posts)→
More from Multimodal
- Fish Audio ships open-weight Drama 3 TTS; creator shows turning old posts into narrated videos — alex_verem · 2026-10-27
- Redditor crafts photorealistic 'morning bus ride' short film entirely in ComfyUI — no3us · 2026-10-27
- New blur/anti-blur slider LoRA for MOSS lets you dial background sharpness with weights — NoMouse9610 · 2026-10-27
- PixGR: Pixel-space generative decoding for fast feed-forward 3D reconstruction — kwangmoo_yi · 2026-10-27
- Krea-2 Turbo + ComfyUI for product ads: prompts and results shared — comfy-king-999 · 2026-10-27
- High-res tiled Marigold V2 depth maps arrive in ComfyUI with See Through model — AntonObukhov1 · 2026-10-27