NVIDIA Releases Open-Source AV-Flamingo Audio-Visual LLM
nvidia · hf · 2026-07-20
NVIDIA has released the fully open-source audio-visual large language model Audio-Visual Flamingo (AV-Flamingo), specializing in understanding and reasoning over long, complex real-world videos and audio.
Key highlights:
- Training Data & Methods: Built a dataset of roughly 7 million caption and QA instances, employing a three-stage curriculum learning approach that transitions from short-term perception to long-horizon, multi-event reasoning.
- Reasoning Framework: Introduces a temporal audio-video interleaved chain of thought, explicitly anchoring reasoning steps to timestamps in the audio-visual stream to improve temporal alignment and interpretability.
- Performance: Across 15+ audio-visual and omni-modal benchmarks, it not only significantly outperforms open-source models of similar scale but even surpasses larger closed-source models on certain long-video tasks.
More from Multimodal
- Open-source FableCut lets AI agents edit a real video timeline via remote MCP — FastPresence9799 · 2026-10-27
- Anima user struggles to keep generated lanterns in the distance via prompts — MadaleineSakurai · 2026-10-27
- MOSS-Image ecosystem roundup: sharpness slider, manga alpha LoRA, box inpainting — optimisticalish · 2026-10-27
- Wan2.2 image-to-video Space trends on Hugging Face — JCisHERE · 2026-10-27
- PlayCanvas' open-source splat-transform tool hits ~35K weekly NPM downloads — willeastcott · 2026-10-27
- Creator builds Snorricam video series with Seedance 2.5, sharing prompts — LudovicCreator · 2026-10-27