NVIDIA Releases Open-Source AV-Flamingo Audio-Visual LLM
nvidia · hf · 2026-07-20
NVIDIA has released the fully open-source audio-visual large language model Audio-Visual Flamingo (AV-Flamingo), specializing in understanding and reasoning over long, complex real-world videos and audio.
Key highlights:
- Training Data & Methods: Built a dataset of roughly 7 million caption and QA instances, employing a three-stage curriculum learning approach that transitions from short-term perception to long-horizon, multi-event reasoning.
- Reasoning Framework: Introduces a temporal audio-video interleaved chain of thought, explicitly anchoring reasoning steps to timestamps in the audio-visual stream to improve temporal alignment and interpretability.
- Performance: Across 15+ audio-visual and omni-modal benchmarks, it not only significantly outperforms open-source models of similar scale but even surpasses larger closed-source models on certain long-video tasks.
More from Multimodal
- LichtFeld Studio v0.5.4.3 ships community-built SteadySplats, POPSpa and Voxelization — janusch_patas · 2026-10-26
- One prompt to a 2-3 min narrated short: hands-on test of Antigravity video generation — AI_Andrew · 2026-10-26
- Creator makes 10,564-frame animated short entirely with Gemini toolchain — AI_Andrew · 2026-10-26
- FLUX.3 Klein step-distilled to 4-8 steps with CFG baked in — Fit-Structure-6489 · 2026-10-26
- ostris pre-training "Primal VAE": 1-channel latents match modern 8x VAEs via patching — ostrisai · 2026-10-26
- Community checkpoint 'Narrative Drift' brings art-directed visuals to Krea 2 Turbo — MoonbearAIArt · 2026-10-26