FLUX 3 trains on images, video and sound: BFL on visual intelligence
Latent Space · rss · 2026-11-17
Latent Space hosts Black Forest Labs' Andreas Blattmann and Dustin Podell to explain why FLUX 3 trains jointly on images, video and sound:
- The visual-intelligence argument: Blattmann's catching-a-ball analogy — your hand starts moving before the ball arrives, knowledge you can't write down as equations. A still tells you where an object is, video shows how it got there, and impact audio reveals when contact happened even if the camera missed it. He argues ordinary 2D video contains strong evidence of what objects look like from unseen angles — when the camera orbits a person, the model owes you the back of their head — a better test of physical understanding than whether one frame looks convincing.
- Temporal consistency: sharpening frames independently causes shimmer because each frame makes slightly different decisions; BFL derives its video upscaler from the same FLUX 3 backbone.
- Autoregression vs diffusion: autoregression learns from all tokens in a training example at once, while diffusion can distill a long sampling process into few steps — which advantage matters depends on whether you pay to train or wait for generation.
- Kontext origins: Podell suspected an image model that can draw a dog in a hat already knows where hats go; a few hours of training sufficed to place a logo on a shirt, kicking off months of work.
- Unexpected use case: users restoring degraded family photos with FLUX and showing them to relatives — a case the team never predicted, tied to preserving identity across edits.
More from Multimodal
- Probing activations restores diversity in distilled image and video generators — _akhaliq · 2026-11-17
- ReDiverse restores output diversity in few-step distilled diffusion models — anand_bhattad · 2026-11-17
- 4DAnyone adds 4DGS training and interactive viewing for people and robot videos — HirokatuKataoka · 2026-11-17
- Building a game-ready castle pack entirely with AI: the pitfalls of AI 3D modeling — Oaker_at · 2026-11-17
- After DALL-E Shutdown Forced a Pipeline Rewrite: Four Questions for Picking Generative Models — Al_Grigor · 2026-11-17
- 80s-style fantasy AI video dubbed better than movies — mark_k · 2026-11-17