Princeton study: diffusion models barely extrapolate basic visual skills, ~50% on counts
NielsRogge · x · 2026-11-23
Princeton researchers Amish Sethi, Boya Zeng, Wenhao Chai, and Zhuang Liu released "Do Diffusion Models Learn to Generalize Basic Visual Skills?" (arXiv:2611.22427), introducing the Visual Generative Lab (VGL), a controlled framework that trains diffusion models from scratch on rule-generated synthetic data to precisely measure generalization of basic visual skills — size, position, rotation, count, color, shape — something FID/IS metrics and prompt-following benchmarks can't reveal.
Key findings:
- Limited extrapolation: size and position degrade within a few pixels outside the training range; out-of-range counting reaches only 50% accuracy; rotation extrapolation varies widely across seeds.
- Compositional generalization: the number of distinct skill combinations matters more than dataset size.
- Skills mostly fail independently, except position.
Code and data-generation scripts are open-sourced on GitHub.
More from Multimodal
- Dev builds interactive 3D human eye with Gemini Neon to teach how vision works — Scobleizer · 2026-11-23
- AI drops Goku into real life with convincing physics and lighting, blurring what's real — heypearlai · 2026-11-23
- FaceRefine Plus v0.2.0 speeds up MiniMax H3 video face refining by 2.7x — AnniCodex · 2026-11-23
- Anima Artifacts demos a 'living magazine' you converse with instead of read — alexcovo_eth · 2026-11-23
- Turning real F1 telemetry into an AI video: 5 nights of failed experiments — BittiAI · 2026-11-23
- Full open-source pipeline for realistic 4K video: 3.7MP base, tiled latent pass, DLSS 5 — Grinderius · 2026-11-23