Diffusion Models Scale Like LLMs, But Need 10x Data Per Parameter
burny_tech · x · 2026-08-25
The paper 'ABRA: Scaling Diffusion Image Training' reveals that while diffusion image models scale predictably like LLMs, their compute-optimal recipe differs significantly. They require roughly 200 image tokens per parameter (about 10x Chinchilla) and are far more tolerant to overtraining than undertraining. Thus, for fixed compute, training smaller models on more data is the safer bet.
More from Models
- GLM-5.3 unprompted reasons about how its actions affect its future training, worries Apollo — MariusHobbhahn · 2026-10-15
- Gemini 4 argon vs Opus 5.5 hands-on: 3 min for $3 vs 25 min for $8 — zacharynado · 2026-10-15
- SentinelOne says OpenAI did 4-6 weeks of reverse-engineering in 8 hours — OpenAI · 2026-10-15
- Qwen3.8 27B scores 23.57% on word-based addition without reasoning, 167/169 with it — dl_weekly · 2026-10-15
- Hacker turns EmbeddingGemma 2 into 288x realtime STT and a fast TTS — kastnerkyle · 2026-10-15
- Claude Fable 5.5 reportedly in partners' hands, release eyed next week — lyraxana · 2026-10-15