torchcompiled: slower diffusion training likely stems from extra gradient noise
torchcompiled · x · 2026-11-12
Continuing the TPP metric discussion, torchcompiled notes that diffusion genuinely does train slower, which he attributes to gradient noise beyond regular mini-batch sample noise — specifically the extra sampling of timestep t and eps/noise per sample.
Related event: Researchers debate TPP as a training metric for diffusion models(3 posts)→
More from Research
- SteerDistill: teacher-guided quantization keeps W4A16 agents within 1 point of BF16 — _akhaliq · 2026-11-12
- MIMIC and BMW train video-action model hitting up to 100% success on industrial tasks — pess_r · 2026-11-12
- Paris seminar: Monte Carlo methods bring exactness guarantees to diffusion model sampling — kastnerkyle · 2026-11-12
- Doris Tsao: recording more neurons won't crack the brain — the field needs new frameworks — patrickmineault · 2026-11-12
- Dyna co-founder Jason Ma keynotes on scaling real-world robot foundation models — chris_j_paxton · 2026-11-12
- 1968 magazine pages show early computer art: IBM 7090 plotters and Japan's CTG pioneers — Merzmensch · 2026-11-12