Diffusion models can backprop end-to-end but costly; policy gradients only necessary for long-horizon language models
kalomaze · x · 2026-08-02
Author kalomaze notes that diffusion models can in principle backprop through the entire chain differentiably, but it's expensive. This cost forces the branching rollouts abstraction from ByteDance papers, which is only necessary for long-horizon language models. He also observes that almost no one uses policy gradients on diffusion models, except possibly ByteDance's branchgrpo-style works with in-house VLM reward models.
Related event: Policy Gradient in Diffusion Models Sparks Debate(3 posts)→
More from Research
- Transforming quadruped robot runs on real hardware: per-pose RL policies in Isaac switched on morph — _Stocko_ · 2026-10-16
- KinoGen synthesizes robot loco-manipulation reference motions directly via kinematically grounded video generation — ChongZzZhang · 2026-10-16
- Science Robotics paper introduces F³, predictive event-based vision beyond optical flow — CSProfKGD · 2026-10-16
- VideoGen-Agent: An 8B Multimodal Agent Turns Video Generation Into a Tool-Use Loop — jiqizhixin · 2026-10-16
- Comprehension Audits: New Paper Proposes a Gate for AI-Written Code at Frontier Labs — ronbodkin · 2026-10-16
- Not Boring #215: Cammarata's Clean Networks Claim, Tesla's Flying Roadster, Tau Cut 73% — Not Boring (Packy McCormick) · 2026-10-16