DDRL ablation: skipping SFT and running RL straight from pretraining works
gowthami_s · x · 2026-10-15
Parts 4-5 of the thread: the paper ablates skipping SFT since DDRL already includes a supervised loss. Removing the separate 20k-step SFT stage and running DDRL directly from pretraining on the same dataset yields comparable rewards and diffusion loss to SFT→DDRL — works in this setup, though the x-axis is data used and expensive RL rollouts still dominate compute. The author's own take: don't skip SFT; you still need good rollouts to do RL.
Related event: DDRL keeps diffusion loss during RL to fix reward hacking in video models(5 posts)→
More from Research
- 1-gram wireless mesoscope images cortex-wide activity in freely moving mice — KordingLab · 2026-10-23
- Reversible computing veteran poses open problem: universal BARCS primitives from unbiased Josephson junctions — MikePFrank · 2026-10-23
- Arc Institute uses genomic language model to map 376M prokaryotic noncoding sequences into 5.14M clusters — tedgreenwald · 2026-10-23
- The rise of AI-assisted peer review: reviewers are making it into the appendix — abursuc · 2026-10-23
- Anthropic uses Claude to formally verify security properties in 100+ open-source libraries at RISC-V assembly level — AnthropicAI · 2026-10-23
- Researchers test 9 strategies to pick LLM teams, beating random selection by up to 36.1% — shangbinfeng · 2026-10-23