DDRL ablations show SFT stage can be skipped, matching rewards straight from pretraining
gowthami_s · x · 2026-10-15
Continuing her DDRL paper thread, gowthamis covers ablations on Cosmos2.5 checkpoints (2B/14B, 93 frames at 720p, 1,044 prompts): DDRL trails DanceGRPO on VideoAlign reward (0.604 vs 0.715) for 2B text-to-video yet wins human preference by 10.5 points (9.6k comparisons). Skipping the separate 20k-step SFT stage and running DDRL from pretraining yields comparable rewards and diffusion loss with far better data efficiency. Caveats: only 15 raters, roughly half authors, and RL rollouts still dominate compute.
Related event: DDRL keeps diffusion loss during RL to fix reward hacking in video models(5 posts)→
More from Research
- 1-gram wireless mesoscope images cortex-wide activity in freely moving mice — KordingLab · 2026-10-23
- Reversible computing veteran poses open problem: universal BARCS primitives from unbiased Josephson junctions — MikePFrank · 2026-10-23
- Arc Institute uses genomic language model to map 376M prokaryotic noncoding sequences into 5.14M clusters — tedgreenwald · 2026-10-23
- The rise of AI-assisted peer review: reviewers are making it into the appendix — abursuc · 2026-10-23
- Anthropic uses Claude to formally verify security properties in 100+ open-source libraries at RISC-V assembly level — AnthropicAI · 2026-10-23
- Researchers test 9 strategies to pick LLM teams, beating random selection by up to 36.1% — shangbinfeng · 2026-10-23