First-order world-model gradients give minor training speedups over zero-order PG
breadli428 · x · 2026-10-26
The author, whose work shows world-model gradients can help train policies in imagination (contradicting RWM's finding that such gradients are sharp at contact changes), argues the speedup from first-order gradients with BPTT over zero-order policy gradient is minor — and even more negligible when using a learned world model where samples are incredibly cheap.
Related event: Debate over world-model gradients: zero-order wins on speed(2 posts)→
More from Research
- DONIC hits 86.79% on bimanual dexterous tracking, crushing prior SOTA's 9.43% — liuziwei7 · 2026-10-27
- Why don't more LLMs ship 10M-token context? A researcher asks about attention tradeoffs — agarwl_ · 2026-10-27
- PSNR below 15 dB in papers likely means the metric was misused, researcher warns — kwangmoo_yi · 2026-10-27
- Ortet's first health eval: 34 open-weight models, 50 tasks, generic rankings fall short — thegautamkamath · 2026-10-27
- Scaling-fit discrepancy: output-head FLOPs are ~65% of compute at the smallest rung, ~3% at 535B — dlwh · 2026-10-27
- Marin's dlwh Explains the Scaling Law Fit: Estimating the Loss Floor Early — dlwh · 2026-10-27