Spotify Uses LLM Explanations for RL Credit Assignment, Beating PPO on Delayed Rewards
_reachsumit · x · 2026-10-12
Spotify researchers propose Explanatory World Models (EWM): use a frontier LLM's explanations to identify which earlier events plausibly caused an outcome, then recursively redistribute reward via a conservative hierarchical operator (preserving undiscounted return), replacing temporal-proximity credit assignment.
Three explanation variants draw on Halpern's actual causation, Lombrozo's explanatory simplicity, and Deutsch's hard-to-vary explanations. On Crafter and WordCraft, EWM substantially beats PPO and LaRe with the largest gains under the most delayed rewards; on KuaiRand it beats SFT/LaRe offline and A2C online, plus an offline evaluation on an industry-scale generative recommender.
More from Models
- Dev hopes next GPT matches Opus 5.5 on taste and clarity, not raw coding — jiayuan_jy · 2026-10-12
- Power user burns 86% of Codex weekly quota in 25 hours, complains resets can't keep up — AlchainHust · 2026-10-12
- Unverified report: consumer Kimi jailbroken, allegedly makes unrestricted mode persistent — lethendworld · 2026-10-12
- Ethan Mollick: GPT-6 Pro keeps finding paper errors, and academia has no mechanism for the flood — emollick · 2026-10-12
- Normies assume OpenAI's 700-point math benchmark win came from stolen mathematician chat logs — basedjensen · 2026-10-12
- Musk: FSD's neural net ate all the C++ code, now safer than most Houston drivers — oilmutt · 2026-10-12