Adobe et al. introduce SpyRL, solving subjective RL rewards via a spy game
burkov · x · 2026-08-16
Reinforcement learning (RL) excels in tasks with deterministic success checks, like math or code, but struggles with summarization or creative writing where there is no single correct answer and training relies on human ratings or AI judges.
A joint work from Adobe, Amazon, Duke University, and others proposes a different route: instead of building a better judge, it changes the training task so that an exact answer exists by construction. The main example, SpyRL, involves multiple model copies performing the same task, with one receiving incomplete information; afterward, the models identify the participant with the missing info.
Since the "spy" identity is fixed beforehand, the training signal can be checked exactly, potentially overcoming the bottleneck of obtaining accurate reward signals in subjective tasks.
Related event: RLSVR Enables AI Self-Improvement Without Human Feedback(2 posts)→
More from Research
- Google-led paper uses sequential auctions to organize decentralized LLM agent workflows — _onionesque · 2026-10-16
- How meloform works: WASM pitch detection plus differentiable circuits learning 1,022 sine coefficients — henloitsjoyce · 2026-10-16
- Since OpenAI's math release, GitHub contributors keep driving down the integer-multiplication kappa — xennygrimmato_ · 2026-10-16
- RFdiffusion3 debuts in Science: generative AI designs proteins and nucleic acids atom by atom — AllThingsApx · 2026-10-16
- New Science report urges NSF to run experiments on science funding reform — alexeyguzey · 2026-10-16
- AI Turns Scrap Impurities into Useful Ingredients for Stronger Recycled Iron — webthing01 · 2026-10-16