When tasks can only be verified in the real world, RL rewards become reality itself
yacineMTB · x · 2026-10-28
Yacine MTB amplifies a point from @hallerite: once tasks can only be verified by running them in the real world, rewards may naturally converge to real outcomes — did your PR get merged by the maintainers of the big repo? Did you make money on this option?
Quoted context adds an RLHF anecdote: the original RLHF had human labelers rate model outputs in real time as the policy trained — a logistical nightmare requiring staggered shifts. The paper even calls out one slow labeler who threw off training on one of the tasks.
More from AGI Musings
- Rob LeClerc on why SWEs survive the AI era: 'it's the Red Queen effect' — robleclerc · 2026-10-28
- Geoffrey Irving: Public companies are allowed to choose not to risk killing everyone — geoffreyirving · 2026-10-28
- AI's most superhuman ability may be its capacity to forget — and keep secrets — corbtt · 2026-10-28
- Security researchers weigh whether AI-driven economic disruption will fuel global cybercrime — joshua_saxe · 2026-10-28
- ~20% of ICLR reviews may be AI-generated: Heng Li pushes back, expert says AI review can beat average humans — anshulkundaje · 2026-10-28
- Noah Smith: AI is the second-to-last important technological advance of the human age — abhiadesai · 2026-10-28