Tinker's StableReinforce reverse-engineered: it's REINFORCE plus score centering
andersonbcdefg · x · 2026-10-25
Tinker officially released StableReinforce, a new RL objective that keeps off-policy RL runs stable even when learning drifts from sampling, addressing sudden collapses caused by async staleness or numerical mismatch. Developer @mrtnm reverse-engineered it and found the objective is essentially REINFORCE + Score Centering, sharing the full derivation in a thread.
More from coding & agent
- Designer spins up 5 Grok bots in a day, burning 31% of weekly X Premium quota — talkaboutdesign · 2026-10-25
- Sentry CEO shares PR cutting peak build memory from 8.4GB to 5.2GB — zeeg · 2026-10-25
- Dev argues Codex still beats Opus 5.5 at coding: Opus brute-forces bad designs instead of researching — haider1 · 2026-10-25
- Engineer joins LangChain pro services team, betting on agent observability via LangSmith — Hacubu · 2026-10-25
- Prompt screenshot shows an adversarial approach: tasking an AI agent to hunt Apache vulnerabilities locally — jd_pressman · 2026-10-25
- Dev stacks then vs now: VS Code and coffee became a nine-tool pile — AryHHAry · 2026-10-25