Salesforce Paper: Only 35.8% of the Cleanest Public RL Environments Pass Audit
dair_ai · x · 2026-09-24
A Salesforce AI Research paper highlights the importance of good verifiers in RL environments: an audit found only 35.8% of environments in the cleanest public RL collection for terminal agents were sound, with two other collections at just 10.1% and 3.3%.
Key findings:
- Reward errors go both ways: some environments give reward 1 for copying leaked answers or passing weak verifiers, others give 0 to correct solutions because the reference answer or oracle is wrong.
- With budget fixed at 3.5K environments, River-8B averaged 19.4 across four terminal benchmarks vs 17.7 for RL trained on randomly sampled environments.
- The authors argue RL mainly shapes behaviors — inspecting before acting, verifying before finishing, dropping dead-end approaches — and good verifiers determine whether those behaviors get reinforced correctly.
More from coding & agent
- Why AI-made dev tools beat one-shot game generation: freedom of process — eschadiol · 2026-09-24
- RealSense VP on AgenticROS: Letting AI Agents Directly Control Physical Robots — chrismatthieu · 2026-09-24
- AWS API Gateway's Hard 10MB Upload Limit and the Presigned URL Fix — _jaydeepkarale · 2026-09-24
- Open-source Pragma gives coding agents a terminal-first workspace with Git worktrees — tech_w0rld · 2026-09-24
- Dev builds dense task annotation system with GPT-6 Astra, ships it as an LLM skill — chris_j_paxton · 2026-09-24
- Jev-as-a-Judge: hybrid agent eval flow escalates low-confidence calls to frontier models — omarsar0 · 2026-09-24