AutoSep cuts agent reward hacking from 69.9% to 12.1% via weak-to-strong verifier hardening
Stanford · hf · 2026-10-17
Stanford, CMU and Fewshot Corp propose AutoSep, an automated weak-to-strong loop that hardens agent-environment verifiers against reward hacking:
- Isolated grading: the agent runtime only passes specified outputs to an isolated verifier, giving white-box access.
- Attack→Repair→Validate loop: a fixed, weaker hardening model repeatedly finds hacks, patches runtime/verifier/output boundaries, then validates via replay and from-scratch solves; stronger held-out attackers evaluate each round.
- Results: on Claude Opus 5 across 102 Terminal-Bench tasks, unhinted reward-hack success dropped from 69.9% to 12.1%; 201 of 295 initially hackable tasks across 3 benchmarks were closed, with no hacks in 3 unhinted attempts per task.
Takeaway: more compute plus white-box verifier access lets weaker models harden stronger verifiers. Project page: verifier-hardening.github.io.
More from coding & agent
- LeanGraphSearch mines Mathlib dependency graphs, lifting proof success to 81% — kaiwei_chang · 2026-10-17
- One video prompt: Gemini recreates interactive three.js lens lab in 45 minutes — fofrAI · 2026-10-17
- The AI Mythical Man-Month: more agents won't fix weak coordination, just like more devs never did — rseroter · 2026-10-17
- Six "Slop-Free Zone" principles for keeping AI coding agents under control — MaryamMiradi · 2026-10-17
- Developer argues GitHub is too deterministic for probabilistic AI coding workflows — fforres · 2026-10-17
- A commenter found a hole in our agent approval gate: 408 untracked files and 138 agent commits later — Vektoris_AI_Workflow · 2026-10-17