Agent-judged rewards: hacking fades as models get smarter and tasks easier to verify
willcb · x · 2026-09-27
A discussion on designing rewards in open-ended RL settings:
- Rewards are typically given by another agent following clear instructions on what counts as good/bad
- Reward hacking remains possible but becomes harder as models grow smarter and more robust
- Another key knob is QC and difficulty tuning: easier-to-validate tasks are less hack-prone
Scaling robustness alongside optimizer power is considered fairly doable with a good starting point.
More from Research
- microVLA runs pi0.5 VLA training and inference on a 4GB RTX 3050, 2.5x latency cut — this_is_surabhi · 2026-11-16
- WebArena-Pro launches: 300-task multimodal web-agent benchmark across 20 apps — xhluca · 2026-11-16
- U-M's CrowdBark crowdsources dog videos to study canine communication with AI — radamihalcea · 2026-11-16
- Top 10 Eval Papers of 2026: verifiers, agent-as-a-judge, and audit of self-improvement — omarsar0 · 2026-11-16
- Koshien robot team shares control stack: nonlinear-opt trajectories hit ±1.2cm average error — 4310sy · 2026-11-16
- A diffusion model trained on CIFAR-10 re-images black hole M87* — and the ring persists — bravo_abad · 2026-11-16