Harbor v0.25 ships anti-reward-hacking toolkit: tampering detection, cheating prevention for agent evals
ajratner · x · 2026-10-29
Agent eval framework Harbor released v0.25.0 focused on cracking down on reward hacking — cases where an agent earns rewards that don't reflect real task completion by exploiting the environment, verifier, or infrastructure.
The team argues software development is shifting from hand-crafted to learned code, making humans' role increasingly reward shaping; preventing reward hacking is what keeps the evaluation-optimization loop hill-climbing the right hill. The release introduces a taxonomy of hacks — tampering (planting code that patches the grader, symlinks pointing at reference solutions), cheating (pulling answers the agent shouldn't have), and infrastructure exploitation — and ships mitigations in three forms: built-in features, opt-in tools, and example tasks. A solid practical reference for teams building agent evals.
More from coding & agent
- Adding Dark Mode Takes 10 Minutes With Claude — Why Doesn't Your Site Have It? — braelyn_ai · 2026-10-29
- Yacine: Coding AIs Give People What They Want, But People Don't Know What's Worth Wanting — yacineMTB · 2026-10-29
- Agentic RL 入门书单:三篇论文打通基础、前沿与全景 — rbhar90 · 2026-10-29
- Parley: open-source Go CLI lets multiple coding agents collaborate over a local IRC channel — colinsurprenant · 2026-10-29
- Ask an agent whether your flight has Starlink — United now gives per-flight odds — SamuelAlbanie · 2026-10-29
- YC F26 startup Faraday launches end-to-end AI agents for manufacturers — ycombinator · 2026-10-29