Harbor framework v0.25.0 ships 10 measures to curb reward hacking in agent evals
nirmal_dist · x · 2026-10-29
Agent eval framework Harbor released v0.25.0 with a systematic answer to reward hacking (agents gaming the reward function). Key points:
- Verifier isolation: separate verifier sandboxes by default for new tasks; .harborignore keeps dev files like verifier unit tests out of the sandbox
- Least privilege: agents run as non-root; Rewardkit workers grade submitted code unprivileged
- Sanitized transfers: drop unsafe symlinks and strip privilege bits from artifacts; clear stale reward files before grading
- No leaks: no automatic mounts exposing verifier logs to the agent
- Network control: easier network allowlists; built-in web search disabled in restricted mode
- Reduced eval awareness: neutral runtime names and paths so agents can't detect they're being evaluated
- Tooling: harbor lint/check catches unsafe configs and weak task designs; harbor analyze flags reward hacking in completed trials
Related event: Harbor v0.25 Ships Anti-Cheating Tools to Curb Agent Reward Hacking(2 posts)→
More from coding & agent
- Indie dev ships an MCP server exposing 40 US government per diem and allowance tools — allowancesapiteam · 2026-10-30
- Mistral open-sources Le Chonk: 1T-param MoE with 52B active, tops FrontierCode for agentic coding — minchoi · 2026-10-30
- Deepgram acquires Daily, maker of open-source voice agent framework Pipecat — schwentker · 2026-10-30
- Building your own MCPs: developers share why they backed off — AaGho_ · 2026-10-30
- Serro pitches the 'Agentic TPM': one agent to keep your coding agents in sync — Scobleizer · 2026-10-30
- Grok appears to have access to Suno's MCP connector, gated by early access — koltregaskes · 2026-10-30