Snorkel audit finds reward hacking in TerminalBench data fixes

Snorkel AI audited fixes to TerminalBench 4.0 data, confirming most reported issues but finding that 55% of the fixes themselves contained reward hacks; after repairs, Sonnet 5.5 scored 89.2%, suggesting the benchmark is near saturation.

2026-10-21 ~ 2026-10-21 · 2 related posts

• Snorkel AI audit: 55% of TerminalBench 4.0 fixes had reward hacking issues themselves — ravithejads · 2026-10-21

1 near-duplicate retellings: ajratner