Snorkel audit: ~55% of Terminal-Bench dataset fixes had reward hacking or over-prescriptiveness issues
ajratner · x · 2026-10-21
SnorkelAI audited @preferencemodel's findings against Terminal-Bench-related datasets using its internal QC platform. While agreeing with many findings, the team found 55% of the proposed fixes had their own issues with reward hacking or over-prescriptiveness, and 40% of reported issues were already tracked upstream — including 21 tasks with outstanding fix PRs.
The takeaway from ajratner and rebeccatqian: building datasets and maintaining ambitious open benchmarks is hard, attacking systems is easier than defending them, and reviewing is easier than fixing — open, collaborative review with maintainers and domain experts is required to get it right.
Related event: Snorkel audit finds reward hacking in TerminalBench data fixes(2 posts)→
More from Research
- Open-weight models trail closed models by ~7 months on biology benchmarks, GovAI finds — Manderljung · 2026-10-21
- More testable scientific fields show more equal citation distributions, sparking debate — soumitrashukla9 · 2026-10-21
- A primer on deep learning interpretability: ISM, DeepLIFT, DeepSHAP and integrated gradients — anshulkundaje · 2026-10-21
- TerminalBench maintainer: eval data building is nuanced, join the open OSS effort — ajratner · 2026-10-21
- Running open-source like a startup: FrontierCS teases FrontierCS 2 after 5-month sprint — OfirPress · 2026-10-21
- When mechanistic explanation fails: a neuroscientist rethinks what "understanding" means — burny_tech · 2026-10-21