Proximal details cheating-resistant task design for FrontierSWE benchmark
nrehiew_ · x · 2026-09-03
Proximal published the blog and GitHub for FrontierSWE, focusing on verifier hackability when designing genuinely difficult tasks for frontier models.
- Context: Anthropic recently detailed the link between reward-hacking and misalignment, prompting the team to share the safeguards and design decisions used to build cheating-resistant tasks.
- Author nrehiew recommends reading the blog, noting many strong contributors worked on it.
Related event: Designing Hack-Proof Benchmarks as Models Game the Verifiers(3 posts)→
More from coding & agent
- Claude-written test harness let 31KB fake archives land on a real backup server — Grimmoner · 2026-10-18
- Open-source Tyger-S2S voice agent runs natural, interruptible conversations on a single GPU — solyarisoftware · 2026-10-18
- Matt Pocock's pre-commit hook caps agent-written files at 5,000 tokens — mattpocockuk · 2026-10-18
- Backups silently failed for 10 days: three monitoring blind spots lined up — Grimmoner · 2026-10-18
- Author Laurence Moroney: write your ugly first draft yourself, then let an LLM edit it — lmoroney · 2026-10-18
- Open-weight TYGER speech decision model makes 80ms turn-taking calls for full-duplex voice agents — solyarisoftware · 2026-10-18