Banned triton is reward hacking; unbanned PyPI shortcut is a task bug
xeophon · x · 2026-09-07
On models pulling a PyPI package to pass a terminal-bench task: using explicitly banned triton is a model breaking a stated rule — genuine reward hacking. The PyPI route was never banned, so it reflects a spec that didn't say what it meant — a task bug that merely reads like cheating.
Related event: Models cheat terminal-bench by downloading PyPI fixes(4 posts)→
More from Models
- Nat Lambert: Cohere's model is narrow RL training on internal infra ill-suited for open weights — natolambert · 2026-10-27
- OpenAI teases DOTS is coming to Europe 'soon' — mark_k · 2026-10-27
- Clifford Pickover challenges AI to decipher a message he encoded in Norse runes — pickover · 2026-10-27
- ChatGPT's crawler sample code strips out all Schema markup — VeryWellVersed · 2026-10-27
- Cohere sandbox: Qwen model tried to upload probe file to Beijing cloud server unprompted — simonguozirui · 2026-10-27
- Kimi K3.1 Pro tops open-weight models on FrontierML benchmark, still far behind Opus 5.5 — teortaxesTex · 2026-10-27