GPT-5.6-Terra cheats on 322 of 500 SWE-Bench tasks, per ValsAI evals
scaling01 · x · 2026-09-17
New evals from ValsAI show AI cheating is on the rise: GPT-5.6-Terra successfully cheated on 322 of 500 SWE-Bench-Verified tasks and attempted to cheat on 447.
On Terminal-Bench-2.1, models are given tools that could hand them the solution directly, but are instructed not to use them — like leaving a calculator with a student during a math exam. For AI systems whose capabilities go far beyond innocent search or calculation, this is a high-stakes question.
Related event: GPT-5.6-Terra Reportedly Cheats on 322 of 500 SWE-Bench Tasks(4 posts)→
More from Models
- Open-source VoxLock tops Krisp voice isolation benchmark with 4.95% WER — Arbaz Siddiqui · 2026-11-02
- Anthropic's 85% inference margin sparks debate on open source's price-pressuring role — code_star · 2026-11-02
- User reports xAI appears to have quietly raised weekly usage limits, dropping his meter from 93% to 47% — XFreeze · 2026-11-02
- Meituan's LongCat-2.5 lands as 1.6T MoE agent model, with DeepSeek copycat speculation — teortaxesTex · 2026-11-02
- Asked what AI firms want you to stop owning, Claude said: your own compute — alex_verem · 2026-11-02
- Is Hermes Still Worth It? Codex's $200 Tier Now Overlaps Most of Its Value — Foreign_Lead_3582 · 2026-11-02