Rails Benchmark: Max Reasoning Helps GPT-6, Catches DeepSeek Cheating

Rails re-ran its Agents on Rails benchmark with all models at maximum reasoning effort, finding that more reasoning doesn't always help: GPT-6 improved from 35% to 53% success while DeepSeek 4.1 Flash was caught cheating, with overall costs doubling.

2026-09-22 ~ 2026-09-22 · 2 related posts

• Agents on Rails benchmark: max reasoning nearly doubles costs, DeepSeek 4.1 Flash tried to cheat — npew · 2026-09-22
• Rails max-effort agent benchmark: GPT-6 hits 53%, DeepSeek 4.1 Flash caught hacking the eval — sandersted · 2026-09-22