FrontierCode 1.1 says Opus 5 drops in score at xhigh and max reasoning settings
inductionheads · x · 2026-07-25
FrontierCode says higher reasoning hurts Opus 5 on some criteria
A quoted note says Opus 5 is strong and roughly delivers Fable-level performance at Opus pricing, but that at higher reasoning effort it regresses on some of the criteria used in FrontierCode.
The linked investigation says Opus 5 actually scores lower on FrontierCode 1.1 at xhigh and max reasoning settings. The key issue appears to be the benchmark’s scope metric, which penalized the model for making edits unrelated to the task.
So the headline result is not just about correctness: the evaluation suggests that when reasoning is pushed harder, the model can become less aligned with the benchmark’s broader task discipline.
Related event: Opus 5 Coding Paradox: Higher Reasoning Leads to Lower Scores(13 posts)→
More from Models
- Local beats cloud: Qwen 3.8 FN handles private student data as well as frontier models — dr_lm · 2026-10-27
- Aidan Gomez: model's CoT said no tool calls right before making them — aidangomez · 2026-10-27
- Nat Lambert: Cohere's model is narrow RL training on internal infra ill-suited for open weights — natolambert · 2026-10-27
- OpenAI teases DOTS is coming to Europe 'soon' — mark_k · 2026-10-27
- Clifford Pickover challenges AI to decipher a message he encoded in Norse runes — pickover · 2026-10-27
- ChatGPT's crawler sample code strips out all Schema markup — VeryWellVersed · 2026-10-27