Mystery solved: Opus 5 vs 5.5 chart gap came from thinking levels, not regression
andrew_n_carr · x · 2026-09-23
Andrew Carr explains why the Opus 5.5 and Opus 5 Frontiercode charts seemed inconsistent: the two were measured under different conditions.
- The Opus 5.5 chart used adaptive thinking at Max, while the Opus 5 release chart used medium thinking.
- Opus 5 scores 48 at max but 53 at medium.
- Frontiercode penalizes out-of-scope/extra edits beyond what the model is told, so the thinking level directly shifts scores.
The numbers aren't a contradiction — just different test setups.
Related event: Opus 5's FrontierCode Score Drops from 53.4% to 48% in New Card(3 posts)→
More from Models
- Alnie Seed 9B model's benchmarks questioned: mismatched numbers, FP4 speed vs FP16 scores — cneuralnetwork · 2026-11-08
- Experiment: Claude commits to answers more firmly than ChatGPT on 29 of 34 science questions — AymericRoucher · 2026-11-08
- Claude Artifacts suddenly blocks writes, users asked to grant full network egress — KeepStandardVoice · 2026-11-08
- Will Qwen4 Plus be open weights? Reddit parses Alibaba's release clues — Axiomexx · 2026-11-08
- OpenAI exec floats $50/month ChatGPT Pro tier between Plus and $100 Pro — haider1 · 2026-11-08
- Users frustrated by Claude's safety blocks refusing keys, databases, repo creation — TheNasky1 · 2026-11-08