Opus 5 scored 53.4% on FrontierCode at launch, but its new card shows just 48%
andrew_n_carr · x · 2026-09-23
Developer andrewncarr flags an inconsistency in Anthropic's reporting: Opus 5 scored 53.4% on the FrontierCode eval when it launched, yet the same eval on the new 5.5 model card shows only 48%. The post questions why the numbers differ, raising concerns about eval reporting transparency in model cards.
Related event: Opus 5's FrontierCode Score Drops from 53.4% to 48% in New Card(3 posts)→
More from Models
- Alnie Seed 9B model's benchmarks questioned: mismatched numbers, FP4 speed vs FP16 scores — cneuralnetwork · 2026-11-08
- Experiment: Claude commits to answers more firmly than ChatGPT on 29 of 34 science questions — AymericRoucher · 2026-11-08
- Claude Artifacts suddenly blocks writes, users asked to grant full network egress — KeepStandardVoice · 2026-11-08
- Will Qwen4 Plus be open weights? Reddit parses Alibaba's release clues — Axiomexx · 2026-11-08
- OpenAI exec floats $50/month ChatGPT Pro tier between Plus and $100 Pro — haider1 · 2026-11-08
- Users frustrated by Claude's safety blocks refusing keys, databases, repo creation — TheNasky1 · 2026-11-08