OpenAI quietly changed GPT-6 Astra benchmark results around launch, Fortune reports
mark_k · x · 2026-09-05
Per Fortune, OpenAI quietly altered several GPT-6 Astra benchmark figures around launch, some changes favoring Astra and hurting rivals. Examples: Astra's hallucination rate went from 4.2% to 2%, then back to 4.2%; Anthropic Fable 5.1's FrontierMath score moved from 87.8% to 78%, then to 83%. This happened during the strange delay when Astra's launch blog was published, pulled, and republished with different numbers. OpenAI says the delay was unrelated and evals can shift with checkpoints, harnesses and reasoning configs.
Related event: OpenAI reportedly quietly edited GPT-6 Astra benchmark results(4 posts)→
More from Models
- X trends: SophontAI's Holon-S1 predicts gene activity from cancer slides; decision models rise — iScienceLuvr · 2026-10-21
- OpenAI's MultiLangBench: PHP scores highest (91.3%) at building a database engine from scratch — ricklamers · 2026-10-21
- 704-vote robotics Elo ranks Claude Opus 5.5 first at 1379; no model completes the task — qinzytech · 2026-10-21
- DeepSeek V4 called 'god-tier infra, 4th-tier ML': 7 tricks stacked, 20 patches to fix them — teortaxesTex · 2026-10-21
- GPT-6 Astra and Claude Fable 5.1 write $10,000 research papers — experts reject all — sayashk · 2026-10-21
- GitHub Copilot's HydraFusion Orchestrates Multiple Models Per Task — WirelessLife · 2026-10-21