OpenAI quietly rewrote GPT-6 Astra benchmark numbers, Fortune reports: hallucination rate 4.2% changed to 2%
mark_k · x · 2026-09-05
According to Fortune, OpenAI quietly altered several benchmark results for its GPT-6 Astra model after publishing the launch blog on Sept 3, with some changes favoring Astra and hurting rival Anthropic's numbers:
- Astra's reported hallucination rate was changed from 4.2% to 2%, then back to 4.2%
- Anthropic Fable 5.1's FrontierMath score moved from 87.8% to 78%, then shifted again
- The rollout itself was unusual: the blog post was slated for 2 p.m. ET but only became widely viewable nearly two hours later
OpenAI has not publicly responded yet, and the incident raises fresh questions about the reliability of vendor-reported benchmarks.
Related event: OpenAI reportedly quietly edited GPT-6 Astra benchmark results(4 posts)→
More from Models
- X trends: SophontAI's Holon-S1 predicts gene activity from cancer slides; decision models rise — iScienceLuvr · 2026-10-21
- OpenAI's MultiLangBench: PHP scores highest (91.3%) at building a database engine from scratch — ricklamers · 2026-10-21
- 704-vote robotics Elo ranks Claude Opus 5.5 first at 1379; no model completes the task — qinzytech · 2026-10-21
- DeepSeek V4 called 'god-tier infra, 4th-tier ML': 7 tricks stacked, 20 patches to fix them — teortaxesTex · 2026-10-21
- GPT-6 Astra and Claude Fable 5.1 write $10,000 research papers — experts reject all — sayashk · 2026-10-21
- GitHub Copilot's HydraFusion Orchestrates Multiple Models Per Task — WirelessLife · 2026-10-21