GPT-6.2 Astra tops Mercor's OpenWorldEngineer benchmark at 67.2%
morqon · x · 2026-11-18
Mercor tested OpenAI's new models before launch and published results on its new OpenWorldEngineer benchmark: GPT-6.2 Astra debuts at #1 with 67.2% mean score, ahead of Fable 5.2 (54.6%), GPT-6 Astra (44.2%), Opus 5.5 (41.6%), and GPT-6.1 Sol (41.2%).
The benchmark targets ultra long-horizon engineering: 26 tasks across software development, engineering, ML, and science, each attempt taking 4–18 hours and 141M tokens. Every task is built by a domain expert (PhD or 10+ years' experience) and requires agents to go beyond the best public methods with measurable improvements. Scores rescale the untouched baseline to 0% and the ceiling to 100%, with final evaluation on held-out cases. Astra leads all four domains (Engineering 83.7%, Software 65.9%, ML 62.9%, Science 60.4%). The benchmark is open source.
More from Models
- Leaked ChatGPT model picker uses planets to personify model variants — cneuralnetwork · 2026-11-18
- Wave of negative GPT-6.2 tweets: bad model or Anthropic PR? — dotey · 2026-11-18
- Warp CEO: we tested every model; Anthropic's are awesome but pricey — thursdai_pod · 2026-11-18
- GPT-6.2 Astra and Sol hit Router at $8/$40 and $2/$10 per million tokens — floguo · 2026-11-18
- Claude.ai and iOS app hit Apple sign-in outage; Anthropic says cause identified — ClaudeAI-mod-bot · 2026-11-18
- GPT-6.2 Astra takes top spot on OpenAI ECI despite mixed reviews, chart shows — ChrisGPT · 2026-11-18