GPT-6.2 scores worse than predecessors across the board on Company Agent Bench
philhchen · x · 2026-11-19
Josh Qin's team published evals showing GPT-6.2's Astra and Sol models scored lower than their predecessors on Company Agent Bench at every reasoning effort tested.
- At high effort: Astra 6 → 6.2 dropped 80.8% → 79.7%; Sol 6.1 → 6.2 dropped 79.1% → 76.4%
- The benchmark uses 205 questions drawn from the company's real Slack
- The 6.2 versions cost less but score worse
The takeaway: the upgrade is a regression for agentic workloads; full results are published.
More from Models
- Qwen4-Max launches with Day-0 support in Qoder coding IDE — usamawahabkhan · 2026-11-19
- RoboLab-120 model runs autonomous lab experiments, beats GPT Astra by ~10 points — Scobleizer · 2026-11-19
- banteg benchmarks unreleased Gemini 4 Hydrogen and Neon, mocks Google's model lineup — banteg · 2026-11-19
- Google's Gemini 4 Neon goes GA in GitHub Copilot, tuned for agentic coding — intellectronica · 2026-11-19
- Ex-bootcamp teacher: frontier models code like talented bootcamp students — dansitu · 2026-11-19
- Gemini 4 Neon wows with Triceratops skeleton rendering — fofrAI · 2026-11-19