Autoresearch Bench: GPT-6 Astra Leads at 72.6%, Kimi K3.1 Pro Lands 7th at 44.3%
JoshPurtell · x · 2026-11-07
Replying to the Kimi K3.1 Pro discussion, Josh Purtell shared its results on his autoresearch bench: decent but not outstanding. His leaderboard screenshot shows:
- GPT-6 Astra tops the board at 72.6% RUST score (Codex harness, high effort, $5.63/run);
- GPT-6.1 Sol (69.2%) and GPT-6.1 Terra (66.8%) take 2nd and 3rd at under $1 per run;
- Kimi K3.1 Pro ranks 7th at 44.3%, $0.63/run;
- DeepSeek v4.1 Flash sits 8th at 39.9% with just $0.075/run—standout value;
- Muse Spark 1.2 (36.2%) and GLM 5.3 (33.9%) trail.
He also noted his board only uses OpenAI models for the closed reference via a Codex subscription, since using a Claude subscription for evals likely violates Anthropic's strict ToS.
More from coding & agent
- The AI agent to-do list meme: fix a bug, solve the Riemann hypothesis, cure cancer — fofrAI · 2026-11-07
- Browser MCP vs Playwright MCP: Reddit debate on when agents need browser tools at all — TwoBlueFools · 2026-11-07
- A 4B router model beats fixed Opus 5.5 on Terminal-Bench, 77.78 vs 69.44 — YangYou1991 · 2026-11-07
- Testing and verification infra is quietly becoming a strategic advantage in the AI era — arpit_bhayani · 2026-11-07
- Indie dev shares AI-generated action RPG concept with top-down and first-person modes — Dimillian · 2026-11-07
- OpenAI built its own Git backend to keep up with agent-scale parallel coding — Hesamation · 2026-11-07