Mistral's two-open-model agent hits 82% on ArXivLean using 10M output tokens per problem
scaling01 · x · 2026-09-12
A new ArXivLean leaderboard entry shows Mistral's agent stack—Leanstral 1.5 as prover plus Kimi K3 as coordinator—scoring a massive 82%, at the cost of roughly 10 million output tokens per problem.
The poster quips that this open-source combo outperformed GPT-6-Astra at the same token budget, and wonders what GPT-6-Astra could do with 130B tokens.
Related event: 6B Open-Source Leanstral 1.5 Beats GPT-6 Astra on ArXivLean(4 posts)→
More from Models
- ToneBench eval: GPT-6.1 Terra ranks #23 at writing — cheaper, not better, than GPT-5.6 Terra — OnlyProggingForFun · 2026-10-28
- Cursor user complains it silently switches to Grok, which fails basic tasks — kylegawley · 2026-10-28
- Musk says Grok Bot keeps improving, calls Vault feature a game changer — beffjezos · 2026-10-28
- Claude refuses to gather legitimately public information, sparking guardrail backlash — mkheck · 2026-10-28
- DesignArena launches Code to Video Arena: models direct 15-45s films purely in code — yshan2u · 2026-10-28
- ChatGPT's 'yes, but with an important clarification' often precedes pure nonsense — Crixusgannicus · 2026-10-28