Agnes 2.5 Pro Beta Uses 50k Output Tokens; Score Rise Driven by Abstention
ArtificialAnlys · x · 2026-08-28
Agnes 2.5 Pro Beta consumes 50k output tokens per task on the Artificial Analysis Intelligence Index, more than double its predecessor Alpha (24k) and exceeding Qwen3.8 27B and GLM-5.3. While its AA-Omniscience score improved from -25 to -11, this is attributed to increased abstention rather than accuracy. It attempted only 45% of questions (vs 94% for Alpha), with accuracy dropping from 33% to 17% despite a lower hallucination rate.
More from Models
- Nathan Lambert slams Meta's Muse Open License as 'trash' — natolambert · 2026-10-16
- Google launches Gemini 4 Argon to Ultra and Pro subscribers, touting physics and 3D skills — ShikharMurty · 2026-10-16
- Google gives X users early access to test Argon, not just companies — lyraxana · 2026-10-16
- Grok Voice 3.0 Tops Design Arena Speech Leaderboard, Up 7.4 Points — XFreeze · 2026-10-16
- Robotics LLMs tested on 16 tasks: GPT-6 Astra hits 17.2% success vs Fable's 9.1% — pabbeel · 2026-10-16
- Meta open-sources Muse Spark: top US open model, scores just below DeepSeek — bindureddy · 2026-10-16