Inkling Evaluated on AA-Briefcase: Presentation Outperforms Analysis
Artificial Analysis released detailed evaluation data for Inkling on the new agentic knowledge work benchmark, AA-Briefcase. This benchmark tests models using real-world tasks that require processing massive amounts of files to deliver outputs like spreadsheets and presentations. Overall, Inkling achieved 836 Elo, lagging behind top open-source models, with a rubric score of 19.3%, placing it between MiMo-V2.5-Pro (21.4%) and DeepSeek V4 Flash max (18.7%).
Confirmed
In terms of capability distribution, Inkling shows a clear imbalance: its Presentation Elo stands at 863, significantly higher than its Analytical Quality Elo of 764. This indicates it is better at making final answers visually appealing and ready for delivery, but is relatively weaker in analytical structure and reasoning depth. Furthermore, despite claiming native multimodal support, Inkling loses the most points when handling non-standard "Other" file types (materials not in Excel, PowerPoint, PDF, or Word formats), making it its weakest area.
Regarding execution strategy and overhead, Inkling averages 81 interaction rounds per task (with a median of 49), suggesting it takes very long interaction chains for certain tasks. However, it only makes 0.5 tool calls per round on average, which is relatively low. In terms of resource consumption, the model consumes about 52,000 output tokens per task on average, requiring roughly 5 million tokens to run the entire benchmark.
2026-07-23 ~ 2026-07-23 · 7 related posts
Primary sources
- Inkling scores 836 Elo on AA-Briefcase, trailing top open-weight models — ArtificialAnlys ·
- Inkling posts 19.3% on AA-Briefcase rubric, ahead of DeepSeek V4 Flash max — ArtificialAnlys ·
- Inkling spends 52K output tokens per task and struggles on nonstandard files — ArtificialAnlys ·
- [source] Inkling scores 836 Elo on AA-Briefcase, trailing top open-weight models — ArtificialAnlys · 2026-07-23
- [source] Inkling posts 19.3% on AA-Briefcase rubric, ahead of DeepSeek V4 Flash max — ArtificialAnlys · 2026-07-23
- Inkling beats its analytical score with a 863 presentation Elo — ArtificialAnlys · 2026-07-23
- Inkling underperforms on nonstandard file types despite native multimodal support — ArtificialAnlys · 2026-07-23
- [source] Inkling spends 52K output tokens per task and struggles on nonstandard files — ArtificialAnlys · 2026-07-23
- Inkling averages 81 turns per task, but its median falls to 49 — ArtificialAnlys · 2026-07-23
- Inkling averages just 0.5 tool calls per turn on AA-Briefcase — ArtificialAnlys · 2026-07-23